Getting (Re)Started with Data Stewardship

Getting (Re)Started
with Data Stewardship
© Copyright 2021 by Peter Aiken Slide # 1
paiken@plusanythingawesome.com+1.804.382.5957 Peter Aiken, PhD
Integrating the role of Data Debt fighting
Peter Aiken, Ph.D.
• I've been doing this a long time
• My work is recognized as useful
• Associate Professor of IS (vcu.edu)
• Institute for Defense Analyses (ida.org)
• DAMA International (dama.org)
• MIT CDO Society (iscdo.org)
• Anything Awesome (plusanythingawesome.com)
• Experienced w/ 500+ data
management practices worldwide
• Multi-year immersions
– US DoD (DISA/Army/Marines/DLA)
– Nokia
– Deutsche Bank
– Wells Fargo
– Walmart …
• 12 books and dozens
of articles
https://plusanythingawesome.com
+
• DAMA International President 2009-2013/2018/2020
• DAMA International Achievement Award 2001
(with Dr. E. F. "Ted" Codd
• DAMA International Community Award 2005

3
Program
© Copyright 2021 by Peter Aiken Slide #
• Why do we need data stewards?
– Definitions: Data steward/stewardship, data debit
– Architectural context (cannot just clean up)
– The role of strategy
• How are stewards supposed to
eliminate data debit while
facing compliance/regulation/initiatives?
– Resolve prerequisite challenges
– Relationship with governance/organization
– Stewardship role (in context of data governance)
– Fire station model (Reactive & proactive foci)
• When (Organizational improvement)
– Differing cadence (Need for different structural approach)
– Foundational prerequisites
– Need for simplicity, agility, practice
• Take aways ➜ Q&A
Getting (Re)Started with
Data Stewardship
Integrating the role of Data Debit fighting
Question?
• How many starting versus how many re-starting?

How old is your profession?
Augusta Ada King
Countess of Lovelace
(1815-52)
• 8,000+ years
• formalize practices
• GAAP
What do we teach knowledge workers about data?
What percentage of the deal with it daily?

What do we teach IT professionals about data?
• 1 course
– How to build a
new database
• What
impressions do IT
professionals get
from this
education?
– Data is a technical
skill that is needed
when developing
new databases
Confusion
• IT thinks data is a business problem
– "If they can connect to the server, then my job is done!"
• The business thinks IT is managing data adequately
– "Who else would be taking care of it?"

Bad Data Decisions/Illiteracy Spiral
Bad data decisions
Technical
decision makers
are not data literate
Business decision
makers are not
data literate
Poor organizational outcomes
Poor treatment of
organizational data
assets
Poor
quality
data
?

Definitions
Steward
• 1. a person who looks after the passengers on a ship, aircraft, or train and brings them meals.
– synonyms: flight attendant, cabin attendant, air hostess, purser "an air steward"
– a person responsible for supplies of food to a college, club, or other institution.
• 2. an official appointed to supervise arrangements or keep order at a large public event, for ex. sporting
event.
– synonyms: official, marshal, organizer "the race stewards"
– short for shop steward.
• 3. a person employed to manage another's property, especially a large house or estate.
– synonyms: (estate) manager, agent, overseer, custodian, caretaker; historical "the steward of the estate"
– a person whose responsibility it is to take care of something."farmers pride themselves on being stewards of the countryside"
Stewarding
• 1. (of an official) supervise arrangements or keep order at (a large public event).
"the event was organized and stewarded properly"
• 2. manage or look after (another's property).
Data Steward
• Manage data assets on behalf of all stake holders and in the best interests of the organization
(McGilvray, 2008)
• Represent the interests of all stakeholders and take an enterprise perspective
• Have dedicated time enough to be accountable and responsible
Trust
• Firm belief in the reliability, truth, ability, or strength of someone or something (google.com)
Fiduciary
• Involving trust, especially with regard to the relationship between a trustee and a beneficiary (google.com)
one who actively directs the use of
organizational data assets in support
of specific mission objectives
Steward
• one who actively directs
, Data

Data
Steward
• What do data stewards do in our organization?
- Improve the organization's data assets value, and
- Advocate/evangelize for increasing the scope/rigor of
data-centric practices
- Ensure efficient/effective data management practices
- Improve data's use achieving organizational objectives
Source: Unknown
Data
Management
Body
of
Knowledge
(DM
BoK
V2)
Practice
Areas
from The DAMA Guide to the Data Management Body of Knowledge 2E © 2017 by DAMA International

Data Debt
• The time and effort it will take to
return your data to a governed
state from its likely current state of
ungoverned
• Getting back to zero
– Involves undoing existing stuff
– Likely new skills are required
• At zero-must start from scratch
– Typically requires annual proof of value
• Now you need to get good at both
– Almost all data challenges involve
interoperability
– Little guidance at optimizing data
management practices
– Very little guidance as to how to get
back to zero
Poor data manifests as multifaceted organizational challenges

Root cause analysis is part of data stewardship
IT
System
Business
Challenge
Business
Process
Business
Challenge
IT
Process
Business
Challenge
Business
System
Business
Challenge
IT
Process
Business
Challenge
IT
System
Business
Challenge
Business
Process
Business
Challenge
Poor results
Consistency Encourages Quality Analysis
IT
System
Business
Challenge
Business
Process
Business
Challenge
IT
Process
Business
Challenge
Business
System
Business
Challenge
IT
Process
Business
Challenge
IT
System
Business
Challenge
Business
Process
Business
Challenge
Eliminating data debt
requires a team with
specialized skills
deployed to create a
repeatable process
and develop sustained
organizational
skillsets

Governance and
Architecture
Example from: https://www.slideshare.net/AnthonyDehnashi/architecture-governance
Corporate Governance
• "Corporate governance - which
can be defined narrowly as the
relationship of a company to its
shareholders or, more broadly,
as its relationship to society….",
Financial Times, 1997.
• "Corporate governance is about
promoting corporate fairness,
transparency and accountability"
James Wolfensohn, World Bank, President
Financial Times, June 1999.
• “Corporate governance deals
with the ways in which suppliers
of finance to corporations assure
themselves of getting a return on
their investment”,
The Journal of Finance, Shleifer and Vishny,
1997.
• "Corporate governance - which
can be defined narrowly as the
relationship of a company to its
shareholders or, more broadly,
as its relationship to society….",
Financial Times, 1997.
• "Corporate governance is about
promoting corporate fairness,
transparency and accountability"
James Wolfensohn, World Bank, President
Financial Times, June 1999.
• “Corporate governance deals
with the ways in which suppliers
of finance to corporations assure
themselves of getting a return on
their investment”,
The Journal of Finance, Shleifer and Vishny,
1997.

• "Putting structure around how organizations align IT strategy with
business strategy, ensuring that companies stay on track to achieve
their strategies and goals, and implementing good ways to measure
IT’s performance.
• It makes sure that all stakeholders’ interests
are taken into account and that
processes provide measurable results.
• Framework should answer some key
questions, such as how the IT department
is functioning overall, what key metrics
management needs and what return IT
is giving back to the business from the
investment it’s making." CIO Magazine (May 2007)
IT Governance Institute, 5 areas of focus:
• Strategic Alignment
• Value Delivery
• Resource Management
• Risk Management
• Performance Measures
• "Putting structure around how organizations align IT strategy with
business strategy, ensuring that companies stay on track to achieve
their strategies and goals, and implementing good ways to measure
IT’s performance.
• It makes sure that all stakeholders’ interests
are taken into account and that
processes provide measurable results.
• Framework should answer some key
questions, such as how the IT department
is functioning overall, what key metrics
management needs and what return IT
is giving back to the business from the
investment it’s making." CIO Magazine (May 2007)
IT Governance Institute, 5 areas of focus:
• Strategic Alignment
• Value Delivery
• Resource Management
• Risk Management
• Performance Measures
IT Governance

Elevator Pitch
7 Data Governance Definitions
24
• The formal orchestration of people, process, and technology to enable
an organization to leverage data as an enterprise asset – The MDM Institute
• A convergence of data quality, data management, business process
management, and risk management surrounding the handling of data in an
organization – Wikipedia
• A system of decision rights and accountabilities for information-related
processes, executed according to agreed-upon models which describe who
can take what actions with what information, and when, under what
circumstances, using what methods – Data Governance Institute
• The execution and enforcement of authority over the management of data
assets and the performance of data functions – KiK Consulting
• A quality control discipline for assessing, managing, using, improving,
monitoring, maintaining, and protecting organizational
information – IBM Data Governance Council
• Data governance is the formulation of policy to optimize,
secure, and leverage information as an enterprise asset
by aligning multiple functional objectives – Sunil Soares
• The exercise of authority and control over the
management of data assets – DM BoK

What is Data Governance?
Managing
Data
with
Guidance
Would
you
want
your
sole,
non-
depletable,
non-
degrading,
durable,
strategic
asset
managed
without
guidance?
What is Data Governance?
Managing
Data
Decisions with
Guidance
Would
you
want
your
sole,
non-
depletable,
non-
degrading,
durable,
strategic
asset
managed
without
guidance?

Architecture
• Things
– (components)
data structures
• The functions of the things
– (individually)
sources and uses of data
• How the things interact
– (as a system, towards a goal)
Efficiencies/effectiveness
Organizational
Architectures
• Amazon
– Traditional
structure
• Google
– Team of 3
• Facebook
– Do you really
have a structure?
• Microsoft
– Eliminate their
own products
• Apple
– Everything
revolves around
one individual
• Oracle
– Buys one
company after
another

Data/Information Architectures – Useful Definition
• Common vocabulary expressing
integrated requirements ensuring that
data assets are stored, arranged,
managed, and used in systems in support
of organizational strategy [Aiken 2010]
• A specific definition
– 'Understanding an architecture'
– Documented and articulated as a (digital) blueprint illustrating the
commonalities and interconnections among the
architectural components
– Ideally the understanding
is shared by systems and
humans
Understanding = Interoperability

Data Architectures: here, whether you like it or not
deviantart.com
• All organizations
have data
architectures
– Some are better
understood and
documented (and
therefore more
useful to the
organization) than
others
Typically Managed Organizational Architectures
• Process Architecture
– Arrangement of inputs -> transformations = value -> outputs
– Typical elements: Functions, activities, workflow, events, cycles, products,
procedures
• Systems Architecture
– Applications, software components, interfaces, projects
• Business Architecture
– Goals, strategies, roles, organizational structure, location(s)
• Security Architecture
– Arrangement of security controls in relation to IT Architecture
• Technical Architecture/Tarchitecture
– Relation of software capabilities/technology stack
– Typical elements: Networks, hardware, software platforms, standards/protocols
• Data/Information Architecture
– Arrangement of data assets supporting organizational strategy
– Typical elements: specifications expressed as entities, relationships, attributes,
definitions, values, vocabularies

The Role of Strategy
Example from: https://slideplayer.com/slide/5082003/
What is Strategy?
• Current use derived from military
- a pattern in a stream of decisions
[Henry Mintzberg]
A thing

Every Day
Low Price
Former Walmart Business Strategy
https://plusanythingawesome.com 35
Wayne Gretzky’s
Definition of Strategy
He skates to where he
thinks the puck will be ...
36

Strategy in Action: Napoleon faces a larger enemy
• Question?
– How do I defeat the competition when their forces
are bigger than mine?
• Answer:
– Divide
and
conquer!
– “a pattern
in a stream
of decisions”
Supply Line Metadata
(as part of a divide and conquer strategy)

First Divide
Then Conquer

Complex Strategy
• First
– Hit both armies hard
at just the right spot
• Then
– Turn right and defeat
the Prussians
• Then
– Turn left and defeat
the British
Whilesomeoneisshootingatyou!
41
Data Governance Environment
Complex Data Governance Environment
Complex
Data Governance Council
• Liaise between agency operations & CDO
• Advise CDO on technology, policy, and governance
strategies
• Administer data governance policies set by the board
• implement data sharing & analytics projects
• Review open data assets
• Report progress & compliance to the Board
Advise CDO
Data sharing and analytics
• Identify goals and objectives
• Prioritize initiatives
• Study & report
• Recommend changes to budget and code
DATA COMMISSION
LEGISLATIVE
Region representatives
EXECUTIVE
JUDICIAL
Advise Governor
TRUSTEE
execute
• Define, approve, and communicate data strategies, policies,
standards, rules, guidelines, & best practices
• Provide a governance, policy, and technology framework
• Define agency data governance responsibilities
• Encourage & facilitate data sharing
• Facilitate coordination to prevent duplication
• Coordinate policy and technology proposals and recommendations
• Administer and manage the commonwealth data trust
• Track and enforce compliance and conformance
• Oversee dissemination of open data
Executive data board
Oversee council
• Translate commonwealth goals to agency performance
targets
• provide resources
• Remove organizational obstacles
• appoint data governance council members
• Oversee the data governance council
• Oversee data sharing & analytics projects

Strategy Guides Workgroup Activities
A pattern
in a stream
of decisions
43
Strategic Approach - Master/Reference Data
• Reference
– Controls accessible data values
• Master
– Controls access to system capabilities
• Transaction
– Instances of values
Countries where we do business?
Types of accounts available?
Controlled vocabulary items
Are you a member of our premium club?
Authorizing uses/users?
Common/standard data structures
$5
Authorized
Like
Example from: Dr. Christopher Bradley of DMAdvisors–he has more, ping him at chris.bradley@dmadvisors.co.uk

Data Strategy provides focus for stewardship efforts
Note: Reducing ROT increases data leverage
Organizational Data
Data Stewards
Technologies
Process
People
Less Data ROT ->
Data Stewardship Considerations
• Why?
– Stewardship terminology is not widely known
– We do not have agreed upon definitions
– It has become a de-facto widely used, ill defined term
– Stewards work effectively with architectural components
– Strategy focuses steward leveraging activities
http://williamnava.com/philosophy-shaves-barber-21/

All
inputs
are
data
All
outputs
are
data
All
inputs
are
data
All
inputs
are
data
All
inputs
are
data
All
outputs
are
data
All
outputs
are
data
All
outputs
are
data
All
inputs
are
data
All
outputs
are
data
All
inputs
are
data
All
outputs
are
data
All
inputs
are
data
All
outputs
are
data
The Data Machine
How to determine what to manage formally?
Too much requires expensive and slow bureaucracy ← → Too little misses opportunities
Interoperability is the primary value determinant
48
Program
Data Stewardship

• Overly dependent upon:
– Human-beings
– Wetwear
– Knowledge workers
– Informal communications
– Often described
as the weakest link
Data / Information Gap
Information
Data
• Have little idea what data they have
• Do not know where it is (and)
• Do not know what their knowledge workers do with it
Put simply, organizations:
50

Data Knowledge is too little and too informal
• Data stewardship happens 'pretty well' at
the workgroup level
– Defining characteristic of a workgroup (Workgroups get work done!)
– Without guidance, what are the chances that all
workgroups are pulling toward the same objectives?
– Consider the time spent attempting informal practices
• Data chaff becomes sand in the machinery
– Preventing smooth interoperation and exchanges
– Death by 1,000 cuts that have been difficult to account for
• Organizations and individuals lack
– Knowledge
– Skills
– Maturity
Separating the Wheat from the Chaff
Is well organized data worth more?

Pre-Information Age Metadata
• Examples of information architecture achievements that
happened well before the information age:
– Page numbering
– Alphabetical order
– Table of contents
– Indexes
– Lexicons
– Maps
– Diagrams
Example from: How to make sense of any mess
by Abby Covert (2014) ISBN: 1500615994
"While we can arrange things
with the intent to communicate
certain information, we can't
actually make information. Our
users do that for us."
https://www.youtube.com/watch?v=60oD1TDzAXQ&feature=emb_logo
https://www.youtube.com/watch?v=r10Sod44rME&t=1s
https://www.youtube.com/watch?v=XD2OkDPAl6s
Remove the structure and things fall apart rapidly
• Better organized data increases in value

Separating the Wheat from the Chaff
• Data that is better organized increases in value
• Poor data management practices are costing organizations
money/time/effort
• 80% of organizational data is ROT
– Redundant
– Obsolete
– Trivial
• The question is which
data to eliminate?
– Most enterprise data is never analyzed
Data Assets Win!
• Today, data is the most powerful, yet underutilized and poorly managed
organizational asset
• Data is your
– Sole
– Non-depletable
– Non-degrading
– Durable
– Strategic
• Asset
– Data is the new oil!
– Data is the new (s)oil!
– Data is the new bacon!
• As such, data deserves:
– It's own strategy
– Attention on par with similar
organizational assets
– Professional ministration
to make up for past neglect
2020 American Airlines market value ~ $6b
AAdvantage valued between $19.5-$31.5
United market value ~ 9$b
MileagePlus ~ $22b
https://www.forbes.com/sites/advisor/2020/07/15/how-airlines-make-billions-from-monetizing-frequent-flyer-programs/?sh=66da87a614e9
Data
Assets
Financial
Assets
Real
Estate Assets
Inventory
Assets
Non-
depletable
Available for
subsequent
use
Can be
used up
Can be
used up
Non-
degrading √ √ Can degrade
over time
Can degrade
over time
Durable Non-taxed √ √
Strategic
Asset √ √ √ √
56
Asset: A resource controlled by the organization as a result of past events or
transactions and from which future economic benefits are expected to flow [Wikipedia]
Data Assets Win!

Data Governance Frameworks
• A system of ideas for guiding
analyses
• A means of organizing
project data
• Priorities for data decision
making
• A means of assessing progress
– Don’t put up walls until foundation
inspection is passed
– Put the roof on ASAP
• Make it all dependent upon
continued funding
A Framework For Stewardship
A Framework For Stewardship
from https://www.trainingjournal.com/articles/feature/stewardship

Organizational Data
Challenges
Stewardship Engine
Regulation and Policy
A Framework for Data Stewardship
Monetary
Proactive Reactive
Stewardship Activities
Address Some
Other Time
Strategic Consideration
Non-monetary
Value

Data and Duct Tape

Leadership
(data decision makers)
Domain expertise is less ← | → Domain expertise is greater
Roles more formally defined ← | → Roles less formally defined
Encounter
governed
data
more
directly
←
|
→
Encounter
governed
data
less
directly
More
time
is
dedicated
←
|
→
Less
time
is
dedicated
IT/Systems Development
Stewards
(data trustees)
Guidance
Decisions
Participants/Experts
(data subject matter experts)
Other Sources/Uses
(data makers & consumers)
IT/Systems
Development
Data/feedback
Changes
Action
R
e
s
o
u
r
c
e
s
Ideas
Data/Feedback
Components comprising the data community
Leadership
(data decision makers)
Stewards
(data trustees)
Guidance
Decisions
Participants/Experts
(data subject matter experts)
Other Sources/Uses
(data makers & consumers)
Data/feedback
Changes
Action
R
e
s
o
u
r
c
e
s
Ideas
Data/Feedback
Basic Version

≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
≈
Data Governance Role: Produce systemic organizational changes that impact
data and work practices over time
Data
Leadership
Feedback
Data
Governance
Data
Improvement
Data
Stewards
Data
Community
Participants
Data
Generators/Data
Users
Data
Things
Happen
Organizational
Things
Happen
DIPs
Data
Improves
Over
Time
Data
Improves As
A Result of
Focus
Feedback
X
$
X
$
X
$
X
$
X
$
X
$
X
$
X
$
X $ $
Data Strategy in Context
Organizational
Strategy
Data Strategy
IT Projects
Organizational Operations
Data
Governance
Data
asset support for
organizational
strategy
What the data
assets need to do to
support strategy
How well data is
supporting strategy
Operational
feedback
How IT
supports strategy
Other
aspects of
organizational
strategy

Data Governance & Data Stewards
Data Strategy
Data
Governance
What the data
assets do to support
strategy
How well the data
strategy is working
(Business Goals)
(Metadata)
Data Stewards
What is the
most effective
use of steward
investments?
(Metadata)
Progress,
plans,
problems
Getting Started with Data Stewardship
How?
• Transform tribal knowledge-based processes to data asset
leveraging
• Understand stewards transform governance into by strategy
focused action
• Apply a framework to your tasks
• Understand and get good at both reactive and proactive activities
• Attempt to incorporate leadership outside of traditional channels
• Know that you cannot accomplish everything
https://hatrabbits.com/en/how-how-diagram/

Keep the proper focus
• Wrong question:
– How should we mange this data?
• Right question:
– Should we include this
data item within the
scope of our current
stewardship
practices?
• Either way document why!
70
Program
Data Stewardship

Data Footprints
• SQL Server
– 47,000,000,000,000 bytes
– Largest table 34 billion records 3.5 TBs
• Informix
– 1,800,000,000 queries/day
– 65,000,000 tables / 517,000 databases
• Teradata
– 117 billion records
– 23 TBs for one table
• DB2
– 29,838,518,078 daily queries
Standard data
Data supply
Data literacy
Making a Better Data Governance Sandwich
Data literacy
Standard data
Data supply
72

Making a Better Data Governance Sandwich
Standard data
Data supply
Data literacy
73
Making a Better Data Sandwich
Standard data
Data supply
Data literacy
This cannot happen without data engineering and architecture!
Quality data engineering/
architecture work products
do not happen accidentally!
74

• Durable asset
- An asset that has a usable
life more than one year
• Reasonable project
deliverables
- 90 day increments
- Data evolution is measured in years
• Data
- Evolves - it is not created
- Significantly more stable
• Readymade data architectural components
- Prerequisite to agile development
• Only alternative is to create additional data siloes!
Data is not a Project
Differences between Programs and Projects
• Programs are Ongoing, Projects End
– Managing a program involves long term strategic planning and
continuous process improvement is not required of a project
• Programs are Tied to the Financial Calendar
– Program managers are often responsible for delivering
results tied to the organization's financial calendar
• Program Management is Governance Intensive
– Programs are governed by a senior board that provides direction,
oversight, and control while projects tend to be less governance-intensive
• Programs Have Greater Scope of Financial Management
– Projects typically have a straight-forward budget and project financial management
is focused on spending to budget while program planning, management and control
is significantly more complex
• Program Change Management is an Executive Leadership Capability
– Projects employ a formal change management process while at the program level,
change management requires executive leadership skills and program change is
driven more by an organization's strategy and is subject to market conditions and
changing business goals
Adapted from http://top.idownloadnew.com/program_vs_project/ and http://management.simplicable.com/management/new/program-management-vs-project-management
76
Your data program
must last at least as
long as your HR
program!

“Your Organization is
all about Data,
until it’s not about just Data”
What Business
are you in?
• A management paradigm that views any
manageable system as being limited in
achieving more of its goals by a small
number of constraints
• There is always at least one constraint, and
TOC uses a focusing process to identify the constraint and
restructure the rest of the organization to address it
• TOC adopts the common idiom "a
chain is no stronger than its weakest
link," processes, organizations, etc.,
are vulnerable because the weakest
component can damage or break
them or at least adversely affect the
outcome
https://plusanythingawesome.com https://en.wikipedia.org/wiki/Theory_of_constraints
(TOC)

Organizational Data Usage Practices
Data
Management
Practices
Duplicated but ETLed Data
(quality & transformations applied)
"Warehoused" Data
Learning/
Feedback
Marts
Analytics Practices
T1
Organizations
without
a formalized
data stewards
T3
Data Steward: Use data
to create strategic
opportunities
T4
Data Steward: both
Improve Operations
Innovation
The focus of data stewards should be sequenced
Only 1 is 10 organizations has a board
approved data strategy!
T2
Data Steward: Increase
organizational efficiencies/
effectiveness
X
X

Data Strategy in Context – THIS IS WRONG!
Organizational Strategy
IT Strategy
Data Strategy
x 81
Organizational Strategy
IT Strategy
This is correct …
Data Strategy

KISS (Keep it simple, stupid!)
Data –
Driven?
Centric?
Focused?
First?
…
the Data Doctrine® (V2)
https://plusanythingawesome.com Source: theagiledoctrine.org
We are uncovering better ways of developing
IT systems by doing it and helping others do it.
Through this work we have come to value:
data programs driving IT programs
informed information investing over technology acquisition activities
stable, shared organizational data over IT component evolution
data reuse over the acquisition of new data sources

the Data Doctrine® (V2)
We are uncovering better ways of developing
IT systems by doing it and helping others do it.
Through this work we have come to value:
data programs driving IT programs
informed information investing over technology acquisition activities
stable, shared organizational data over IT component evolution
data reuse over the acquisition of new data sources
That is, while there is value in the items on
the right, we value the items on the left more.
Source: theagiledoctrine.org
Data Decisions Variety
https://www.healthcatalyst.com/why-are-data-stewards-so-important-for-healthcare
Data Source Steward

Data Steward
• Business data steward
– Manage from the perspective of business elements (i.e. business definitions
and data quality)
• Technical data steward
– Focus on the use of data by systems and models (i.e. code operation)
• Project data steward
– Gather definitions, quality rules and issues for referral to business/technical stewards
• Domain data steward
– Manage data/metadata required across multiple business areas (i.e. customer data)
• Operational data steward
– Directly input data or instruct those who do; aid business
stewards identifying root cause and addressing issues
• Metadata Data Steward
– Manage metadata as an asset
• Legacy Data Steward
– Manage legacy data as an asset
• Data steward auditor
– Ensures compliance with data guidance
• Data steward manager
– Planning, organizing, leading and controlling
(list adapted from Plotkin, 2014)
Strategy Example 1
Good Guys
(Us)
Bad Guys
(Them)
88

Strategy Example 2
Good Guys
(Us)
Bad Guys
(Them)
89
Strategy Example 3
Good Guys
(Us)
Bad Guys
(Them)
90

– Definitions: Data steward/stewardship, data debt
eliminate data debt while
Program
Data Stewardship
Integrating the role of Data Debt fighting
Take Aways
• Since stewards own the results,
they control the remediation process
• Need for professional data stewards is
increasing
– Increase in data volume
– Lack of practice improvement
• Data stewardship is a relatively new discipline
– Must conform to organizational constraints
– No one best way
• Data stewards must be driven by a data
strategy complimenting organizational
strategy
• Data stewards direct data management
efforts
• The language of data stewards is metadata
• Process improvement can improve data
stewardship and
organizational practices

• Data Governance: How to
Design, Deploy, and Sustain
an Effective Data
Governance Program
• John Ladley
• Amazon Best Sellers Rank:
#641,937 in Books (See Top
100 in Books)
– #242 in Management Information
Systems
– #209 in Library Management
– #380 in Database Storage & Design
Upcoming Events
Approaching Data Quality–Engineering Success Stories
14 September 2021
Essential Metadata Strategies
12 October 2021
Necessary Prerequisites to Data Success:
Exorcising the Seven Deadly Data Sins
9 November 2021
Brought to you by:
Time: 19:00 UTC (2:00 PM NYC) | Presented by: Peter Aiken, PhD
Note: In this .pdf, clicking any webinar title opens the registration link

Event Pricing
• 20% off
directly from the publisher on
select titles
• My Book Store @
http://plusanythingawesome.com
• Enter the code
"anythingawesome" at the
Technics bookstore checkout
where it says to
"Apply Coupon"
peter@plusanythingawesome.com +1.804.382.5957
Questions?
Thank You!
+ =

8/10/21, 3:31 PM
You Probably Don't Need a Data Dictionary - Locally Optimistic
Page 1 of 9
https://locallyoptimistic.com/post/data_dictionaries/
Home About Getting Involved LO Events
In Tools Tags Analytics, Business Intelligence, Data Dictionary, Data Engineering June 17, 2019 Michael Kaminsky & Alex J
You Probably Don’t Need a Data Dictionary
While eﬀorts to build a data dictionary are often undertaken out of a zeal for documentation
applaud, in practice data dictionaries and data catalogs end up being a large maintenance
value, and tend to very quickly become out of date.
Instead of investing in building out traditional data dictionaries, we recommend a few diﬀere

8/10/21, 3:31 PM
Page 2 of 9
achieving the same goals in ways that are less burdensome to maintain and better serve th
well.
What is a data dictionary?
In this post we’ll use the term “data dictionary” to describe a range of products that include
and “data catalogs”. In general, we’re referring to documents that contain metadata about
warehouse – most commonly, they have one section per table with one row per column inc
column-name, the column-type, any relevant foreign-keys, and a brief description of the co
will be additional metadata about the source of the table (a source system or potentially a p
A screenshot from an unhelpful data dictionary.
Why are data dictionaries created?
Listen. There’s no activity the two of us like better for enjoying a beautiful spring Saturday t
desk, cracking open a cold Kombucha, and writing some damn documentation. While we d
desire to document-all-the-things, we also believe that it is important to find the right docum
the goals that you want to achieve. People tend to embark on the data dictionary journey w
vague goals in mind. They want to 1) enable more data exploration, 2) codify a single sourc
definitions, and 3) document the provenance of particular pieces of business logic.

8/10/21, 3:31 PM
Page 3 of 9
These are all great goals! However, we do not believe that data dictionaries are the best too
these objectives. We will talk a bit about why we do not believe that data dictionaries, as tra
work very well before diving into alternative ways to achieve those goals.
What goes wrong with data dictionaries?
With your traditional data dictionary, you start small and simple. Maybe just documenting th
tables, or the definitions and business logic behind your top three KPIs.
But new use cases are always coming up, new ways of looking at data and measuring succ
to standardize) – and so the list grows.
If your KPIs aren’t just pulling from a single table or a single system, now you need to start
diﬀerent sources join together in a wild mess of foreign key relationships. Maybe you even c
pictures speak a thousand words after all.
And everything is great…until you run into the issue of diﬀerent audiences:
Non-technical business users: Just want the data, or at most, a high-level summary
they’re important.
Business analysts: Want the data, but also the business logic behind it, and probably
populate the source and/or the relevant context.
Data scientists / Engineers: Just want the column-level metadata, relationships, and
raw data correctly – “I can write my own SQL/Python, thanks.”
Which means that realistically, to solve everyone’s use case, you’ll need to write three versi
one, very large version) for any given piece of data. Not fun.
But even if you’re willing to go down that path, eventually something will change. Maybe yo
Conversion just shifted a little bit (as we all know, business logic can be extremely slippery)

8/10/21, 3:31 PM
Page 4 of 9
have to update not only your data model and dashboards + reports, but also 3 documents
The maintenance burden very quickly grows into a scalability issue, and a serious bloc
iteration speed of the entire data team. Not only does maintaining documentation slow you
documentation starts to slip (as it inevitably will) the consumers of the data dictionary will b
you will be investing time in maintaining this document that people do not even trust (and th
As a result, the heroes who start down the path of the perfect data dictionary often end up
1. Is hard to maintain
2. Is hard to use, and doesn’t answer everyone’s questions
3. Is generally out-of-date (in one way or another)
4. Ends up getting deprecated
Fortunately, we have better ways of achieving the same goals with less maintenance burde
What are the other options?
We believe the best way to approach the problem is by using the right tools for the right job
solutions for users at each level of the data literacy curve.
For the non-technical business user:
They’re really looking for an actionable, easy-to-use, single source of truth. Data – signed, s
our opinion, the best solution involves showing rather than telling: a well-architected and w
general, we believe that non-technical business users should not be writing SQL and theref
dictionary as traditionally constituted.
We do not want to go into a full-fledged comparison between BI platforms here, but whatev
here are the key features you’ll want to keep an eye out for:

8/10/21, 3:31 PM
Page 5 of 9
1. Single Source of Truth
Data used to drive business decisions comes from a single system/platform
Business logic is centrally defined, curated, and easily reproducible
KPIs are clearly defined, well-organized, and shared across the organization
2. Ad-hoc Exploration
High-level KPIs are rarely actionable – so allowing users to take the next step in s
discovering, and drilling down into the data is critical
3. Right Level of Abstraction
Naively exposing every piece of data will create a diﬃcult, confusing, and intimida
Develop a curated, properly labeled, easy-to-navigate dataset (with complex joins
abstracted away from the end user)
4. Balance between governance and democratization
Do you allow people to save their own reports/dashboards? Define their own dim
transformations?
5. Self Documenting Code
In a good BI tool, it should be easy for business users to find existing reports/dashboards t
modify (or leverage as a starting point for additional exploration). The tool should provide g
avoid making common mistakes, and provide enough metadata (either through tool-tips or
business users both find what they are looking for and avoid common pitfalls. The diﬀerenc
and a data dictionary is that the tool tips live with the data as they are actually used by bus
the post-business-logic description of the entity which is generally much more useful to end
If your BI tool has these qualities, then your business users will not need (or wouldn’t get an
dictionary. Since the BI tool should help business users answer business questions, your bu
have to know anything about the underlying data structures in order to achieve their goals.
users asking for a data dictionary, you should be thinking about building them better tools r
documenting the current suboptimal system.
Depending on the size of your team, the structure of your organization, and the complexity

8/10/21, 3:31 PM
Page 6 of 9
may need to augment your BI tool with some additional tooling in the form of a wiki or know
below). For your non-technical users this is best used as a reference for “who is the expert”
to if they want to get started in a new domain area.
For the business / data analyst: For the people in your organization who are rolling up the
the data, there should be two sources of documentation that are most relevant:
A self-documenting and searchable code-base
A knowledge-base or wiki of high-level descriptions
The first place an analyst should look when trying to understand a table or column is in the
and manages that table or column. If, for example, you are using Looker as your BI tool on
analysts should be able to quickly and easily search those code bases to find the correct se
code base is well-architected and you follow best practices for programming style and read
straightforward for the analyst to review and comprehend the relative logic. If your analyst c
comprehend the logic, you should consider refactoring your code base or implementing mo
practices.
For the types of knowledge that are not easy to see in the code itself, it can be helpful to ha
that people can review for high-level information about a given concept. This documentatio
catalog of “what is in the data warehouse” but rather should include information about the
behind broad concepts in the business:
What is (NPS, ARR, Conversion, insert your concept here), how do we define it, how is
business logic), why is it important, and what are the next steps?
Important context and history (“this is the story of why it takes 5 joins to connect our h
coupon code system”).
Any potential “gotchas” (“we know that this logic is incorrect for some number of user
checked it impacted less than 1% of users and so we ignored it”).
Between these two sources of information, analysts should have everything they need to un

8/10/21, 3:31 PM
Page 7 of 9
of the data they have.
For the data scientist / engineer:
For the most technical folks on your team, you can add some additional tooling that can he
that will be most useful to them. In addition to the self-documenting code-base (which is
concept in this blog post), you can think about adding:
Knowledge-repo: This tool is used to keep track of analyses that have happened in the
resource for both analysts and data scientists to see 1) example uses of certain data s
are related to the topic they are working on. Seeing how someone else used a given ta
for someone to understand its intricacies.
Programmatically-generated ERDs: rather than maintaining these types of documenta
programmatically-generated ERDs or data-heritage DAGs (from a tool like dbt or airflo
ensure that your documentation actually matches your code.
Architecture Design Documentation: this is often an artifact of the development proces
depth description of requirements, technical decisions made, architecture, etc.
Other important things to keep in mind:
You need a strong data organization to maintain these diﬀerent avenues of documenta
You need a data-driven culture of cross-functional collaboration to make all of the piec
What are data dictionaries useful for?
There are two use-cases where having something like a data dictionary can be valuable:
1. If you are a data-as-a-service (DaaS) provider, then your data dictionary is eﬀectively d
will be critical to your clients
2. For compliance / security reasons, you have complex restrictions on who can access

8/10/21, 3:31 PM
Page 8 of 9
!
warehouse
These two use-cases are pretty niche, and we do not believe they apply to most internal-fa
second use-case is a bit tricky because what you want to build is not a “data dictionary” as
“helpful” metadata about tables and columns) but rather an access-spec which can hierarc
diﬀerent parts of a warehouse (at the schema, table, column, and row level). We have some
build an eﬀective tool for this, so if you’re working on something like that, please reach out!
Conclusion
Overall, we think that data dictionaries tend not to be very useful, and a data team’s time w
maintaining a well-organized code-base that is self-documenting and providing well-design
that do not require much documentation. Where possible, teams should make use of tools
generating documentation directly from code in order to prevent concept drift.
Michael Kaminsky & Alex Jia
Check out the Author pages for Alex and Michael
Related Posts:
" # $
Previous Post:
KPI Principles
%
%

8/10/21, 3:31 PM
Page 9 of 9
The Attribution Dilemma Adding Context to Your Analysis with
Annotations
The Data S

Getting (Re)Started with Data Stewardship

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Getting (Re)Started with Data Stewardship

Ähnlich wie Getting (Re)Started with Data Stewardship (20)

Mehr von DATAVERSITY

Mehr von DATAVERSITY (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Getting (Re)Started with Data Stewardship