Real-World Data Governance: Metadata & Data Governance

Real-World Data Governance
Metadata & Data Governance
Monthly Webinar Series Hosted by Dataversity
Robert S. Seiner – KIK Consulting / TDAN.com
July 19, 2012 – 2:00 p.m. EST

Robert S. Seiner
KIK Consulting & Educational Services – KIKconsulting.com
The Data Administration Newsletter – TDAN.com

Twitter at #RWDG 1

Non-Invasive Data Governance™ is a trademark of Robert S. Seiner & KIK Consulting
Twitter About This Webinar at #RWDG
Copyright © 2012 Robert S. Seiner – KIK Consulting & Educational Services / TDAN.com

Real-World Data Governance
Upcoming Webinars

• Real-World Data Governance – Monthly Webinar Series
– August 16, 2012 – Master Data Management & Data Governance – 2pm EST
– September 20, 2012 – Business Glossaries & Data Governance – 2pm EST
– October to December to be announced shortly.
– Register On-Line at Dataversity.Net

Twitter at #RWDG 2


Abstract

• The relationship between Metadata and Data Governance can be
considered a two-way street.

• Metadata, as data about the people associated with the data, is a natural
by-product of an effective Data Governance program.

• And metadata itself must be governed in order for that metadata to add
value to business and technical opportunities.

Twitter at #RWDG 3


Abstract

• How can we leverage this relationship to quickly and efficiently improve
the management of data?

• How can we enable our organization to view Metadata and Data
Governance side-by-side as complimentary teammates in the effort to
manage data as a valued corporate asset?

• The answers to these questions
may not be as difficult as you think …

Twitter at #RWDG 4


Don’t Let This Happen to You

Typical
2-Way Street?

Typical
Accident?

Typical
Reaction?

Twitter at #RWDG 5


Considerations for Setting Expectations

• Top Avoidable Mistakes (in no specific order) :
– Lack of Executive & Strategic Level Understanding & Support
– Lack of Program Accountability & Responsibility
– Lack of Definition of Best Practices as a Target Behavior
– Lack of Critical Analysis of Present Environment
– Inability to Select & Follow Suitable Approach
– Lack of Planning and Ability to Support Meaningful Initiatives
– Framework of Roles & Responsibilities
– Making Data Governance & Data Quality Optional
– Lack of Focus on Communications
– Improper Expectations of the Data Quality & Data Governance Solution
– Failure to Stay Iterative and Incremental in the Approach

Twitter at #RWDG 6


The Relationship – One Way

Metadata
Governance
Governance must be applied to meta-data
as the discipline of managing meta-data.

DATA GOVERNANCE & META-DATA
STEWARDSHIP MANAGEMENT

Metadata is a by-product and an enabler
to improved Data Governance.

Governing Through
Metadata

Twitter at #RWDG 7

Copyright © 2012 Robert S. Seiner – KIK Consulting & Educational Services / TDAN.com 7

Metadata as a By Product

• Governance – Execution of Authority, Responsibility & Accountability
• Stewardship – Formalizing Accountability
– Formal – Recorded
– Informal – Not Recorded – “People Just Know” – or do they?
• Identifying Data Stewards rather than Assigning Data Stewards
• “Writing it down” rather than “Best Guess”.
Formal rather than Informal.
• Key Component of a Non-Invasive Data Governance™ Program
• Why is it important to “write it down”?

Twitter at #RWDG 8



• Tribal Knowledge Vs. Formal Knowledge
• Identify & Record the People that Have Responsibility for:
– Defining the Data/Metadata Is Necessary to operate the Business
• Are they associated with a specific area of the organization?
• Is there ever more than one person who has the responsibility?
• How do you know who to contact with questions about:
– Origin
– Validity
– Location
– Accessibility, …
• Just because you know who to contact, do other people?
– Producing the Data/Metadata Is Necessary to operate the Business
– Using the Data/Metadata that is Necessary to operate the Business

Twitter at #RWDG 9



• Ask yourself these questions:
– Could these people be the Data/Metadata Stewards?
– How could we benefit by knowing who these people are?
– Does it make sense to record who these people are?
– Is this information about the Stewards meta-data?
• This information is a by-product and results of the
Stewardship aspect of Non-Invasive Data Governance™.

Twitter at #RWDG 10



• “Having this information has greatly improved our
ability to communicate efficiently & effectively.”
Client 2008

• “Non-Invasive is the way to go …
However, let me warn you …
At some point we just may need the stick.”
Client 2007

• “How many stewards will we need and
how long will we need them for?”
Client 2006

Twitter at #RWDG 11


Data About People in Data/Metadata Governance Roles

• Further Description of Data/Metadata Governance Metadata
– Conceptual Data/Metadata Governance Metadata Meta-Models
– Selecting the “Right” Metadata to Manage
• Where do we store the Data/Metadata Governance Metadata
• Data/Metadata Governance Metadata Tools
– Common Data Matrix
– Data Steward Repository

Twitter at #RWDG 12


Business Unit Code (PK) Project Code (PK)
Business Unit Business Unit Name Project Name
Project Code IMS-IMS
Code
WAFR-WAFR
CHAZ-CHAZ
Person Id (PFK)
Position Code (PFK)
Business Unit Code (PFK)
Person Id (FK) Data Name (PFK) Data Name (PK)
Position Code (FK) Data Level Type Code (PFK) Data Level Type Code (PFK)
Business Unit Code (FK) Steward Type Code (PFK) Subject Matter Code (FK)

Position Link Steward Link Data Level
Person Data
(Relationship) (Relationship) Code

Person Id (PK) Data Level Code (PK)
Person Name Data Level Desc
Contact Info DM-Data Model
Include “Unassigned” ENT-Entity
ATR-Attribute
DB-Database
TBL-Table
Position Steward Subject COL-Column
Code Type Code Matter

Position Code (PK) Steward Type Code (PK) Subject Matter Code (PK)
Position Name Steward Type Desc Subject Matter Name
Position Grade DD-Data Definer
DP-Data Producer
DU-Data User
SS-System Steward

ISDM ISDM
Task Integration

ISDM Task Number (PK) ISDM Task Number (PK)
ISDM Task Desc Stewards Type Code (PFK)
Twitter at #RWDG 13


ORGANIZATIONS

DOMAINS

STAKEHOLDERS

Twitter at #RWDG 14



Enterprise
Perspective

Twitter at #RWDG 15


Roles & Responsibilities
Non-Invasive Data Governance™ Operating Model
of Roles & Responsibilities

Data Governance Program Team Executive Level
Data Governance Team Manager Data Steering Committee
Responsible for Administering the Program, Senior-most level knowledge of the program.
facilitating use of the Data Governance Council, Leverage existing business structure as possible.
communicating program components and value to the Organization. If no structure exists, utilize an IT Steering Committee
Executive
Advisory assistance from other levels. or an executive founded board or council.

IT Subject Resource Experts Strategic Level
System/Data Resource Experts Data Governance Council
IT Staff including Application Data Governance Sponsors Group
Development, Data Design, or Group of similar participation. One person, plus
Security, and other Data alternate, for each Business Unit represented & IT.
Strategic -
Resource Management. Consider financial services & human resources.
Enterprise

Tactical Level
T)

Data Domain Stewards
GP
(D

Per assigned Subject Areas
ts
m

Data Steward Coordinators
r
pe
a
Te

Ex

One per Business Unit
m

e
ra

rc
og

ou
Pr

es

Tactical Level - Cross Business Unit
ce

tR
an

ec

Operational Level
rn

bj
ve

Su

Operational Data Stewards (existing)
Go

IT

Es
Data Definers, Producers, Users
ta

c
Da

ala
These people are presently defining,

tio

Co
producing and using data as part of their

n

m
jobs. Recording of the Operational Data

/A

m
un
pp
Stewards will be an important enabler of

i ca
ro
improved communications, coordination

va

tio
lP
and cooperation among stewards.

n
at
h
Operational Level - Business Unit Specific

Twitter at #RWDG 16


Metadata as a By-Product of a Data Governance Program

• Where Do We Start
– Define Data Domains
• Customer Data
• Product Data
• Vendor Data
• Material Data
• Financial Data
• Human Resource Data
• Transaction Data
• Research Data
• Shared Data
• Note that this is a sample list that of what companies use.
• Beyond this they break the domains into “aspects of customer”,
“aspects of product”, … very particular to each company.

Twitter at #RWDG 17



– Define Metadata Domains
• Data Definition Metadata
• Data Production Metadata
• Data Usage Metadata
• Privacy, Security, Compliance Metadata
• Business Rule Metadata
• Hardware Metadata
• Software Metadata
• Technology Metadata
• Vendor Metadata
• Note that this is a sample list that of what companies use.
• Beyond this they break the domains into “aspects of definition meta-data”,
“aspects of production meta-data”, … very particular to each company.

Twitter at #RWDG 18



– Define Organization Units
• The Organizational Units can be pulled from an
organizational chart or they can be determined
by Company, Division, Department, Team, …
• If the Organizational Units are defined by the
Company it is often left to the Company to
determine how granular they want to get with
their units.
• It is not uncommon for organizational units to use different levels (some at
the departmental level and some at the division level).
• Once the definition of units have been completed, the most senior level
manager of that lowest granularity group identifies a logical person to help
manage the stewards in their group.

Twitter at #RWDG 19



• How Do We Populate the Stakeholder of the CDM?
– Naturally! That is the Non-Invasive way!
• By the Metadata Steward Coordinator
• Through project management
• Through recall of past projects
• Through existing committees & working groups

Twitter at #RWDG 20



• How Do We Populate the Stakeholder of the CDM?
– Metadata Steward Coordinator
• The Metadata Steward Coordinator often
has the responsibility of identifying the
operational meta-data stewards for their
business unit by meta-data domain.
• This may require the coordinator to
research and inventory the data that is
being defined, produced and used in their business unit.
• This requires that the domains of data are defined and documented to the
level of detail that will enable the Data Steward Coordinator to complete this
task.

Twitter at #RWDG 21


Applying Governance to Metadata

• First Order of Business:
– Identify Data Stewards and Record Who They Are
– Identify When and How They Should Be Involved
– Follow Governed Process

Twitter at #RWDG 22



• Pro-Active Data Governance
– Build application of stewards into existing project management methodology

• Reactive Data Governance
– Build application of stewards into data quality methodology
– Build application of stewards into issue resolution process

• Triggers
– Using Stewards & Recorded Steward Metadata to Demonstrate Communications
Improvement

• Escalation & Resolution
– Using Stewards & Recorded Steward Metadata to Get … the “Right” People Involved in
the “Right” Decisions at the “Right” Time.

Twitter at #RWDG 23



• Pro-Active Data Governance – Build Into Methodology
DATA GOVERNANCE ACTIVITY MATRIX

DATA IT DATA DOMAIN / LOB/FUNCTIONAL AREA
DEVELOPMENT
OPERATIONAL DATA STEWARDS
LIFE CYCLE DATA DOMAIN DATA STEWARD
SYSTEM SMEs DATA SMEs
(DDLC) STEWARD COORDINATOR DATA
DATA DEFINER DATA USER
(or designee) PRODUCER

Coordinate and manage all Coordinate and manage all Activity initiated re: this domain, Activity initiated re: this domain, Participate in data Participate in data Participate in data
INFORMATION technical aspects of system technical aspects of data Data Domain Steward is involved Steward Coordinator is involved in definition information production usage information
GATHERING information gathering with Data information gathering with Data in identifying and recording info identifying Operational Stewards gathering as needed. information gathering gathering as needed.
Domain Stewards and Steward Domain Stewards and Steward related Domains, Systems, in this LOB/FA that should as needed.
(DOMAIN, SYSTEM, Coordinators. Assure appropriate Coordinators. Assure appropriate Stewards per issue or task. participate in this work flow.
STEWARDS) technical meta-data is recorded. technical meta-data is recorded. Assure appropriate business
meta-data is recorded.
Coordinate and manage all Coordinate and manage all Coordinate tasks associated Coordinate activities of Provide definition Provide production Produce user
technical aspects of system technical aspects of data conducting the assessment and Operational Stewards including requirements & requirements per requirements. Use
ASSESSMENT & assessment & requirements assessment & requirements requirements steps. project timelines, status reporting, gathering document. definition. Produce data needed to
gathering with Data Domain gathering with Data Domain tactical decision making. Assure Define data needed to data needed to operate business &
REQUIREMENTS
Stewards and Steward Stewards and Steward appropriate business meta-data is operate business & operate business & report results.
Coordinators. Coordinators. recorded. report results. report results.

Coordinate and manage all Coordinate and manage all Coordinate tasks associated Coordinate activities of Provide data Provide data Provide data usage
technical aspects of system technical aspects of data planning conducting planning and analysis Operational Stewards including definition guidance to production guidance guidance to build the
PLANNING & planning and analysis with Data and analysis with Data Domain steps. project timelines, status reporting, build the appropriate to build the appropriate solution.
Domain Stewards and Steward Stewards and Steward tactical decision making. solution. appropriate solution.
ANALYSIS Coordinators. Coordinators.

Coordinate system design with Coordinate data design with Data Coordinate tasks associated Coordinate activities of Propose alternatives Propose alternatives Propose alternatives
Data Domain Stewards and Domain Stewards and Steward conducting planning and analysis Operational Stewards including for data definition for data production for data usage
Steward Coordinators. Coordinators. steps. Accept or Resolve solution timelines, reporting. Propose solution that best fits that best fits with the solution that best fits
SOLUTION DESIGN from the Steward Coordinator. solution to Data Domain Steward. with defined data definition with defined
Assure appropriate business requirements. solution and defined requirements.
meta-data is recorded. requirements.

Coordinate and manage all Coordinate and manage all Coordinate activities of all Provide input to the
technical aspects of system technical aspects of data solution individuals related to development development of the
SOLUTION solution development. Assure development. Assure that work including timelines, status, data production
that appropriate technical meta- appropriate technical meta-data is tactical decision making. solution.
DEVELOPMENT
data is recorded. recorded.

Perform EIS system testing. Perform EIS data testing. Assure Coordinate business tasks Coordinate activities of the Test data solution Test data solution Test data solution
Assure that appropriate technical that appropriate technical meta- associated with solution testing Operational Stewards related to from a definition from a production from a usage
meta-data is recorded. data is recorded. steps. testing work including timelines, perspective. Assure perspective. Assure perspective. Assure
SOLUTION TESTING status, tactical decision making. that appropriate that appropriate that appropriate
business meta-data is business meta-data is business meta-data is
recorded. recorded. recorded.

Coordinate and manage all Coordinate and manage all Coordinate business tasks Coordinate activities of the
technical aspects of system technical aspects of data solution associated with solution Operational Stewards related to
SOLUTION solution implementation. implementation. implementation. solution implementation including
timelines, status, tactical decision
IMPLEMENTATION
making.

Coordinate and manage all Coordinate and manage all Coordinate business tasks Coordinate activities of the As part of daily job, As part of daily job, As part of daily job,
technical aspects of system technical aspects of data solution associated with solution Operational Stewards related to identify operation identify operation identify operation
SOLUTION
solution monitoring & monitoring & maintenance. monitoring & maintenance. solution monitoring & data issues pertaining data issues pertaining data issues pertaining
MONITORING & maintenance. maintenance including timelines, to their domain of to their domain of to their domain of
MAINTENANCE status, tactical decision making. data. data. data.

Twitter at #RWDG 24



• Reactive Data Governance – Build Into Response
• Data Quality Methodology
– Qualify & Prioritize Data Issue
– Identify Affected Data Domain
– Identify Affected Data Stewards
– Conduct Data Systems & Data Resource Discovery
– Conduct Root Cause Analysis
– Conduct Cost-Benefit Analysis
– Analyze & Recommend Resolution
– Gain Approval, Funding, Resources
– Resolve Data Issue
• Cross Reference DQ Methodology & Resources
– Who does What and When?

Twitter at #RWDG 25


The Relationship – The Other Way

Metadata
Governance
Governance must be applied to meta-data
as the discipline of managing meta-data.

DATA GOVERNANCE & META-DATA
STEWARDSHIP MANAGEMENT

Metadata is a by-product and an enabler
to improved Data Governance.

Governing Through
Metadata

Twitter at #RWDG 26


Governing Business & Technology Metadata

• Traditional Metadata Classifications:
– Data Definition Metadata
• Data Modeling Tools
• Database Catalogs
• Data Dictionaries
• Data Standards
• Data Quality Tools
• Business Rules
– Compliance Rules
– Classification Rules
– Privacy Rules
– Security Rules

Twitter at #RWDG 27



• Traditional Metadata Classifications:
– Data Production Metadata
• Data Mappings Spreadsheets
• Extract Transformation Load Tools
– Data Usage Metadata
• Reporting Tools
• Scheduling Tools
• Report Catalog Tools
• Ad-Hoc Reporting Tools

Twitter at #RWDG 28


IT ASSET MANAGEMENT
META-DATA FRAMEWORK
There are IT ASSETS Need
a lot of process
DATA GOVERNANCE FRAMEWORK
different System Asset Data Non-System Asset Data to manage
types of Applications
Jobs/Procs
Mainframe Hardware
Mainframe Software the meta-
Programs/Scripts Networks
meta-data. Copybooks, ... SYSTEM NON-SYSTEM Printers
Telephone data.

Database DATA DATA PC Hardware/Software
Table Space
Table/View Technical Architecture
Column Mission/Vision
Index, … Best Practices
Policies
Data Model Standards
Entity RELATIONSHIPS Procedures
Attribute/Key Methodology
Domain Configuration
Valid Value, ...
People/Stewards
Reporting Tool Locations

Need people Report/Query
Data/Logic
Timing/Dependencies
Business Rules
Distribution STRUCTURED UNSTRUCTURED Metrics/Measurements Need discipline
to define, Relationships between System
DATA DATA
Relationships between Non-
Infrastruct Data specified above. System Infrastructure Data to maintain the
produce, use Example: Database to Table to
Column
specified above. Example: Method
to Procedure to Steward
meta-data.
the meta-data. more … Inventory & Audit
more … Inventory & Audit

Structured Data Relationships Unstructured Data

Resides in … “Data” Resides in … “Artifacts” if at all Resides in … “Artifacts”
Databases/Tables/Columns Between Assets & the Data Documents (Non Web-Based)
IT Data Management Tools Between System & Non-Sys Data Content (Web-Based)
Business Technology Tools Within System & Non-System Data Desktop Tools
Spreadsheets/Desktop Tools Between Structured & Unstructured Data Video/Graphics/Video
Other Example: Applications (System Asset) to Other
Hardware (Non-System Asset),
Hardware to Software (Non-System Asset)

Twitter at #RWDG 29


The Role / Tool of Data Stewards

• The Role of Metadata Data Stewards:

The Definition Metadata will not manage itself.
Metadata Definers: Responsible for Data Definition Metadata
2 Perspectives:
1. Someone has to Identify What Definition Metadata is Needed
2. Someone has to Provide the Definition Metadata
Operational Data Stewards

The Definition of Metadata will not manage itself.

Twitter at #RWDG 30




The Production Metadata will not manage itself.
Metadata Producers: Responsible for Data Production Metadata
2 Perspectives:
1. Someone has to Identify What Production Metadata is Needed
2. Someone has to Provide the Production Metadata

The Production of Metadata will not manage itself.

Twitter at #RWDG 31




The Usage Metadata will not manage itself.
Metadata Users: Responsible for Usage Data Metadata
2 Perspectives:
1. Someone has to Identify What Usage Metadata is Needed
2. Someone has to Provide the Usage Metadata

The Usage of Metadata will not manage itself.

Twitter at #RWDG 32


Applying Metadata Stewards to Metadata Process

• Second Order of Business:
– Identify Metadata Stewards and Record Who They Are
– Identify When and How They Should Be Involved
– Follow Governed Process

Twitter at #RWDG 33



Twitter at #RWDG 34



• Pro-Active Metadata Governance
– Build application of meta-data stewards into existing meta-data project
management methodology

• Reactive Metadata Governance
– Build application of meta-data stewards into meta-data quality
methodology
– Build application of meta-data stewards into meta-data issue
resolution process

Twitter at #RWDG 35


Revisiting the Common Data Matrix

Twitter at #RWDG 36


Revisiting the Data Governance Framework
Non-Invasive Data Governance™ Operating Model
of Roles & Responsibilities

Data Governance Program Team Executive Level
Data Governance Team Manager Data Steering Committee
Responsible for Administering the Program, Senior-most level knowledge of the program.
facilitating use of the Data Governance Council, Leverage existing business structure as possible.
communicating program components and value to the Organization. If no structure exists, utilize an IT Steering Committee
Executive
Advisory assistance from other levels. or an executive founded board or council.

IT Subject Resource Experts Strategic Level
System/Data Resource Experts Data Governance Council
IT Staff including Application Data Governance Sponsors Group
Development, Data Design, or Group of similar participation. One person, plus
Security, and other Data alternate, for each Business Unit represented & IT.
Strategic -
Resource Management. Consider financial services & human resources.
Enterprise

Tactical Level
T)

Data Domain Stewards
GP
(D

Per assigned Subject Areas
ts
m

r
pe
a
Te

Ex

One per Business Unit
m

e
ra

rc
og

ou
Pr

es

Tactical Level - Cross Business Unit
ce

tR
an

ec

Operational Level
rn

bj
ve

Su

Operational Data Stewards (existing)
Go

IT

Es
Data Definers, Producers, Users
ta

c
Da

ala
These people are presently defining,

tio

Co
producing and using data as part of their

n

m
jobs. Recording of the Operational Data

/A

m
un
pp
Stewards will be an important enabler of

i ca
ro
improved communications, coordination

va

tio
lP
and cooperation among stewards.

n
at
h
Operational Level - Business Unit Specific

Twitter at #RWDG 37



SysSMEs & DataSMEs

Meta-Data Producers
Meta-Data Definers
Domain Stewards

Meta-Data Users
Meta-Data Stewards

Meta-Data
Meta-Data

Meta-Data
Mapped to Meta-Data
Processes

Process Step Outcome / Deliverables

1. Develop Conceptual Model / Conceptual Model
Data Taxonomy X X X X
of Taxonomy

2. Further Develop Logical Model Attributed Logical Model of
Including Attribution of Entities X X X X
Taxonomy

3. Specify Which Taxonomy Attributes Will Be Data List Per
Included in the Repository X X X X
Business Requirements

4. Conduct Data Analysis of Attributes Using Data Analysis for Data Listed in
Data Analysis Template X X X X X
Step 3

5. Map Sources of Source Data to Mapping of Source Data to
Repository Data Structures X X X X X
Data

6. Assess Change Control, Versioning, Security X X X X X Change Control, Versioning &
Security Requirements

7. Selection of the “Best” Source Documentation Validating
for the Data X X X X X X
“Best” Source

8. Design and Develop the Data Movement, User Interface
Data “Engine” and User Interface X X X X X
& Functional Development

9. Validate and Test Data X X X X X X Sign Off of Data Validation
in the Repository

Legend: Gray Block w/Bold X = Active Participant, Block w/Dull X = Optional Participant, * See Appendix for Template

Twitter at #RWDG 38


Governing Metadata Through PMM

Fill in With Fill in With
Metadata Domain Specs
Definition Metadata

Fill in With Fill in With
Metadata Steward
Location Responsibility

Twitter at #RWDG 39


Crossing the 2-Way Street

• Metadata & Data Governance are like “Love & Marriage”

“You can’t have one without the other!”

• Metadata is a
By-Product of Governing & Stewarding Data.

• The Metadata Needs to be
Governed Just Like Business Data.

Twitter at #RWDG 40


Metadata Will Not Govern Itself

• Follow a Non-Invasive Metadata Governance™ Approach:
– Identifying rather than assigning meta-data stewards.
– Applying roles and responsibilities to meta-data processes.
– Providing formality and structure to what has always been informal.

• Thank you for attending my session!

Twitter at #RWDG 41


Real-World Data Governance: Metadata & Data Governance

Real-World Data Governance: Metadata & Data Governance

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (6)

Ähnlich wie Real-World Data Governance: Metadata & Data Governance

Ähnlich wie Real-World Data Governance: Metadata & Data Governance (20)

Mehr von DATAVERSITY

Mehr von DATAVERSITY (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Real-World Data Governance: Metadata & Data Governance