This is a slide deck that was assembled as a result of months of Project work at a Global Multinational. Collaboration with some incredibly smart people resulted in content that I wish I had come across prior to having to have assembled this.
2. MDM BI – Vision
Transaction
Systems
Business Process Management
Capture Consume
Master Data Management
Instantiate Provide
Monitor
Business Intelligence
Data
Warehouse
Big Data
Visualization
Goals:
• Enable Speed of Innovation for MDM BI activities across the
Enterprise, the Operational Units, and the Business Users
• Define Governance and Data Management Practices
throughout the Enterprise
• Improve the accuracy of Master Data for Unity and Legacy
Systems
• Build Enterprise Class Metrics and KPIs based on Business
Processes
3. Enterprise
• Data Stewards
• Data Governance
• BI Team
• Technology Team
Operational Unit
• MDM BI COEs
• BRM Teams
• Data Scientists
• Data Analysts
Business Users
• Data Stewards
• Reporting Stewards
Operational Unit
Business UsersEnterprise
Ideation
Transaction
and KPI
Consumption
Self
Service
Requirements
Data ProfilingGovernance
Visualization Testing
Publishing
Architecture
Standards
Methods
Technologies
Training
Solution Design
Data Ingestion
Analytic
Algorithms
Data
Exploration
Application
Development
Master Data
Business
Intelligence
Big Data
Data
Cleansing
MDM BI – Roles and Functions
4. Business Data Governance – Overview
Master Data
Management
DW & BI
Management
Data Quality
Management
Metadata
Management
Data Security
Management
Data
Architecture
Management
Data
Development
Misconception Reality
It's an IT responsibility
Data Governance requires a partnership between Business,
Technology, and Operations
One size fits all
The organization, processes, and technology must be tailored to
fit the culture and leverage existing governance structures and
technology
It can succeed through a grass
roots bottoms up effort
Success requires executive advocacy and sponsorship
It's about having the right tools
Data governance requires the integration of organization,
processes, and technology tools
It can be an add on
responsibility that doesn't need
to be measured or rewarded
Data stewardship may require full time staff commitment. If the
role is not measured or rewarded, the result will be ineffective
governance action
It’s a big bang implementation
Standing up data governance structures is an evolutionary
process that requires effective change management
Data governance is the orchestration of people, process, and
technology to enable the leveraging of data as an enterprise
asset through a well-defined organizational structure,
policies, rules, decision rights and accountabilities for
decision making and management of Master Data.
5. Data Governance – 180 Day Plan
•Create the appropriate review and escalation methods for managing data quality and integrity
•Enable the linkage between business process/data owners who “champion” data and metrics with the data
architects and data stewards who manage the transaction level detail
Governance
&
Stewardship
•Integrate roles across functions (e.g., data cleansing, data architects, data stewards, process/data owners)
•Understand the needs of the consumer of the data and connect appropriately
People/
Organization
•Define end to end, consistent processes across all data types, linking transaction level data with Management
Information
•Define proper controls to manage data quality and integrity on a sustainable basis
Process
•Identify Tools for cleansing, mapping, identifying anomalies, etc.
•Leverage data management tools and infrastructure that have rapid scalability and functionality
•Define consistent architecture that enables “one version of the truth”
Technology
6. MDM BI
Data Cleansing
MDM BI
Product
MDM BI
Customer
MDM BI
Vendor
MDM BI
Operational
MDM BI
Financial
MDM BI
Stat/Mgmt Rptg
EBPM - PTP
EBPM - PTD
EBPM - OTC
EBPM - RTR
EBPM - FTP
As requirements for
MDM and KPIs are
assembled,
PRIORITIZATION and
sequence can be
further refined
Outcomes
MDM BI – Discovery Approach
Enterprise
Data Model
Master Data
Models
Required
KPIs
7. MDM BI – Deployment Approach
Discovery Analyze / Define Design
Deliverable
• Conduct Discovery
Sessions
• Define Solution Scope
• Define Solution Concept
• Define General System
Concept
• Describe Potential
Impact
• Plan Project
• Analyze Guidance
Architecture
• Analyze Data
Architecture
• Create Data Schema
Map
• Assess Data Quality
• Analyze System
Architecture
• Analyze System
Requirements
• Design Guidance
Architecture
• Design Data
Architecture
• Design System
Architecture
Design Application
Specification
• Design Data Migration
• Design Human
Transition Support
Activity
• Discovery Summary
• Solution Scope
• Solution Concept
• System Concept
• Impact Summary
• Project Plan
• Organizational Model
• Guidance Model
• Data Model
• Data Schema Map
• Data Quality
Assessment
• System Interaction
Diagram
• System Requirements
Summary
• Organizational Model
• Guidance Model Data
Model
• System Interaction
Diagram
• Application Specification
Data Migration Plan
• Organization Change
Management Plan
• Training Plan
• Discovery & Analyze initiated via
common stakeholder interviews
• Design & Build executed on
Global Template
• SAP Deployments will review
efforts from previous phases but
largely be testing and refining
exercise
Build Test Deploy Sustain
8. MDM – Implementation Approaches
Consolidation Registry Coexistence Centralized
For Reporting, analysis, and
central reference
Mainly for real-time central
reference
For harmonization across
databases and for central
reference
Acts as system of record to
support transactional activity
Matches and physically stores a
consolidated view of master
data
Matches and links to create a
“skeleton” system of record
Matches and physically stores a
consolidated view of master
data
Matches and physically stores
the up-to-date consolidated
view of master data
Updated after the event and not
guaranteed up-to-date;
authoring remains distributed
Physically stores the Global ID,
links to data in source systems
and transformations
Updated after the event and not
guaranteed up-to-date;
authoring remains distributed
Supports transactional
applications directly – both new
and legacy – typically through
SOA interfaces
No publish and subscribe; not
used for transactions but could
be used for reference
Virtual consolidated view is
assembled dynamically and is
often read-only; authoring
remains distributed
Publishes the consolidated view;
not usually used for transactions
but could be used for reference
Central authoring of master data
Analytical Focus Operational Focus Operational Focus Operational Focus
System of Reference System of Reference System of Reference System of Record
9. Enterprise
Master Data
Model
Legacy
ERP
MDM People
MDM Processes
MDM Tools
Transformation
Certified Master
Data
2
3
4
1. Non-Certified Data from the
Legacy Systems is ingested and
transformed into the Enterprise
Master Data Model built during
the Discovery Phase
2. The Enterprise Data Model is
adjusted as necessary and
cleansed data is pushed back to
the Legacy ERP System
3. Certified Master Data is produced
and the required refinements to
processes and data architecture
are made to enable downstream
consumption
4. All of this is enabled by dedicated
MDM personnel, utilizing MDM
tools and processes
MDM – Pre-Cleanse & Deployment Process
1
Non-Certified
Data
10. BI – Ownership Structure
Sandbox (50%)
[user created content]
Shared (30%)
[user created and shared
content]
Production
(20%)
Gather
Data
Visualize
PublishConsume
Ideate
Business
Users
Require-
ments
Profile
Data
DesignDevelop
Test
IT
Sandbox Environment
• Business users author and use BI content with no
constraints or limitations. This is where data
exploration, discovery, and what-if analyses
happen.
• Tools and technologies: Microsoft Office
• IT involvement is strictly limited to infrastructure
and tools support plus monitoring to identify
usage patterns, commonalities, and opportunities
(using BI on BI) for potential production
hardening.
• Content produced here is used in individual tasks
and low-risk applications.
Shared Environment
• Business users share and collaborate on BI content
with their colleagues.
• Tools and technologies: SharePoint BI, Office 365
• IT steps up monitoring and now watches for red
flags (too much data, too many users, too critical
or risky applications) and opportunities (using BI
on BI) for production hardened BI Content.
• Content produced here is shared within
departments and workgroups. Low-risk, low-
criticality decisions can be made based on this
content.
Production Environment
• Business uses and authors BI content within the
limitations and constraints of the enterprise data
model, standards, policies, rules, guidelines, etc.
• Tools and technologies: EDW, Visual Studio,
SharePoint BI, Office 365
• Owned, run, and managed by IT.
IT Benefits
• Backlog Reduction – only heavily-used,
complex, or critical applications come to IT for
production-hardening.
• Requirements already defined; Project Lifecycle
is greatly reduced; Enhancements during
testing cycle minimized.
• Shadow IT is embraced as a competitive
advantage; however, using the strategic
technology stack defined by IT.
Business Benefits
• Business users are empowered to
create BI content on their own
schedule without any constraints
or limitations – at the speed of
business innovation.
• They modify the model and
visualizations through iteration until
the requirements are identified and
met.
11. Hadoop = Data Lake
• Land all data in Hadoop as-is from any source
• Enables Analytics Sandbox
• Enables MDM Pre-Processing
• Enables EDW Population with Relevant Data
• Enables Application Access via API Layer (including 3rd party
developers)
Actively Archive from EDW to Hadoop
• Little-Used Historic EDW Data resides in Hadoop (lower cost
storage)
• Define an archive strategy for various data types
Enable business analytics
• Identify tools, methods, and security requirements for
interaction with the distributed file system
• Introduce exploratory analytics without jeopardizing SLAs
• Introduce new machine learning, or data mining techniques
on years worth of data
Enable BU Innovation
• BU Teams continue to innovate with their business users
within the Enterprise Framework – ingestions driven by BU OR
Enterprise requirements
• Data Scientists and Analysts can access all Data for Analytics
• BU IT & Business Teams can access all Data for Visualizations
with proper security
• Enterprise Data Model, HDFS Standards, and Access Methods
extensible to manage localizations at the BU level and below
BI – Data Flows
Data Sources/Transports
Transaction Data
Customer Data
External Data
Industry Data
Sensor Data
DB
Files
REST
JMS
HTTP
SOAP
Hadoop
Compute +
storage
… … …
… … … …
… … … …
… … …
Compute +
storage
supporting technologies& packages
EDW
BI Tools & Applications
Query & Visualization Tools
JDBC/ODBC Compliant
Tools & Applications
Analytic & Reporting Tools
R
Mahout
Excel
Excel
PowerPoint
Power View
MDM
API Layer
BU1 BU2 BUn
BU1 BU2 BUn
BU1 BU2 BUn
12. Establish, Maintain, and
Periodically Review and
Recommend Changes to Data
Governance Policies, Standards,
Guidelines, and Procedures
The Team responsible to develop the strategy,
govern the tools selected to acquire and transform
relevant data into knowledge to drive business
decisions and actions to achieve desired results. In
addition, the resulting information has to be
tailored to – and distributed to – the appropriate
levels of management and operations in a timely
manner to be most effective. In some cases BI
Execution of Reporting and Analytics is performed
as well.
Provide Quality Assurance –
Oversight, Monitor, Report Results
to Data Governance Council
VP MDM & BI
Business
Governance
Leader
Data Stewards
by Domain
Data Stewards
by Business Unit
Data
Governance
Leader
Data Quality
Data
Architecture
Data
Conversions
Big Data
Architect
BI Governance
Leader
BI Leads
BI Visualization
Developers -
Enterprise
BI ETL
Developers -
Enterprise
BI Developers –
Business Units
Technology &
Tools Leader
Technology
SMEs
DBAs
System SMEs
• Develop and Deliver Data Governance
Program Educational, Awareness &
Mentoring Materials
• Assist in Defining Data Quality Metrics
for Periodic Release
• Support Data Quality Issue Analysis and
Remediation for “Strategic” Data
• Oversee Enterprise Data Governance
Program Development / Architect
Solution & Framework
• Administer the Program including
facilitate the Data Governance Council
meetings
• Provide the Agenda for the Data
Governance Council Meetings to the
Approved by Council Owner Pre-
Meeting
• Facilitate Data Governance Organization,
Tactical & Operational Stewards, the
Data Governance Council Involvement
• Conduct Audits to Ensure that Policies,
Procedures and Metrics are in Place for
Maintaining/Improving the Program
Functionally
Aligned Roles
Organizationally
Aligned Roles
Sample Organization