Analyst Webinar: Best Practices In Enabling Data-Driven Decision Making

ANALYST WEBINAR
Best Practices In Enabling
Data-Driven Decision
Making
Based on a survey by Intelligent
Business Strategies

Speaker
Mike Ferguson
Managing Director, Intelligent
Business Strategies
Sales Engineer, Denodo
Emily Sergent

Leverage Data Virtualization for Data
Integration and Data Management
Emily Sergent, Sales Engineer

4
Data Integration Challenges
Sales
HR
Supply
Executive
Marketing
Common hurdles:
× Heterogeneous data sources
× Data silos
× Inconsistent business views
× Complex governance
× Synchronization is slow and costly
× IT infrastructure lacks flexibility

5
Six Critical Capabilities of Data Virtualization
Modern Data Platform
Sales
HR
Executive
Marketing Apps/API
Data Science
AI/ML
1. Data Abstraction: decoupling
applications/data usage from data
sources
2. Data Integration without
replication or relocation of
physical data
3. Easy access to any data, with high
performance in real-time/right-
time
4. Data Catalog for self-service
data services and easy discovery
5. Unified metadata, security &
governance across all data
assets
6. Data delivery in any format with
intelligent query optimization
that leverages new and existing
physical data platforms
A logical data layer – a “data fabric” – that provides high-
performant, real-time, and secure access to integrated business
views of disparate data across the enterprise

6
Sales
HR
Executive
Marketing Apps/API
Data Science
AI/ML
Denodo Data Virtualization: A Modern Data Platform
Connect
✓ Access disparate data sources in real-time
✓ Efficiently leverage capabilities of different technologies
✓ Abstract complexities of format, location and protocols
Combine
✓ Build views tailored to business needs and use cases
✓ Provide on-demand data access via a state-of-the-art optimizer
✓ Transparently apply governance and security rules
Publish
✓ SQL access: JDBC, ODBC and ADO.NET
✓ Data Services: SOAP, REST, OData, GraphQL
✓ Built-in data catalog and self-service exploration tool

7
Sales
HR
Executive
Marketing Apps/API
Data Science
AI/ML
Denodo Data Virtualization: A Modern Data Platform
Base
views
Standardized
views
Business views
Unified security, semantic layer and query optimizer
Data
sources
Data
Consumers

8
AXA XL: A Streamlined Data Infrastructure
›
Outcomes Achieved
• Business agility: get data
to the right people more
quickly
• Improved data
consistency: Single
version of the truth
• Privacy and regulatory
compliance
›
Denodo Solutions
• Provide tool-agnostic
semantic “business
translation layer”
• Federate access to
cloud and legacy data
sources
• Role-based security
for all sources and
consumers
Business Needs
• Support multiple
stakeholders with
different BI tools
• Decrease replication
and data access
latency
• Improved governance
and security

9
AXA XL: Data Virtualization in the Cloud

10
Logitech: Successful Cloud Modernization
›
Denodo Solutions
• Provide secure and
governed business
layer
• Combine Snowflake
data with data from
Salesforce, Zendesk,
Google Analytics
• Support BI and data
science initiatives
›
Outcomes Achieved
• Flexible cloud migration
with minimal impact to
business users
• Increased speed: weekly
demand forecast went
from 3 days to a few
hours
• Cost-cutting
Business Needs
• Integrate diverse
internal and external
data sources
• Break data silos
• Manage costs with a
cloud data
infrastructure

11
Logitech: Successful Cloud Modernization

12
Data Catalog within Data Virtualization
Data Catalog within data virtualization to seamlessly integrate data catalog and data delivery
Dynamic Catalog of curated, timely, contextual, and reusable
information assets and data services.
Govern – Fine-grained privileges that govern access to the catalog;
both metadata and information assets.
Describe – Ability to describe data assets with categorization, tagging,
annotations, lineage and other business-oriented metadata.
Usage-based metadata – who, when, what, why, and how of data
consumption.
Data Preparation – Ability to transform, refine, and customize data
assets for business use.
Enhanced UI – Business-friendly user interface geared to roles such as
data stewards, data analysts, and citizen analysts

13
In Conclusion
• Data trustworthiness and pertinence
▪ Decrease replication and increase relevance and reliability
• Flexibility for business users
▪ Self-service, tool-agnostic access to data from across the enterprise
• IT flexibility and continuity of service
▪ Migration to the cloud and the modernization of data and applications
• Control and governance for data management
▪ Reduce compliance risk while enhancing security and privacy
Data virtualization can help you leverage the strengths of a variety of data source
and infrastructure technologies while ensuring:

15
Next Steps
Access Denodo Platform in the Cloud!
Take a Test Drive today!
G E T S TA R T E D TO DAY
www.denodo.com/TestDrive
Trends in Establishing a Data-Driven Enterprise
A whitepaper based on trends identified in the 4th Industrial Revolution
Survey conducted over 500 enterprises.
By Mike Ferguson, Intelligent Business Strategies
DOWNLOAD WHITEPAPER

Thanks!
www.denodo.com info@denodo.com
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm,
without prior the written authorization from Denodo Technologies.

Copyright © Intelligent Business Strategies, 1992-2020, All Rights Reserved 1
Mike Ferguson
Managing Director
Intelligent Business Strategies
Denodo Webinar
October 2020
Best Practices In Enabling Data-Driven Decision Making
1
2Copyright © Intelligent Business Strategies 1992-2020
Topics
§ Data-driven enterprise – vision, expectation and thirst for new data
§ Challenges to becoming data driven
§ Critical success factors to becoming a data driven enterprise
§ What are companies doing to become data driven?
§ What progress are they making?
2

The
Data Driven
EnterpriseDisruption
Speed &Agility
Competitive
Advantage
DATA
&
Analytics
Transformation
3
Data Driven Vision Implementation
- The Urgency To Invest Is Rising With Focus On AI / ML And Cloud
Source: Big Data and AI Executive Survey 2019, New Vantage Partners
4

But What About Data? Data Quality Is Critically Important If
Decisions Dependent On It Are To Be Effective
=Garbage In Garbage Out
5
We Need A Data And Analytics Hub With Trusted Data Assets And Analytics
Built Once, Reused Everywhere In A Data Driven-Enterprise
Trusted
Data and
Analytics
HR
Sales
Marketing
Service
Finance
Procure
-ment
Operations Distribution
Partners
Customers
Suppliers
Employees
Things
The Intelligent
Business
Commonly understood, trusted
data, and analytical services
available across the enterprise
6

Topics – Where Are We?
ØChallenges to becoming data driven
7
Challenges - The Operating Environment That We Have Now
Created Spans Edge, Multiple Clouds And The Data Centre
gateway
gateway
edge
devices
Data Centre(s)
gateway
gateway
data
flow
data flow
Cloud computing
data flow
data flow
edge
devices
edge
devices
edge
devices
8

Challenges
– Ever Increasing Types of Data That Businesses Want To Analyse
Type Of Data Examples Uses
Traditional
structured data
• Master data
• Transaction data
• Customer, product, employee, supplier, site,…..
• Orders, shipments, returns, payments, adjustments..
Machine
generated data
• Clickstream web server logs
• IVR logs, App Server logs
• DBMS logs
• On-line behaviour analysis
• Cyber security
• Consumer IoT (Sensor data)
• Industrial IoT (Sensor data)
• Location, temperature, movement,
vibration, pressure
• Product usage behaviour
• Product or equipment performance
Human
generated data
• Social network data
• Inbound email
• Competitor news feeds
• Documents
• Voice interaction data
• Unstructured text , sentiment analysis
External data • Open government data
• Weather data
• Structured data
• Semi-structured data, e.g. JSON, XML, AVRO
• Sales impact, distribution impact
9
Challenges – Data Is Being Ingested Into Multiple Types Of Data
Store Both On-Premises And In The Cloud
Enterprise
cloud
storage
Data.Gov
C
R
U
prod cust
asset
D
MDM
NoSQL
DBMS DW
I
D N
A G
T E
A S
T
10

Challenges – Managing, Governing And Integrating Data Is
Becoming Increasingly Complex As Data Sources Grow
<XML>
Digital media
RDBMSs
Web
content
E-mail
Flat files
Packaged
applications
Office
documents
Cloud storage
DW/BI
systems
Big data applicationsCloud based
applications
ECMS
“Where is all the
Customer Data?”
More and more data sources now need to be integrated to provide
information for business use
Divergence!
11
Challenges – How Do You Govern Self-Service Data Preparation To
Avoid Chaos In The Enterprise?
social
Web
logs
web cloud
sandbox
Data Scientists
sandbox
Data Scientists
sandbox
Data Scientists
HDFS
ETL
/ DQ
Self-service
BI tools with data prep
ETL
new
insights
SQL on
Hadoop
DW
ETL
/ DQ
DW
marts
ETL
SCM
CRM
ERP
Data
prep
marts Self-service
BI tools with data prep
Data
prep
Built by IT
data
prepData
prepData
prep
Governance?
Everyone is blindly integrating data with
no attempt to share what we create !!
12

Challenges - The Danger Of Self-Service Data Preparation
– An Explosions Of Personal Silos!
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
Analytical
tools
Data prep
tools
Data
store
Silo
sources
=Garbage In Garbage Out
Inconsistent data!!
Multiple versions!!
13
OR
Companies Want Organised, Findable, Trusted, Re-Usable Data
Assets!
Image source: https://ebcwblog.wordpress.com/2014/10/02/how-to-decorate-with-books/ Image Source: Maughan Library, London (King's College London Library)
14

ØCritical success factors to becoming a data driven enterprise
15
Key Requirements For Producing, Managing And Governing
Reusable Data Assets In A Distributed Data Landscape
1. Start with business value – what are your business objectives and drivers?
2. Map business drivers to data management capabilities e.g. DW, MDM, data science…
3. Assess your maturity (SWOT) in managing, producing and governing trusted data
4. Identify key data needed to deliver the value you want to achieve your business goal
5. Create an inventory of your data sources
6. Establish an organised data lake to ingest and process data
7. Ingest, discover and catalog your data landscape to understand what you have
8. Organises people into projects to produce data assets and publish them as a service
9. Create standard definitions for data assets you need to produce to achieve a goal
10. Establish a common methodology and architecture to produce needed ready made data assets
11. Use common technology to prepare, integrate, produce and govern specific data assets
12. Publish trusted data in an enterprise data marketplace for others to find, consume and use
13. Use data virtualisation to render data as virtual trusted data services for use in driving new value
16

Technology Requirements – Need A Data Catalog And Data Fabric Software
To Govern & Integrate Data Across Edge, Multiple Clouds And Data Centre
Enterprise Data Fabric (Data Management) Software
Data
catalog
gateway
Edge devices
Date Centre
Data Fabric software helps avoid or reduce the chances of data silos
Data Discovery, Profiling, Semantic Tagging, Info Catalog, Data Lake, Data
Governance, Data Preparation / integration, Data Virtualisation, APIs, MDM
17
ØWhat are companies doing to become data driven?
ØBusiness drivers
ØOrganising to succeed
ØChallenges
ØPriorities
18

Improving Customer Experience Is The Dominant Business Driver
For Becoming A Data-driven Enterprise
(500 companies surveyed)
19
Organising To Succeed - Business Strategy Aligned Project Teams Producing
Data And Analytics Using An Extensible Common Data And Analytics Platform
Programme Office +
A Data & Analytics
Centre of Excellence
C-Suite
Business
Strategy
D&A To
Do List
D&A To
Do List
D&A To
Do List
Analytical
ecosystem
CIO or CDO
Common Extensible Approach To Data & Analytics Development
Common Extensible Data Fabric & Info Catalog
D&A =
Data & Analytics
Data Warehouse
RDBMS
EDW
DW & marts
mar
t
DW
Appliance
Advanced Analytics
(structured data)
Analytical
RDBMS
Streaming
data
Streaming
analytics
NoSQL
DBMS
Hadoop
data store
Advanced Analytic
(multi-structured data)
C
R
U
D
Pro
d
Asset
Cust
MDM
NoSQL
Graph DB
Cloud
storage
20

How Are Companies Organising to Succeed?
- CIOs and CDOs Are The Most Popular Executives Accountable For Data
21
Organise To Become Data Driven
– Information Producers And Information Consumers
§ Need to make use of
• A business glossary and information catalog
• Role-based data management tools aimed at IT AND business
• A collaborative approach by business and IT to produce data and analytical assets
• A catalog to quickly find reusable trusted assets to drive business value
raw data
trusted
data
Information
catalog
BI tool or
application
search
find
shop
order consume
provision
Data scientist /
Data engineer
IT professional
information producers
publish
business analysts
information consumers
clean
integrate &
analyse
Data
Marketplace
22

Challenges - Many Companies Are Struggling To Transform Their
Organisations Into Data Driven Businesses
§ 71.7% of firms report that they have yet to forge a data culture
§ 69.0% of firms report that they have not created a data-driven culture
§ 53.1% of firms state they are not yet treating data as a business asset
§ 52.4% of firms claim that they are not competing on data and analytics
§ 40.3% of firms cited lack of organisational alignment / agility as biggest
challenge to adoption of Big Data and AI
§ 92.5% of firms cite people (62.5%) and process (30%) as the principal
challenge to becoming data driven
Source: NewVantage Partners - Big Data and AI Executive Survey 2019
No improvement – getting worse!
23
How Longer Is It Taking A Data Driven Culture?
24

What Challenges And Inhibitors Are Companies Facing In Trying To
Create A Data Driven Enterprise? - Culture Challenges
25
Lack Of Skills In Data Engineering And Cloud Are Preventing
Companies From Achieving Their Business Goals
Data engineers with cloud skills are In hot demand but short supply
26

Challenges - Algorithmic Bias Caused By Poor Data Quality Is
Proving To Be A Major Issue When Implementing AI
27
Priorities - Data Quality, Data Governance, Data Tools And Data
Skills Are Top Of The Priority List In A Data Driven Organisation
Data is getting higher priority than analytics
28

ØWhat progress are they making?
29
There Is A Major Push To Migrate Analytical Workloads To The
Cloud
30

Data Virtualisation Eases DW Migration To The Cloud
– E.g. Migrate DW And Change To Virtual Marts All Without Users Noticing
Sales DW
UK DE FR NL
ESP CH IT BEUS
ETL ETL ETL ETL
ETL ETL ETL ETL ETL
Business Impact
Very high total cost of ownership
No Agility – Takes time and is costly to change
Risk of inconsistency across DWs and marts
No drill down across marts
E.g. Country Specific Data Marts
BI Tools
Sales DW
UK DE FR NL
ESP CH IT BEUS
Business Impact
Low total cost of ownership
Agility – Easy to change
No inconsistency across DWs and marts
Virtual aggregation and drill down across marts
Migrate &
change
the design
BI Tools
Data Virtualisation
31
Data Warehouse Modernisation Using Virtual Data Marts
- Data Virtualisation On Top Of A Data Warehouse
Data
Warehouse
Applications/
BI Tools
Data
Virtualisation
Server
mart mart mart
Applications/
BI ToolsApplications/
BI Tools
SQL X/Query
32

Modern Data Architecture Is A Key Requirement Of A Data Driven Enterprise
– Build Ready Made Data Assets To Shorten Time To Value
IoT
RDBMS
office docs
social
Cloud
clickstream
web logs
XML,
JSON
web
services
NoSQL
Files
DataIngestion
DataCuration/enrichment
Trusted data
assets
DW
Data Curation process
customer
product
orders
Raw
data
Raw
data
shipments
payments
Ready made
data products
DataVirtualisation
Data science
Application
Data Marketplace
Trusted virtual
data assets
Landing
zone
Raw
data
Raw
data
Trusted zone
Stream
processing
BI tool
Data Lake
BI toolpublish
(Build once, re-use everywhere)
Graph
DBMS
Provision trusted data and simplify access
using data virtualization
provision
provision
provision
provision
33
A Logical Data Lake – Multiple Data Stores Used For Data
Ingestion, Data Curation And Data Publishing
IoT
RDBMS
office docs
social
Cloud
clickstream
web logs
XML,
JSON
web
services
NoSQL
Files
D
A
T
A
I
N
G
E
S
T
I
O
N
Ingestion data
stores
streams
DW
staging
NoSQL
cloud
storage
e.g. Kafka
D
A
T
A
C
U
R
A
T
I
O
N
Curation
staging data
stores
Master Data
customer
product
orders
shipments
payments
D
A
T
A
P
U
B
L
I
S
H
I
N
G
Trusted
Data stores
DataVirtualisation
DW
Enterprise Data Fabric
Access
……
Information Catalog
34

A Logical Data Lake – Data Virtualisation ‘Fabric’ Over Source Data Allows It
To Be Left Where It Is (e.g. for legal reasons or it is too big to move)
IoT
RDBMS
office docs
social
Cloud
clickstream
web logs
XML,
JSON
web
services
NoSQL
Files
DataIngestion
DataCuration/enrichment
Trusted data
assets
DW
customer
product
orders
Raw
data
Raw
data
shipments
payments
Ready made
data products
DataVirtualisation
Data science
Application
Trusted virtual
data assets
Landing
zone
Trusted zone
Stream
processing
BI tool
Data Lake
BI tool
publish
(Build once, re-use everywhere)
Graph
DBMS
Provision trusted data and simplify access
using data virtualization
Data Marketplace
Catalog
(trusted
data)
Catalog
(raw
data)
provision
provision
provision
provision
publish
DataVirtualisation
Data Curation Pipelines
35
Create A Data Lake And Information Supply Chain To Curate ‘Business Ready’
Data And Analytical Assets Published In A Marketplace For Users To Consume
IoT
RDBMS
office docs
social
Cloud
clickstream
web logs
XML,
JSON
web
services
NoSQL
Files
information
consumers access the
data marketplace to
shop for business
ready data and
analytical assets
shop
for
data
Info
Catalog
Data
marketplace
Information supply chain (curation process)Project
Business ready data assets
(common vocabulary)
Data Fabric ELT Processing
Ingestion zone Curation zone Trusted zone
36

Trusted Business Ready Data In An Enterprise Data Marketplace
For Users To Consume And Use
Data available as a Service
Master Data
• Customers
• Products
• Suppliers
• Assets
• Employees
• Materials
Transaction Data
• Orders
• Shipments
• Payments
• Adjustments
• Returns
Business ready data
products are often
logical entities
37
(500 Companies surveyed)
4th Industrial Revolution Survey Shows That Implementing A Data
Lake Is The #1 Priority in 2020
Source: Intelligent Business Strategies Nov 2019
38

What Is An Enterprise Data Marketplace?
Enterprise Data Marketplace
A catalog containing ready made,
trusted, data and analytical assets
available as services with common
data names documented in a business
glossary, full metadata lineage and that
are tagged and organised to make
them easy to find, access, share and
reuse across the enterprise
39
Reducing Time To Value
– Shop For Trusted Ready-Made Data And Deliver Value Rapidly
information
consumers access
the data
marketplace to
shop for ready-to-
go data and
analytical assets
shop
for
data
Info
Catalog
Data
marketplace
Trusted data
service
Query service
BI report /
dashboard /
story
BI Insights pipeline
Trusted data
service
Analytical
service
Predictive insights pipeline (rapid assembly)
Trusted data
service
Analytical
service
Decision
service
Prescriptive analytical pipeline (rapid assembly)
BI report /
dashboard /
story
Trusted data
service
New virtual
data service
Enrich data
Trusted data
service
40

Provisioning lots of physical
copies could create more chaos
Data Virtualisation Plays A Key Role In Enterprise Data Marketplace
Operations Because It Minimises The Need To Provision Data Physically
Ordered data
Provision
Provision
Provision
All copies of data need to be catalogued
DataData
Data
Data
Data Data
COPY
COPY
COPY
DataVirtualisation
Provision
data virtually
41
Data Lake Ingestion Zone
Logical Data Lake
EDW
mart
DW & marts
DW
Appliance
Advanced Analytics
(structured data)
C
R
U
D
Prod
Asset
Cust
MDM
Streaming
data
NoSQL DB
e.g. graph DB
Simplifying Data Access Via A Logical Data Warehouse Architecture
- Use Data Virtualisation To Access Data In Multiple Stores
Enterprise Data Fabric (Self-Service Business & IT)
Data&InsightCuration
Analytical tools &
applications
Logical Data Warehouse (Data Virtualisation with a common vocabulary)
shared
metadata
Commonly understood, trusted data and insights
Data
catalog
feedsIoT
XML,
JSON
RDBMS Files office docssocial Cloud
clickstream
web logs web
services
NoSQL
42

Data Virtualisation - Integrated Customer insight
The Power Of Data Virtualisation And Logical Data Warehouse
- Integrated Customer Insights Plugged Into All Front Office Channels
EDW
DW & marts
NoSQL DB
e.g. graph DB
mart
DW
Appliance
Advanced Analytics
(structured data)
Advanced
Analytics
Streaming
data
RT Analytics
C
R
U
prod cust
asset
master data
Customer sentiment,
interactions,
online behaviour,
& new data
Customer
relationships*,
social network
influencers
Customer real-
time location,
product usage &
on-line behaviour
Customer
master data
Customer
purchase activity
& transaction
history
Customer predictive
analytical model
development
Sales Force
automation apps
Customer facing
bricks & mortar apps
Front-Office Operations
Customer
service apps
Customers
Improve customer
engagement
E-commerce
application
M-commerce
Mobile apps
Social commerce
applications
Digital channels are
generating big data
e.g. In-store apps
In-branch apps
43
The Role Of Data Virtualisation Is Critical In A Logical Data Lake
IoT
RDBMS
office docs
social
Cloud
clickstream
web logs
XML,
JSON
web
services
NoSQL
Files
Logical
Ingestion zone
streams
DW
staging
NoSQL
cloud
storage
e.g. Kafka
DataVirtualisation
Access data using
virtual data sources for
data that can’t be
moved for legal reasons
or because it is too big
DW
DataVirtualisation
Master Data
customer
product
orders
shipments
payments
Logical Trusted
Zone
……
supplier
Provision data without
the need for copies
Trusted physical
data assets
Data
Marketplace
Trusted virtual
data assets
44

Progress On The AI Ladder
- Insurance Leads The Way In Implementing Prescriptive Analytics
Descriptive
analytics
Diagnostic
analytics
Predictive
analytics
Prescriptive
analytics
45
Conclusions
§ Ungoverned self-service data preparation on data across a distributed data
landscape will result in inconsistent data, mass re-invention and chaos
§ Organising to become data-driven is the most difficult part but is critical to success
§ A data-driven organization needs a modern data architecture that can work across
data center, multiple clouds and edge
§ Data virtualization software is a key technology component in a modern data
architecture to quickly provision trusted data and simplify access to insights
§ Companies who store data in a data center, one or more clouds and collect it from
edge devices need data virtualization software if they want to become data-driven
46

Thank You!
www.intelligentbusiness.biz
mferguson@intelligentbusiness.biz
@mikeferguson1
(+44)1625 520700
Thank You!
Mike Ferguson is Managing Director of Intelligent Business
Strategies Limited. And Chairman of Big Data LDN. As an
independent analyst and consultant he specialises in business
intelligence, analytics, data management and big data. With over 37
years of IT experience, Mike has consulted for dozens of companies,
spoken at events all over the world and written numerous articles.
Formerly he was a principal and co-founder of Codd and Date
Europe Limited – the inventors of the Relational Model, a Chief
Architect at Teradata on the Teradata DBMS and European
Managing Director of DataBase Associates.
47

Analyst Webinar: Best Practices In Enabling Data-Driven Decision Making

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Analyst Webinar: Best Practices In Enabling Data-Driven Decision Making

Ähnlich wie Analyst Webinar: Best Practices In Enabling Data-Driven Decision Making (20)

Mehr von Denodo

Mehr von Denodo (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Analyst Webinar: Best Practices In Enabling Data-Driven Decision Making