Do You Really Need a Data Warehouse? Avoid the Downsides Typically Associated with Data Warehouses

DOYOUNEEDADATAWAREHOUSE?
WHYPROPERLYSTAGEDDATAISCRITICALTOBI SYSTEMSUCCESS

questions
here
Copyright2014Senturus,Inc.
AllRightsReserved
This slide deck is part of a recorded webinar. To view the FREE recording of the entire presentation and download the slide deck go to
www.senturus.com/resources/do-you-really-need-a- data-warehouse/
Hear the Recording

Resource Library
Senturus’ whole purpose is to make you successful with Business Analytics. Thus, we offer a series of technology-neutral webinars, training on specific software, demonstrations, and no-holds-barred reviews of new software releases. We host dozens of live webinars every year and we offer a comprehensive library of recorded webinars, demos, white papers, presentations and case studies on our website--a wealth of learning resources. Most of our content is custom created and constantly updated, so visit us often to see what’s new in the industry.
www.senturus.com/resources/
3
Copyright 2014 Senturus, Inc. All Rights Reserved

•Introduction
•The QuickAnswer
•Why Business Intelligence (BI)
•Challenges & BasicRequirements of BI Systems
•Reporting Direct from SourceSystems
•Technical Solution Alternatives
•Data Warehouse Benefits
•How to Build a Data Warehouse (20,000 foot view)
•Additional Resources & Upcoming Events
•Q & A
TODAY’SAGENDA
4
Copyright 2014Senturus, Inc. All Rights Reserved

CRITICALSUCCESSFACTORSINBI
•Architectures & Data Transformation
•BI Tools
•Methodologies & Techniques
•People & Processes
Chapters in the
BI Demystified Series

CRITICALSUCCESSFACTORSINBI
•Architectures & Data Transformation
•Data Marts & Data Warehouses
•BI Tools
•Methodologies & Techniques
•People & Processes Chapters in the BI Demystified Series

John Peterson CEO & Co-FounderSenturusTODAY’SPRESENTER
7

WHOWEARE
SENTURUSINTRODUCTION

questions
here
Copyright 2014Senturus,Inc.
AllRightsReserved
Hear the Recording
Senturus’ comprehensive library of recorded webinars, demos, white papers, presentations and case studies is available on our website.
www.senturus.com

Our Team:
Business depth combined with technical expertise. Former CFOs, CIOs, Controllers, Directors, BI Managers & Enterprise BI/DW Architects
SENTURUS: BUSINESSANALYTICSARCHITECTS
10Copyright 2014 Senturus, Inc. All Rights ReservedBusiness IntelligenceEnterprise Planning
Predictive Analytics
Creating Clarity from Chaos

750+ CLIENTS, 1600+ PROJECTS, 14+ YEARS
11

•Former Head of BI/ Lead Architect –VISA
•Former BI Architect –JambaJuice
•Former Head of BI –Dole
•Former Chief BI Architect –Cisco
•Former BI Architect –Daimler AG
•Former Lead of IT Architecture –Paramount Pictures
•Former Head of BI –Experian
•Former Head of BI –Robert Half International
•Former Head of Training (IBM Cognos, Southern California)
•Former Controller –The GAP
•Two former CFO’s
•Several former Vice Presidents of Marketing
•Several former COO’s
•Several Former CIO’s
•Former Partner -PWC ($50million+ projects)
•Average experience = over 20 years
A Few of Our Team Members(former roles)
Deep & Pragmatic Experience
Copyright 2014 Senturus, Inc. All Rights Reserved.
12

WHATDOYOUUSEFORBI DATA“STORAGE” TODAY?
QUICKPOLL

DOYOUINTENDTODEPLOYANENTERPRISEDATAWAREHOUSEATSOMEPOINT?
QUICKPOLL

DOYOUREALLYNEEDADATAWAREHOUSE?
QUICKANSWER

The short is answer is:
Almost always, YES
DOYOUREALLYNEEDADATAWAREHOUSE*?
16
* or Conforming Data Marts

The rest of this presentation will focus on why…
WHY?
17

DATA-DRIVENINSIGHTLEADSTOBOTTOMLINERESULTS
WHYBUSINESSANALYTICS?

BUSINESS INTELLIGENCE DRIVES COMPETITIVE ADVANTAGE
Copyright 2014 Senturus, Inc. All Rights Reserved. 20
11.3%
14.0%
12.1%
0.5%
9.4% 9.3% Value Integrators
All other enterprises
EBITDA
5-year CAGR, 2004-2008
Revenue
5-year CAGR, 2004-2008
ROIC
5-year average, 2004-2008
> 20x 49% more 30% more more
Source: IBM Institute for Business Value, The Global CFO Study 2010

GETTINGTHERIGHTINFORMATIONTOTHERIGHTDECISIONMAKERSATTHERIGHTTIME
THECHALLENGE

SOURCEDATAISNOTACTIONABLEINFORMATION
22
Standard
Reports (Push-Pull)
Dashboards/
Scorecards
Self-service Reporting
& Ad-Hoc Analysis
Alerts
The Chasm ERP, CRM Data
Planning Data
Decisions & Actions
Source Systems of Record
Other Sources
“What do you want?”
“What do you have?”

THETYPICALSOLUTION
23
Standard
Reports (Push-Pull)
Dashboards/
Scorecards
& Ad-Hoc Analysis
AlertsERP Data
CRM Data
Planning Data
Decisions & Actions
Other Sources
Or more specifically….

THETYPICALSOLUTION* (DETAILED)
24
Standard
Reports (Push-Pull)
Dashboards/
Scorecards
& Ad-Hoc Analysis
AlertsERP Data
CRM Data
Planning Data
Decisions & Actions
Other Sources
* Often coupled with individual acts of Macro & VLOOKUP heroism, done infrequently and inconsistently
Excel
Powerpoint
Access

Solutions
Manually process in Excel
Combine multiple sources
Find, organize and align data
Filter non-relevant data
Calculate missing measures
Publish and distribute reports
Use BI Tools to
produce reports
(scheduled and on-demand)
Use ETL
to populate a mart/DW
(write once, run daily)
OR
But, most reports require business logic be applied to data
Problem
WHYITPAYSTOBUILDAAUTOMATEDBI SYSTEM
25
Save money, make money
“We just want a report”
Need
Repeat for everyreport, everymonth
Build OnceBuild Once

THEREALCHALLENGEINANUTSHELL
The Data has to be Transformedsomewhere between the source systems and the end-user
The question is simply –WHERE ?
1.By the End-User(In Excel, etc)
2.By the Front-end BI Tool (with live queries)
3.By an Intermediate process & staging area (ETL, DW)

BUTFIRST, SOMEBUSINESSINTELLIGENCEMUST-HAVES& GIVENS
BASICREQUIREMENTS

•Deliver a stable & user-friendly data structure
–Reports will not break if source system files change
–Foundation for true “Self-service” reporting and analytics
•Provide fast performance
–Especially for ad hoc reporting and interactive dashboards
•Handle multiplesources of data
–Cross-functional facts (metrics) and dimensions
•Deliver high quality, validateddata
•Maintain historicaldata in a common format
–Even if source systems change or grow
–Also, maintain historical context of data (SCD’s)
–Allows for trending and “as-of” analysis
A FEWUNIVERSALBI SYSTEMREQUIREMENTS
29

•Provide additional ways to “roll-up” data
–Hierarchies, attributes, defined metrics
•Provide field, table & measure names that make senseto business users
•Enable pre-calculationsfor commonly used measures
–E.gGross margin, ratios, special qualities (pounds, gallons, etc)
•Provide user & role based security
–Often different than authentication within OLTP environment
A FEWUNIVERSALBI REQUIREMENTS(CONT.)
30

WHYNOTSIMPLYPOINTTHEBI TOOLSATTHESOURCESYSTEMS?
ARCHITECTURALOPTIONS

DIRECTCONNECTIONTOSOURCESYSTEM
ERP Data
Labor Data
Standard
Reports Web Portal
Other Sources Ad hoc Querying
Planning Data Slicing & Dicing
Dashboard Authoring
Report Authoring
Dashboards/
Scorecards
Threshold
Alerting
& Analysis
Threshold-based
Alerts
Excel
Planning “Data Set”
Sales “Data Set”
Finance “Data Set” HR “Data Set”
Other “Data Set”
…

•Transaction processing (OLTP) systems are optimized for Data Entry, not Reporting
–Highly normalized, atomic level data
–Few indexes
–Cryptic naming (tables, columns)
–Odd formats (e.g. Julian dates, non-decimal numbers
–Priority often given to transaction processing
•OLTP systems change over time
–System upgrades, inducing structural changes
–System migrations
–Company acquisitions bring new sources
•OLTP systems not designed for rich metadata and hierarchies
–Limited fields and flex (UD) fields
–Little to no control over uniqueness of rollups
–Dimension maintenance is tedious at best
OTHERCHALLENGESOFDIRECTCONNECTION
33
Performance
Usability
Stability
Usability

•Reporting queries can adversely impact OLTP data entry
–Queries are often intensive
•OLTP systems lack historical data and context
–Deleted records
–Legacy data often lost
–Only current values stored
•OLTP systems not capable of storing data from other/all sources
–Despite claims, source systems are not good repositories of other system data
–Multiple sources often don’t have common keys, structures relationships, granularity, etc.
•OLTP system security typically does not match BI needs
–Additional users and roles
–Extra licenses
–Unnecessary (& risky) access and complexity
CHALLENGESOFDIRECTCONNECTIONORREPLICATION(CONT.)
34
PerformanceUsabilityUsability
Secure

OPERATIONALDATASOURCE: SAP EXAMPLE
* Just a few of the over 70,000 tables in SAP R/3

A FEWOTHERPROBLEMSWITHREPORTINGDIRECTFROMSOURCE(OLTP) SYSTEMS
MORECHALLENGES

ERP, plus…
CRM
Master data of all types
Plans, forecasts, budgets
Security data
POS or channel data
Shop floor (or equiv) data
3rdparty data
Big Data
…
THEREISALWAYSMORETHANONESOURCE
38

ONEPROBLEMWITHTRYINGTOREPORTACROSSSUBJECTS
39
Cross joins (between query subjects [Reporting View, DEPT BUDGET, Reporting View, DEPT EXPENSE] are not permitted in the identity.
Without conformed dimensions

Upgrades
Migrations
Re-implementations
Acquisitions
…
SOURCESYSTEMSCHANGEOVERTIME
40
Yet, good BI relies on historical data trending and context

My Big Epiphany:
Business Intelligence Success Hinges on Dimensional Data
Source systems NEVERsupport all the rollups, attributes & hierarchies
Rollups, attributes & hierarchies changeALL the time
ROLLUPS& HIERARCHIESMUSTBEADDED

Date & Time
Financial -Departments
Financial -Chart of Accounts
Product (often multiple)
Brand
Sales Territory
Customer
Employees/Management
Supplier
Asset
Geography/Location
Etc.
A FEWHIERARCHYEXAMPLES

Company reorgs
Multiple product hierarchies
Finance version vs. Marketing version
Sales territory realignment
Management hierarchies vs. geographic territories
Pre-and post-acquisition rollups
Multiple division rollup disparities
External supplier and third-party data hierarchies vs. internal
Temporary groupings (promos, tiger teams, etc.)
A FEWCLASSICEXAMPLESOFCHANGE

SOWHATDOWENEEDTODO…
TECHNICALSOLUTION

•Separate intensive queryand reporting tasks from servers & disks used by transaction processing (OLTP) systems
•Create data models and technologies optimized for query and reporting that are NOT appropriate for transaction processing.
–E.g. bit-mapped indexes, denormalizedtables…
•Transform data and embed “knowledge,” roll-ups and business logic into the data structures so that non-IT users can perform “self-service BI”
•Create a single locationwhere information from multiple source systemscan be accessed and combined for reporting purposes.
SOWHATDOWENEEDTODO(TECHINICALLY)
46

•Provide a validatedrepositoryof data that has been cleaned of inaccurate or spurious data quality issues.
•Maintain a repository of historical data gathered from prior and legacy sources, as well as data that would otherwise be purged from the current transaction processing system(s).
•Allow for secured accessto data for analytics without opening up access to systems where data might inadvertently be modified, or transaction processing performance hindered.
•Provide a stable platform upon which end-users can build customized reports, dashboards and analytics
–Regardless of source system gyrations over time
SOWHATDOWENEEDTODO… (CONT.)
47

Create a
Data Warehouse
INOTHERWORDS…
48

THEREALSOLUTION
1.Properly staged data
Extracted
Transformed
Enhanced & Combined
Validated
Delivered
2.Good tools to “consume” and use the information
Report
Monitor
Analyze

Properly Staged DataBI Tools
The Real Solution
50Copyright 2014 Senturus, Inc. All Rights Reserved.
Single Version of the Truth
Data Abstraction Model
Information Security
Report
Authoring
Dashboard
Authoring
Slicing &
Dicing
Ad Hoc
Querying
Threshold
Alerting
ERP
Data
Labor
DataOtherSourcesPlanningData
Standard
ReportsDashboards/ ScorecardsSelf-ServiceReporting & Analysis
Threshold-
based Alerts
Web Portal

WHATISADATAWAREHOUSE?
DEFINITIONS

“ A data warehouse is a subject oriented, integrated, nonvolatile, time variant collection of data in support of management's decisions"
Bill Inmon
Building the Data Warehouse
John Wiley & Sons, Inc., 1992
Classic Definition: Data Warehouse
52

WHATISNOT ADATAWAREHOUSE?
DEFINITIONS(PART2)

questions
here
Copyright 2014Senturus,Inc.AllRightsReserved
Hear the Recording
www.senturus.com

DATAWAREHOUSE= COPYOFSOURCESYSTEM?
ERP Data
Labor Data
Standard
Reports
Web Portal
Other Sources
Ad hoc Querying Planning Data
Slicing & DicingDashboard AuthoringReport Authoring
Dashboards/
Scorecards
Threshold
Alerting
& Analysis
Threshold-based
Alerts
ERP Data
“Warehouse”
Labor Data
“Warehouse”
Other Data
“Warehouse”
Planning Data
“Warehouse”
Excel
Planning “Universe”
Sales “Universe” Finance “Universe”
HR “Universe” Other “Universes” …
Replication

DATAWAREHOUSE= NON-INTEGRATEDDATAMARTSILOS?
ERP Data
Labor Data
Standard
Reports
Web Portal
Other Sources
Ad hoc Querying
Planning Data
Slicing & DicingDashboard Authoring
Report Authoring Dashboards/ Scorecards Source Systems of Record
Threshold
Alerting
& Analysis
Threshold-based
Alerts
Excel Spreadmart
Planning Datamart
Sales Datamart
Finance Datamart
HR Datamart
Other Datamarts
…
ETL Processes

Integrated DataBI ToolsTRUEINTEGRATEDDATAWAREHOUSE
57
* Also known as a Star Schema
Conforming
Business Process
Dimensional Models*
Single Version of the Truth
Data Abstraction Model
Information Security
Report
Authoring
Dashboard
Authoring
Slicing &
Dicing
Ad Hoc
Querying
Threshold
AlertingERPDataLaborData
Other
Sources
Planning
Data
Data Integration (ETL)
Standard
Reports
Dashboards/
Scorecards
Self-Service
Reporting &
Analysis
Threshold-
based Alerts
Web Portal

WHYINTEGRATEDDATAMARTSORWAREHOUSE?
Enhances Reporting Performance and Flexibility
–Data Marts are organized as denormalizeddata structures for speed and ease of Reporting vs. Transactional system.
–Offloads the transactional system of reporting requests
–Drill-to-detail (regardless of data location)
Enables Data Integration or Cross Business Analysis
–Enables analysis across business processes and functional areas
–Allows data from multiple sources to be integrated into one source of truth with common dimensionality (GL, Planning, Payroll, Sales)
–Discussion of conformed dimensions
–Example: Budget vs. Actuals
Allows HistoricalData and TrendAnalysis
–Captures historical perspective vs. snapshot in time. (ex.Sqft)
–Allows shifts in sources systems seamlessly

WHYINTEGRATEDDATAMARTSORWAREHOUSE? (CONTINUED)
4.Allows for Automation of business rules & transformations to human-readable information
–Insulates Business Users from cryptic structuresand changes in the source systems
–Discussion of Transformation Layers
5.Allows for Additional org/hierarchy rollups & groupingsnot provided by source systems
–ALWAYS needed, never 100% supported by sources
–Should be table drivenManual EffortVlookups
Fragile
Macros

WHYINTEGRATEDDATAMARTSORWAREHOUSE? (CONTINUED)
6.Flexible Architectures enables reporting flexibility, i.e. the right tool for the right job
–Robust Reports for operational needs (plus, automatic delivery)
–Cubes for analytics, what if and scenarios
–Ad Hoc Reporting
–Dashboards & Scorecards for Management
7.Empowers Business User Self Service through any of the avenues from above
–Provides the ability to drill into the “Why?”

SPECIFICEXAMPLESOFWHENA DATAWAREHOUSEADDSVALUE
BENEFITS

Aggregation
Pre-calculation
Fewer joins
Simple joins
Incremental loads
Indexing and Optimization for DW
Less logic at the reporting layer
PERFORMANCEENHANCEMENTEXAMPLES
63

Nomenclature transformation
Both measures and dimensions
Lookups -Measures (e.g. cost)
Lookups –Dimensions (rich master data)
Date conversions
“Pre-computed” Date logic
Granularity matching (e.g. plan vs. actual)
Business logic application
e.g. Definitionof Revenue
USABILITY& VALIDITYENHANCEMENTEXAMPLES

ENABLEQUERIESACROSSSUBJECTAREAS
Product
ProductType
ProductCategory
ProductClass
SuperBallpoint Pen
Ballpoints
Pens
Education
Metal Writer Pen
Ballpoints
Pens
Business
Felt Great
Felt Tips
Markers
Education
Product
Supplier
MaterialType
Product Category
Product Class
Super Ballpoint Pen
Acme
Ballpoints
Pens
Plastic
Metal WriterPen
XYZ
Ballpoints
Pens
Metal
Felt Great
Acme
Felt Tips
Markers
Hybrid
Marketing’s Product Dimension table:
Manufacturing’s Product Dimension table:
Product
Product Type
Product Category
Marketing Product Class
Manufacturing ProductClass
Supplier
Super BallpointPen
Ballpoints
Pens
Education
Plastic
Acme
Metal WriterPen
Ballpoints
Pens
Business
Metal
XYZ
Felt Great
FeltTips
Markers
Education
Hybrid
Acme
ConformedProduct Dimension table:
Before
After

Maintaining accurate historical context (SCD’s)
Snapshots and balances (e.g. inventory)
TransactionlessFacts (e.g. promo periods)
Trending
EXAMPLESOFSPECIALCHALLENGES

ACCURATELY MAINTAIN HISTORICAL CONTEXT
2009
2010
Store #23:
8,000 sq ft
Store #23:
20,000 sq ft
Year Store ID Store
Size
Revenue Rev/
sq ft
2009 23 8,000 $500,000 $63
2010 23 20,000 $1,300,000 $65
Year Store ID Store
Size
Revenue Rev/
sq ft
2009 23 20,000 $500,000 $25
2010 23 20,000 $1,300,000 $65
Accurate Historical Context:
Report that uses Store Size attribute from ERP table:
Store gets remodeled
Incorrect !

Rich, built-in date functionality (MTD,QTD…)
Pre-calculatedtime intervals, and other derived metrics
Data-driven security
Reduced licensing costs
And lots more….
A FEWMOREEXAMPLES

HOWTOBUILDADATAWAREHOUSE--A20,000 FOOTVIEW
FINALTIP

DATAWAREHOUSESWITHOUTTHENEGATIVES
•Recommendation: Don’t set out to build a data warehouse [i.e. “Boil the ocean”]
•Instead, build a series of business process dimensional models with conformed dimensions
•The result will provide the benefits of a data warehouse without you ever having done a data warehouse project.

Business Process Dimensional ModelsDateTime StoreProduct, etc.QtyExt Cost Ext Amount Margin,.
Dimensions (Attributes)
Measures (Metrics)
Store Key
Store ID
Store Name
Store Loc
Store Region
Store Size
Store Age
… Product IDProduct NameProduct ClassProduct LineProduct WeightShipping Cost…
Date
Year
Quarter
Month
Week
Day
Day name
…
Added (Rolled-Up)
Averaged
Calculated
…

CONFORMEDDIMENSIONS= KEYTOINTEGRATION
DateTime StoreProductQty,Revenue, Gross Margin
Dimensions (Attributes)
Measures (Metrics)
Store ID
Store Name
Store District
Store Region
Store Mgr
Store Age
… Product IDProduct NameProduct ClassProduct LineProduct WeightShipping Cost…
Date
Year
Quarter
Month
Week
Day
Day name
… Added (Rolled-Up) AveragedCalculated…
DateTime StoreProductPlanQty,Plan Rev, Plan Margin

Product
Line
Measures / Facts
Amount, Quantity
Units = 10
Amount = $17,525
Cost = $8,000District
Store
Product (SKU)
Product Subclass
Channel
Calculations & Consolidations
Margin, Roll-ups…
Quarter
YearMonth
WeekDay
Period 1
Versions
Scenarios
Actuals
Forecast
MTDQTD
YTD
WTD
Season
Period 2
Product Class
Territory
Sales RepOld Region
New Region
DIMENSIONALMODEL(SIMPLIFIED)

We agree that some Additional Repository (other than the source system) is needed.
We agree that some Transformationsshould be done once during ETL, not live in every query.
We agree that Rich Dimensionality and Transformation adds tremendous value to data.
We agree that it is critical to lay the Proper FoundationBEFOREyou start building tons of reports, etc.
CLOSINGARGUMENTS
79

Therefore, we agree that:
We need a RealIntegrated Data Warehouse
We need to do it Right
We can & should build it Incrementally
And we need to do it Now
CLOSINGARGUMENTS
80

FROMIBM ANDSENTURUS
ADDITIONALRESOURCES

PRESENTATIONSLIDEDECKONWWW.SENTURUS.COM
82

www.senturus.com/events
•Sept 11Beginning Authoring Tips & Tricks in Cognos BI
•Sept 17Houston Cognos Users Group
•Sept 18Improving the Planning Cycle for Sophisticated Business Needs
UPCOMINGEVENTS
83

Save $400
By registering before October 3rdand using
Senturus’ preferred customer code
G14SNTURUS, you’ll get an additional $100
off the early bird discount for a total of $400
savings.
PLUS $200 Training Credit
Receive $200 in Senturus training credit, good toward any of our 20 live instructor-led Cognos online training classes.
PLUS $25 in Gambling Chips
At Senturus, we’re betting on you! Registrants will enjoy a welcome gift of $25 in Mandalay Bay Hotel gambling chips upon arrival.
Save $600 on IBM Insight (formerly IOD)
84
http://www.senturus.com/ibm-insight-iod-2014/

*Custom, tailored training also available*
COGNOSTRAININGOPTIONS
85

questions
hereCopyright 2014Senturus,Inc.
AllRightsReserved
Hear the Recording
www.senturus.com

Thank
You!!
www.senturus.com
888-601-6010
info@senturus.com
Copyright2014bySenturus,
Inc.
Thisentirepresentationiscopyrightedandmaynotbereusedor
distributedwithoutthewrittenconsentofSenturus,Inc.

Do You Really Need a Data Warehouse? Avoid the Downsides Typically Associated with Data Warehouses

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Do You Really Need a Data Warehouse? Avoid the Downsides Typically Associated with Data Warehouses

Ähnlich wie Do You Really Need a Data Warehouse? Avoid the Downsides Typically Associated with Data Warehouses (20)

Mehr von Senturus

Mehr von Senturus (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Do You Really Need a Data Warehouse? Avoid the Downsides Typically Associated with Data Warehouses