We recently presented our technology solution for metadata discovery to the Boulder Business Intelligence Brains Trust in Colorado. (www.bbbt.us)
The whole session was also videod and there is a link to the recording at the end of the presentation.
2. Roland Bullivant
Sales and Marketing Director
rbullivant@silwoodtechnology.com
@rolandatsilwood
www.silwoodtechnology.com
Nick Porter
Technical Director
nporter@silwoodtechnology.com
4. Perspective
“Our management team is becoming inseparable from the technology which supports it.”
Paul Allaire, President, Xerox Coporation, The EIS Report, 1989, Business Intelligence
EIS/DSS
Financials
Manufacturing
Distribution
Sales
News
Market Data
5. What is our one thing?
Image courtesy of Hortonworks.com
“Where’s the data?”
SAP, Salesforce, Oracle
etc..
VENDORTOOLSPROLIFERATE
6. The Big Idea
“Metadata for the masses”
“The Google for SAP metadata”
“GPS for application packages”
“90,000 tables on your laptop”
“Discover - Scope - Deliver”
7. Perhaps it is easier to show you..
Quick demonstration
General ledger accounting
8. Agenda
• Silwood Technology
• Why are we here?
• Our market
• Background
• What is Safyr?
• Case studies
• Demonstration
• Wrap up and close
9. Silwood Technology
• UK based
• Privately held
• Data modelling (ERwin)
• Developed Safyr
• Major partners
• World class customers
• Continuous development
12. Why are we here?
• Visibility
• Education
• Feedback
13. Our market
• Increase value from
applications
• IT challenged with
data complexity
• SAP, Salesforce and
Oracle applications
14. New Low Latency world
IN MEMORY / BIG DATA / HADOOP / DATA LAKES
• Real time data
• Faster analytics
• Faster ERP/CRM etc
Source Data Intelligence
• Same challenge
• Delays have more impact
• Cannot wait for consultants
“The biggest internal debates so far have been around where we source the data
from and how we do integrated data modeling,” says Brian Raver, IT Manager of BI
Strategy and Systems Architecture at Medtronic. “Even though SAP HANA is a
high-performance appliance, you still have to think about the optimal way to model
the data.”
15. Why are these applications so challenging?
• Large
• Complex
• Customised
• Specialists only
• ‘Invisible’ data model
“The data in these (ERP) systems makes sense and are useful, but only in the context of the hard-coded processes. In
short, the data is trapped inside a complex web of thousands of database tables whose integrity is solely controlled by
a rigid fossilized collection of software algorithms. If you don’t believe me, just ask your SAP support staff for access to
directly update (or even read) a data table.”
John Schmidt (vice president of Global Integration Services at Informatica Corporation)
16. Barry Devlin
“..as any data warehouse manager
will confirm from bitter experience
the biggest technical challenge they
face is in understanding the source
systems for the warehouse,
extracting the data from them and
building a consistent set of
information from the combined
sources”
Barry Devlin (2011)
Data Warehouse Design Redux
17. Claudia Imhoff
“Another best practice for getting
started is to start with the database
schema of the existing operational or
transaction (source) systems. It is
possible to convert these designs
into technology and system models.
These can in turn be used as a
starting point for the enterprise data
model and subject area model.”
Claudia Imhoff (January 2010)
Fast-tracking Data Warehouse and
Business Intelligence Projects via
Intelligent Data Modelling
18. Quote from Hydro Tasmania
“The team was originally
informed that no data model
was available for the SAP
application or for SAP BW”.
Scott Delaney
BI Team Leader
Hydro Tasmania
19. Implications of not understanding data model in context of project
• Delay in benefits
• Late or under delivery
• Increased risk
• Over budget
• Loss of trust
20. “Where’s my data?”
Typical environment
• 000’s tables (but only
need a few)
• Complex relationships
(how are tables joined?)
• Descriptions
• Customisations
“How do I quickly and accurately find the right tables needed for my project?”
21. Safyr summary
• Extracts metadata
• Easy search and filter
• Visualise models
• Metadata in context
• 3rd Party export
22. Packaged Application Metadata: How do most companies do it now?
• Read documentation
• Ask technical specialists
• Ask consultants
• Re-key into spreadsheets
• Informed guesswork
• Internet search
• Use modelling tool
• Expect vendor to provide
23. Typical vendor approaches
• Interface to get data
– Connectors
– Templates
– Lists
• Inadequate context
David Marco
EDW 2015
24. You can try reverse engineering database with modelling tool
26. Quote from AMD
“After doing a quick prototype
metadata extract from SAP, the
response has been very
positive!
I’m really grieving for the lost
years without access to this
tool. It has met and exceeded
my lofty expectations.”
Brian Farish
IT Architecture Manager
AMD
27. Safyr approach
Automates rapid harvesting and
discovery of metadata including
customisations
Powerful scoping and
introspection tools usable by data
specialists
Fast and easy integration with 3rd
party tools
28. Safyr – single source of trusted application metadata
Export
results of
scoping
SAP Business Suite
SAP BW
SAP Business Suite on HANA
Safyr™
Metadata
Discovery
Modelling
Data
Warehouse
Data
Integration
Metadata
management
Master Data
Management
Oracle eBusiness Suite
PeopleSoft
Siebel
JD Edwards EnterpriseOne
Source Applications Extract, discover and export
Other Packaged Applications
Salesforce (and Force)
29. Safyr main features
Reverse engineers application metadata (inc. customisations)
Finds all tables, fields, view, descriptions (logical AND physical)
Automatically discovers all relationships and Application module
hierarchy
Search, filter, navigate
Compare (complete applications or individual subject areas)
Visualise as models for easier understanding & communication
Export to modelling, metadata management, integration and others
Pre-configured Subject areas for SAP, JD Edwards, Siebel, Oracle EBS
“ETL for Metadata” supports other packages (eg Dynamics)
Rapid – extraction < 3 hours, analysis in days not months
Accurate – works with system as implemented
31. Customer return and value from rapid source metadata discovery
• Faster project delivery
• Manage/reduce costs
• Higher productivity
• Accuracy of deliverables
• Fewer surprises during
project
32. Case study – Oil company
Challenge
JD Edwards EnterpriseOne
Replacing SAP
Customisations
Operational reporting
Under time pressure
‘Discovery’ bottleneck
Solution - Safyr™
Accelerate development
Meeting deadlines
Rapid implementation (hours)
Used by data architects
Automated discovery
Models for OBIEE
33. Case study – RS Components
Challenge
SAP
Heavily customised (117k)
Individual project delays
Reporting
Integration
‘Discovery’ bottleneck
Obstructs understanding
Reduces IT effectiveness
Hinders communication
Solution - Safyr™
Project deadlines met
Rapid implementation (days)
Better understanding of SAP
No guesswork
Enhanced communications
Additional uses:
JD Edwards to SAP migration
34. Summary from RS Components
RS are succeeding in achieving a level of understanding of data in SAP that
we previously thought impossible
We have quickly assembled a set of detailed subject area data models
which we can now use to guide project activities. The Safyr models deliver
a level of detail that we would not otherwise be able to achieve without
extensive user research (and a large helping of guesswork)
We have high confidence in the detail in each model as it is coming directly
from SAP itself
Based on the success of the Safyr option for SAP, we are looking to assess
the Safyr option for JD Edwards to accelerate the data mapping and
migration process for our SAP rollout to Asia
35. Case Study - Hydro Tasmania
• New SAP and BW
• New DWH and BI
• “No SAP data model”
• Reduced productivity
• Business losing faith
• Safyr for SAP data
model
• Rapid implementation
• Quick learning curve
• Back on track
• No backlog
“As a result of our investment in Safyr we are able to take a more agile approach
to meeting the demands for new reports and data within acceptable timescales
and the business’ trust in the information provided is growing”
Scott Delaney,
Hydro Tasmania
36. Case study – Global Semi-conductor maker
• Situation
– Multiple SAP instances
(30+)
– Customisations
• Global datawarehouse
– BW as staging area for
Teradata
• Application
consolidation
• ‘Discovery’ bottleneck
• Solution - Safyr™
– All reversed engineered in 1
month
– Understanding SAP and BW
– Huge productivity gain
• Was 4 staff for a month to find
transaction tables
• Now 1 person for a week
– Enhanced communications
– Time/cost saving
– Project delivery
38. • It’s all about Scoping
• There are thousands of tables, but probably only interested
in a few 10s or 100s – but which?
• Metadata discovery is the key
– ‘scope’ the required tables
– Then visualize as a data model
• Utilize metadata in project EIM tools
How to identify ‘required’ tables and relationships?
38
39. • Need to make ‘Subject Areas’ relevant to the task
• Relationships really help
– Give context to a table
– Provide an important means to find tables that are ‘in
scope’
• Seeing tables in the context of function
– Which tables are used by a program, or component?
Divide and Conquer
39
40. Want to find data behind key Business Concepts
o Manufacturing
o Shipping to Warehouse
o Customer Orders
o Bill of Materials
o Invoicing
o Payments
o Returns
o Customer Master
o Vendor Master