Retour d'expérience sur un projet de Business Intelligence réalisé à l'EVAM selon une méthodologie Agile et avec un modèle de données Data Vault. Présentation faite lors du Swiss Data Forum du 24 novembre 2015 à Lausanne
2. Plan
• Introduction ( F. Kang à Birang)
• Pre-project (F. Kang à Birang & J-M. Delacrétaz)
• Agile project management (A. Martino)
• Agile architecture (E. Fidel)
• Data quality (A. Martino)
• EVAM Feedback (B. Albietz)
10. Agility
We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
14. Normal Process for a B.I. need
Business
Analysis
Design of the
model
Implementation
Unit Testing
Volume
testing
User
Acceptance
Testing
New
Need
Rework
Rework Rework
Rework
Deployment
to Validation
Deployment Production
18. Agile Objectives
• Adapt to change
• Deliver working software frequently
• At regular intervals, the team reflects on how
to become more effective
• Work close to business
24. What is Data Vault ?
• Data Modelling Method for Data Warehouses in Agile Environments
• Developed by Dan Linsted
• Suitable for
• DWH Core Layer
• Optimized for
• Agility / Integration /
Historization
25. Data Vault composition
• Decomposition of Source Data
• Split Data into Separate Parts
Hubs Business Entity
Links Relations
Satellites Contexts
Business Oriented
26. Data Vault composition
• Elements : Hub – Link – Sat
Customer
Sat
Sat
Sat
Customer Product
Sat
Sat
Sat
Product
Hub = List of Unique Business Keys
Link = List of Relationships, Associations
Satellites = Descriptive Data
Order
Sat
Sat
Sat
Order
Link
27. Avantages and challenges
• Standard ETL Rules to Load Data Vault
• Easy Extensibility of Data Vault Model
• Integration of Multiple Source Systems
• Traceability and Complete History
• High Number of Tables in Data Vault
28. What does the Data Vault generator do ?
• Tables
• Indexes
• Surrogate keys
• Foreign keys
• Partitions
• Loading process
• SCD1 / SCD2
• Loading audits
• Handling Errors
29. Generator value
29
Business spec
Technical spec
Development
Test
Deployment
Qualityassurance
Documentation
Simplify
Generator
Documentation
QS
Total Savings
Fast and short implementation cycles
Broad flexibility of change
Auto-generated quality assured components
Huge time and cost savings
On-going and recurrent with each
step of modification or enlargement!!!
33. Data Mart
• Business Need Oriented
• Virtualized DM (materialized view)
• Can be regenerated from scratch
• Find value at a point in time
• Good perfomance
• Automatically regenerated (no deployment)
37. Keys Learnings
• Show business value as early as possible and keep the ball rolling
• Project: December 2014 – June 2016
• Phased implementation: 1st output in June 2015, then regular outputs
on a monthly basis
• Be prepared to spend most of your time on data quality
• The lifeblood of B.I. projects
38. Keys Learnings
• Prepare knowledge transfer to your staff during the project
• Modelling, ETL, Reporting
• Good project management practice, from business requirements to
report development
• Increase user buy-in with Scrum
• Key users and management involved from day 1
39. Keys Learnings
• Learn to say “ No ”
• B.I. quality versus business process quality
• B.I. is also here to show process deficiencies, do not try to solve all
business issues within the B.I. project