Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
#Centennium Presentation Data Vault Automation seminar #dvauto Oct 6th 2011
1. CDM Centennium Datawarehouse Methodology Structure – Model – Generate – Knowledge Partnership 1 Erik Fransen Antoine Stelma October 6, 2011
2. Agenda 2 Centennium BI expertisehuis CDM Structure Reference Architecture Model Data Vault Generate Template Based Development Knowledge Partnership Knowledge transfer, coaching CDM process
3. Centennium BI expertisehuis Independent We provide our customers with the right skills, at the right time and the right way Consultancy, ad-interim support, project implementation and training services, but primarily acts as a BI/DWH knowledge partner for its clients We take or share responsibility for the execution and management of BI and DWH projects and provide second or third line support for existing implementations We offer an extensive portfolio of courses and training services We provide our customers with the knowledge and practical insights required to be(come) self-sufficient in maintaining and expanding their BI-environments www.centennium.nl 3
4. Centennium Datawarehouse Methodology A methodology, including DWH-tools Build or migrate datawarehouses fast, with high qualityand low cost Automatic generation of datawarehouse based on descriptivemetadata CDM includes: Template Based Development Best practices Quality control mechanism Knowledge partnership Best Practices: Data Vault, Kimball, Reference Architecture Qualitycontrol mechanism: extensive checklists anddocumentation Knowledge transfer by training, certificationandlearning on-the-job 4
6. Knowledge partnership Goal: Self-supporting, CDM-awarecustomers Training options BI and DWH fundamentals CDM designer / TBD developer training Data Vaultfundamentals Data Vaultcertification(Genesee Academy) Dimensionalmodelling More info on www.bi-opleidingen.nl On-the-job coaching, learningbydoing Centennium supports it’scustomersby(management) consultancy, assessments, projects, training andsourcing 6
8. Template Based Development:Generating TBD generatesall ETL and objects for Registrationlayer Presentation/Publicationlayer Repositoryand scripts are free of charge Creating the staginglayer is not part of TBD but canbeautomated per customer 8
9. TBD databases 9 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
10. TBD repository 10 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
11. Functionality TBD repositorylogtables (1) Functionality: Registration of all ETL processes Insight in data logistics Historic profile of loadingprocesses Goal: Traceability Maintenance: localisationandidentification of all data/transactions (wrong andcorrect) Linkage to source systems 11 REP PUB STA CDW
12. Functionality TBD repositorytemplate tables Functionality: Central storage of template scripts forgeneratingtablesand ETL procedures Easy promotion to other environments (OTAP) Transparant Goal: Version management of templates Object (re-)creationand maintenance 12 REP PUB STA CDW
13. Functionality TBD repositorymeta data tables Functionality: Definesregistrationandpublicationlayer Contains object and ETL definitions Contains data logistics Goal: Generating: Registrationlayer(Data Vaulttables) Publicationlayer (Star schemes) ETL processes via ‘stored procedures’ Object creationand maintenance 13 REP PUB STA CDW
14. Functionality TBD repositorymeta data tables Table ‘Repos_Table’: Contains object namesfor Staging Registration Publication Table is filledbyanapplication or Excel sheet Table ‘Repos_Mapping’: Contains mapping of staging, registration and presentation Table is filledbyanapplication or Excel sheet 14 REP PUB STA CDW
16. Registrationlayer Script ‘CDW 01a table hub’ Generatesall hub tables in the CDW, as defined in repository Every hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Hubs generated according to Data Vault Standards! 16 REP PUB STA CDW
17. Registrationlayer Script ‘CDW 01a tabel hub’ Generatesall hub tables in the CDW, as defined in repository Every hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Hubs generated according to Data Vault Standards! 17 REP PUB STA CDW
18. Registrationlayer Script ‘CDW 02a table sat’ Generatesallsatellitetables in the CDW, as defined in repository Everysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Satsgenerated according to Data Vault Standards! 18 REP PUB STA CDW
19. Registrationlayer Script ‘CDW 02a tabel sat’ Generatesallsatellitetables in the CDW, as defined in repository Everysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Satsgenerated according to Data Vault Standards! 19 REP PUB STA CDW
20. Registrationlayer Script‘CDW 03a table link’ Generatesalllink tables in the CDW, as defined in repository Everylink is connected to two or more hubs Variables fromrepository are applied Once the links are generated, the repository is updated Links generated according to Data Vault Standards! 20 REP PUB STA CDW
21. Registrationlayer Script‘CDW 03a table link’ Generatesalllink tables in the CDW, as defined in repository Everylink is connected to two or more hubs Variables fromrepository are applied Once the links are generated, the repository is updated Links generated according to Data Vault Standards! 21 REP PUB STA CDW
22. Registrationlayer Script‘CDW 04a table link satellite’ Generatesalllink satellitetables in the CDW, as defined in repository Variables fromrepository are applied Once the link satellites are generated, the repository is updated Link satellites are generated according to Data Vault Standards! 22 REP PUB STA CDW
23. Registrationlayer Script‘CDW 04a table link satellite’ Generatesalllink satellitetables in the CDW, as defined in repository Variables fromrepository are applied Once the link satellites are generated, the repository is updated Link satellites are generated according to Data Vault Standards! 23 REP PUB STA CDW
25. Publicationlayerdimensionsandfacts Fedonlyanddirectlyfrom CDW Conforms to Kimball’sstar scheme standards Or anyother format Business rulescanbeapplied Currentlyusing views Business rule editor in next release Type SCD I, II, etc. Conformeddimensionswhenneeded 25 REP PUB STA CDW
40. EducationSome of our clients: Woonbron, Albron, NZa, CAK, OBR, Vopak, several Dutch Municipalities, Aegon, Nutreco, TNO,Genzyme, Tata Steel, KPN, DELTA, IKEA, Accell, TomTom, KAS BANK, LeasePlan, Brabant Water
41. Certification and Industry Recognition All Centennium consultants are Certified BI Professionals (CBIP) awarded by TDWI. CBIP, backed by TDWI's reputation as the leader in BI education and research, makes a strong statement that we are among the leaders in the industry—up to date with BI technologies, knowledgeable about best practices and state-of-the-art solutions, and informed of emerging trends All Centennium Data Vault specialists and architects are certified on Data Vault modeling by Dan Linstedt, Data Warehouse industry thought leader and founder of the Data Vault modeling standard. Centennium is the business partner of Genesee Academy in organizing the Data Vault Modeling & Certification – Europe Course. We are accredited to perform the Genesee Academy Data Vault trainings. Centennium’s customer Nutreco was awarded the world’s first Certified Data Vault Data Model in 2009 for their corporate financial and procurement data warehouse model.