SlideShare ist ein Scribd-Unternehmen logo
1 von 33
CDM Centennium Datawarehouse Methodology Structure – Model – Generate – Knowledge Partnership 1 Erik Fransen Antoine Stelma October 6, 2011
Agenda 2 Centennium BI expertisehuis CDM Structure		Reference Architecture Model			Data Vault Generate		Template Based Development Knowledge Partnership		Knowledge transfer, coaching CDM process
Centennium BI expertisehuis Independent  We provide our customers with the right skills, at the right time and the right way Consultancy, ad-interim support, project implementation and training services, but primarily acts as a BI/DWH knowledge partner for its clients We take or share responsibility for the execution and management of BI and DWH projects and provide second or third line support for existing implementations We offer an extensive portfolio of courses and training services  We provide our customers with the knowledge and practical insights required to be(come) self-sufficient in maintaining and expanding their BI-environments www.centennium.nl 3
Centennium Datawarehouse Methodology A methodology, including DWH-tools Build or migrate datawarehouses fast, with high qualityand low cost Automatic generation of datawarehouse based on descriptivemetadata CDM includes: Template Based Development Best practices Quality control mechanism Knowledge partnership Best Practices: Data Vault, Kimball, Reference Architecture Qualitycontrol mechanism: extensive checklists anddocumentation Knowledge transfer by training, certificationandlearning on-the-job 4
5 Centennium Datawarehouse Methodology Knowledge Partnership Modelling Structuring Generating
Knowledge partnership Goal:  Self-supporting, CDM-awarecustomers Training options	 BI and DWH fundamentals CDM designer / TBD developer training	 Data Vaultfundamentals Data Vaultcertification(Genesee Academy)	 Dimensionalmodelling More info on www.bi-opleidingen.nl On-the-job coaching, learningbydoing Centennium supports it’scustomersby(management) consultancy, assessments, projects, training andsourcing 6
Reference Architecture: Structuring 7
Template Based Development:Generating TBD generatesall ETL and objects for Registrationlayer Presentation/Publicationlayer Repositoryand scripts are free of charge Creating the staginglayer is not part of TBD but canbeautomated per customer 8
TBD databases 9 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
TBD repository 10 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
Functionality TBD repositorylogtables (1) Functionality:  Registration of all ETL processes Insight in data logistics Historic profile of loadingprocesses Goal: Traceability Maintenance: localisationandidentification of all data/transactions (wrong andcorrect) Linkage to source systems 11 REP PUB STA CDW
Functionality TBD repositorytemplate tables Functionality:  Central storage of template scripts forgeneratingtablesand ETL procedures Easy promotion to other environments (OTAP) Transparant Goal:  Version management of templates Object (re-)creationand maintenance 12 REP PUB STA CDW
Functionality TBD repositorymeta data tables Functionality:  Definesregistrationandpublicationlayer Contains object and ETL definitions Contains data logistics Goal: Generating: Registrationlayer(Data Vaulttables) Publicationlayer (Star schemes)  ETL processes via ‘stored procedures’ Object creationand maintenance 13 REP PUB STA CDW
Functionality TBD repositorymeta data tables Table ‘Repos_Table’:  Contains object namesfor Staging Registration Publication Table is filledbyanapplication or Excel sheet Table ‘Repos_Mapping’:  Contains mapping of staging, registration and presentation Table is filledbyanapplication or Excel sheet 14 REP PUB STA CDW
Registrationlayer 15 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
Registrationlayer Script ‘CDW 01a table hub’ Generatesall hub tables in the CDW, as defined in repository Every hub contains the columns:  id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Hubs generated according to Data Vault Standards! 16 REP PUB STA CDW
Registrationlayer Script ‘CDW 01a tabel hub’ Generatesall hub tables in the CDW, as defined in repository Every hub contains the columns:  id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Hubs generated according to Data Vault Standards! 17 REP PUB STA CDW
Registrationlayer Script ‘CDW 02a table sat’ Generatesallsatellitetables in the CDW, as defined in repository Everysatellitecontains the the columns:  h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Satsgenerated according to Data Vault Standards! 18 REP PUB STA CDW
Registrationlayer Script ‘CDW 02a tabel sat’ Generatesallsatellitetables in the CDW, as defined in repository Everysatellitecontains the the columns:  h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Satsgenerated according to Data Vault Standards! 19 REP PUB STA CDW
Registrationlayer Script‘CDW 03a table link’ Generatesalllink tables in the CDW, as defined in repository Everylink is connected to two or more hubs Variables fromrepository are applied Once the links are generated, the repository is updated Links generated according to Data Vault Standards! 20 REP PUB STA CDW
Registrationlayer Script‘CDW 03a table link’ Generatesalllink tables in the CDW, as defined in repository Everylink is connected to two or more hubs Variables fromrepository are applied Once the links are generated, the repository is updated Links generated according to Data Vault Standards! 21 REP PUB STA CDW
Registrationlayer Script‘CDW 04a table link satellite’ Generatesalllink satellitetables in the CDW, as defined in repository Variables fromrepository are applied Once the link satellites are generated, the repository is updated Link satellites are generated according to Data Vault Standards! 22 REP PUB STA CDW
Registrationlayer Script‘CDW 04a table link satellite’ Generatesalllink satellitetables in the CDW, as defined in repository Variables fromrepository are applied Once the link satellites are generated, the repository is updated Link satellites are generated according to Data Vault Standards! 23 REP PUB STA CDW
Publicationlayer 24 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
Publicationlayerdimensionsandfacts Fedonlyanddirectlyfrom CDW Conforms to Kimball’sstar scheme standards Or anyother format  Business rulescanbeapplied Currentlyusing views Business rule editor in next release Type SCD I, II, etc. Conformeddimensionswhenneeded 25 REP PUB STA CDW
CDM: process Incremental approach Timeboxes of 4-6 weeks 26
CDM: typical increment 90% Centennium 30% Customer 70% Customer40% Centennium 100% Customer10% Centennium 100% Centennium Phase 1 Phase 2 Phase 3 Phase 4 Increment 31-12-2011 1-8-2011 Supporting customer Training and Coaching on-the-job CDM-training Data Vault Certification ,[object Object]
Centennium role changes from LEAD to FOLLOW
Customer is fully CDM-aware at the end of the increment
Centennium continuessupportingcustomersthroughknowledge partnership,[object Object]
About centennium 29
Centennium BI expertisehuis houses all the experts under one roof, hereby offering all knowledge and expertise to address the complex business intelligence issues facing our clients today Facts and figures: ,[object Object]
45+ business intelligence consultantsCore values: ,[object Object]

Weitere ähnliche Inhalte

Was ist angesagt?

Datawarehousing and Business Intelligence
Datawarehousing and Business IntelligenceDatawarehousing and Business Intelligence
Datawarehousing and Business Intelligence
Prithwis Mukerjee
 
Data warehouse 101-fundamentals-
Data warehouse 101-fundamentals-Data warehouse 101-fundamentals-
Data warehouse 101-fundamentals-
AshishGuleria
 
White Paper - Data Warehouse Documentation Roadmap
White Paper -  Data Warehouse Documentation RoadmapWhite Paper -  Data Warehouse Documentation Roadmap
White Paper - Data Warehouse Documentation Roadmap
David Walker
 
Bi presentation to bkk
Bi presentation to bkkBi presentation to bkk
Bi presentation to bkk
guest4e975e2
 
Oracle - Next Generation Datacenter - Alan Hartwell
Oracle - Next Generation Datacenter - Alan HartwellOracle - Next Generation Datacenter - Alan Hartwell
Oracle - Next Generation Datacenter - Alan Hartwell
HPDutchWorld
 

Was ist angesagt? (20)

Horizons 2014 - Enterprise Solutions
Horizons 2014 - Enterprise SolutionsHorizons 2014 - Enterprise Solutions
Horizons 2014 - Enterprise Solutions
 
Enterprise Services Solutions
Enterprise Services SolutionsEnterprise Services Solutions
Enterprise Services Solutions
 
Lower Cost and Complexity with Azure and StorSimple Hybrid Cloud Solutions
Lower Cost and Complexity with Azure and StorSimple Hybrid Cloud SolutionsLower Cost and Complexity with Azure and StorSimple Hybrid Cloud Solutions
Lower Cost and Complexity with Azure and StorSimple Hybrid Cloud Solutions
 
Agile NoSQL With XRX
Agile NoSQL With XRXAgile NoSQL With XRX
Agile NoSQL With XRX
 
Data Warehouse Methodology
Data Warehouse MethodologyData Warehouse Methodology
Data Warehouse Methodology
 
Pluto7 meetup v2
Pluto7    meetup v2Pluto7    meetup v2
Pluto7 meetup v2
 
Datawarehousing and Business Intelligence
Datawarehousing and Business IntelligenceDatawarehousing and Business Intelligence
Datawarehousing and Business Intelligence
 
Data warehouse 101-fundamentals-
Data warehouse 101-fundamentals-Data warehouse 101-fundamentals-
Data warehouse 101-fundamentals-
 
Business Intelligence Architecture
Business Intelligence ArchitectureBusiness Intelligence Architecture
Business Intelligence Architecture
 
Bi&dw methodology
Bi&dw methodologyBi&dw methodology
Bi&dw methodology
 
Oracle Enterprise Staffing Solutions
Oracle Enterprise Staffing SolutionsOracle Enterprise Staffing Solutions
Oracle Enterprise Staffing Solutions
 
BICC Conceptual Overview
BICC Conceptual OverviewBICC Conceptual Overview
BICC Conceptual Overview
 
A #Pink14 Presentation: Optimizing for the #SDDC
A #Pink14 Presentation: Optimizing for the #SDDCA #Pink14 Presentation: Optimizing for the #SDDC
A #Pink14 Presentation: Optimizing for the #SDDC
 
BI the Agile Way
BI the Agile WayBI the Agile Way
BI the Agile Way
 
White Paper - Data Warehouse Documentation Roadmap
White Paper -  Data Warehouse Documentation RoadmapWhite Paper -  Data Warehouse Documentation Roadmap
White Paper - Data Warehouse Documentation Roadmap
 
Agile BI: How to Deliver More Value in Less Time
Agile BI: How to Deliver More Value in Less TimeAgile BI: How to Deliver More Value in Less Time
Agile BI: How to Deliver More Value in Less Time
 
Bi presentation to bkk
Bi presentation to bkkBi presentation to bkk
Bi presentation to bkk
 
Data Vault Introduction
Data Vault IntroductionData Vault Introduction
Data Vault Introduction
 
Business Process and Design Report
Business Process and Design ReportBusiness Process and Design Report
Business Process and Design Report
 
Oracle - Next Generation Datacenter - Alan Hartwell
Oracle - Next Generation Datacenter - Alan HartwellOracle - Next Generation Datacenter - Alan Hartwell
Oracle - Next Generation Datacenter - Alan Hartwell
 

Andere mochten auch

Data vault seminar May 5-6 Dommel - The factory and the workshop
Data vault seminar May 5-6 Dommel - The factory and the workshopData vault seminar May 5-6 Dommel - The factory and the workshop
Data vault seminar May 5-6 Dommel - The factory and the workshop
johannesvdb
 

Andere mochten auch (13)

Data vault seminar May 5-6 Dommel - The factory and the workshop
Data vault seminar May 5-6 Dommel - The factory and the workshopData vault seminar May 5-6 Dommel - The factory and the workshop
Data vault seminar May 5-6 Dommel - The factory and the workshop
 
Data vault what's Next: Part 2
Data vault what's Next: Part 2Data vault what's Next: Part 2
Data vault what's Next: Part 2
 
Tdwi agile data warehouse - dv, what is the buzz about
Tdwi   agile data warehouse - dv, what is the buzz aboutTdwi   agile data warehouse - dv, what is the buzz about
Tdwi agile data warehouse - dv, what is the buzz about
 
Metadaten und Data Vault (Meta Vault)
Metadaten und Data Vault (Meta Vault)Metadaten und Data Vault (Meta Vault)
Metadaten und Data Vault (Meta Vault)
 
CDC und Data Vault für den Aufbau eines DWH in der Automobilindustrie
CDC und Data Vault für den Aufbau eines DWH in der AutomobilindustrieCDC und Data Vault für den Aufbau eines DWH in der Automobilindustrie
CDC und Data Vault für den Aufbau eines DWH in der Automobilindustrie
 
Data Vault automation conference - all presentations
Data Vault automation conference - all presentationsData Vault automation conference - all presentations
Data Vault automation conference - all presentations
 
Experiences from a Data Vault Pilot Exploiting the Internet of Things
Experiences from a Data Vault Pilot Exploiting the Internet of ThingsExperiences from a Data Vault Pilot Exploiting the Internet of Things
Experiences from a Data Vault Pilot Exploiting the Internet of Things
 
Shorter time to insight more adaptable less costly bi with end to end modelst...
Shorter time to insight more adaptable less costly bi with end to end modelst...Shorter time to insight more adaptable less costly bi with end to end modelst...
Shorter time to insight more adaptable less costly bi with end to end modelst...
 
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile ApproachUsing OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
 
Operational Data Vault
Operational Data VaultOperational Data Vault
Operational Data Vault
 
Real-life Customer Cases using Data Vault and Data Warehouse Automation
Real-life Customer Cases using Data Vault and Data Warehouse AutomationReal-life Customer Cases using Data Vault and Data Warehouse Automation
Real-life Customer Cases using Data Vault and Data Warehouse Automation
 
Agiles Data Mining mit Data Vault 2.0
Agiles Data Mining mit Data Vault 2.0Agiles Data Mining mit Data Vault 2.0
Agiles Data Mining mit Data Vault 2.0
 
Dv 20 sdlc_oss_automation
Dv 20 sdlc_oss_automationDv 20 sdlc_oss_automation
Dv 20 sdlc_oss_automation
 

Ähnlich wie #Centennium Presentation Data Vault Automation seminar #dvauto Oct 6th 2011

SenaritraMSBI_Resume
SenaritraMSBI_ResumeSenaritraMSBI_Resume
SenaritraMSBI_Resume
Senaritra Das
 

Ähnlich wie #Centennium Presentation Data Vault Automation seminar #dvauto Oct 6th 2011 (20)

Presentation by Erik van der Hoeven (Wisdom as a Service) at the Data Vault M...
Presentation by Erik van der Hoeven (Wisdom as a Service) at the Data Vault M...Presentation by Erik van der Hoeven (Wisdom as a Service) at the Data Vault M...
Presentation by Erik van der Hoeven (Wisdom as a Service) at the Data Vault M...
 
KEDAR_TERDALKAR
KEDAR_TERDALKARKEDAR_TERDALKAR
KEDAR_TERDALKAR
 
A Tight Ship: How Containers and SDS Optimize the Enterprise
 A Tight Ship: How Containers and SDS Optimize the Enterprise A Tight Ship: How Containers and SDS Optimize the Enterprise
A Tight Ship: How Containers and SDS Optimize the Enterprise
 
Chinmay_Kulkarni_CV
Chinmay_Kulkarni_CVChinmay_Kulkarni_CV
Chinmay_Kulkarni_CV
 
Resume
ResumeResume
Resume
 
Eladio Abquina on FOSS TOOL for Mining Big Data;OCC PORTAL Dashboard Story
Eladio Abquina on FOSS TOOL for Mining Big Data;OCC PORTAL Dashboard StoryEladio Abquina on FOSS TOOL for Mining Big Data;OCC PORTAL Dashboard Story
Eladio Abquina on FOSS TOOL for Mining Big Data;OCC PORTAL Dashboard Story
 
Computational Patterns of the Cloud
Computational Patterns of the CloudComputational Patterns of the Cloud
Computational Patterns of the Cloud
 
Cendien - Senior Cognos Developer & Consultant
Cendien - Senior Cognos Developer & ConsultantCendien - Senior Cognos Developer & Consultant
Cendien - Senior Cognos Developer & Consultant
 
Wwt Corp. Overview &amp; Data Center Presentation For Hendee
Wwt Corp. Overview &amp; Data Center Presentation For HendeeWwt Corp. Overview &amp; Data Center Presentation For Hendee
Wwt Corp. Overview &amp; Data Center Presentation For Hendee
 
Bi Dw Presentation
Bi Dw PresentationBi Dw Presentation
Bi Dw Presentation
 
Amit_Kumar_CV
Amit_Kumar_CVAmit_Kumar_CV
Amit_Kumar_CV
 
Augmentation, Collaboration, Governance: Defining the Future of Self-Service BI
Augmentation, Collaboration, Governance: Defining the Future of Self-Service BIAugmentation, Collaboration, Governance: Defining the Future of Self-Service BI
Augmentation, Collaboration, Governance: Defining the Future of Self-Service BI
 
Oracle Analytics Live Webinar August 2021
Oracle Analytics Live Webinar August 2021Oracle Analytics Live Webinar August 2021
Oracle Analytics Live Webinar August 2021
 
SenaritraMSBI_Resume
SenaritraMSBI_ResumeSenaritraMSBI_Resume
SenaritraMSBI_Resume
 
Customer Success Stories
Customer Success StoriesCustomer Success Stories
Customer Success Stories
 
Building and Deploying Microservices with Event Sourcing, CQRS and Docker
Building and Deploying Microservices with Event Sourcing, CQRS and DockerBuilding and Deploying Microservices with Event Sourcing, CQRS and Docker
Building and Deploying Microservices with Event Sourcing, CQRS and Docker
 
EOSC Digital Innovation Hub (EOSC DIH)
EOSC Digital Innovation Hub (EOSC DIH)EOSC Digital Innovation Hub (EOSC DIH)
EOSC Digital Innovation Hub (EOSC DIH)
 
How to Leverage SAFe 5.0 for Your Enterprise Cloud Strategy
How to Leverage SAFe 5.0 for Your Enterprise Cloud StrategyHow to Leverage SAFe 5.0 for Your Enterprise Cloud Strategy
How to Leverage SAFe 5.0 for Your Enterprise Cloud Strategy
 
OpenStackSummitTokyo - CloudKitty an Open Source rating and chargeback compon...
OpenStackSummitTokyo - CloudKitty an Open Source rating and chargeback compon...OpenStackSummitTokyo - CloudKitty an Open Source rating and chargeback compon...
OpenStackSummitTokyo - CloudKitty an Open Source rating and chargeback compon...
 
Xcelsius tips and tricks for your Enterprise
Xcelsius tips and tricks for your EnterpriseXcelsius tips and tricks for your Enterprise
Xcelsius tips and tricks for your Enterprise
 

Kürzlich hochgeladen

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Kürzlich hochgeladen (20)

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 

#Centennium Presentation Data Vault Automation seminar #dvauto Oct 6th 2011

  • 1. CDM Centennium Datawarehouse Methodology Structure – Model – Generate – Knowledge Partnership 1 Erik Fransen Antoine Stelma October 6, 2011
  • 2. Agenda 2 Centennium BI expertisehuis CDM Structure Reference Architecture Model Data Vault Generate Template Based Development Knowledge Partnership Knowledge transfer, coaching CDM process
  • 3. Centennium BI expertisehuis Independent We provide our customers with the right skills, at the right time and the right way Consultancy, ad-interim support, project implementation and training services, but primarily acts as a BI/DWH knowledge partner for its clients We take or share responsibility for the execution and management of BI and DWH projects and provide second or third line support for existing implementations We offer an extensive portfolio of courses and training services We provide our customers with the knowledge and practical insights required to be(come) self-sufficient in maintaining and expanding their BI-environments www.centennium.nl 3
  • 4. Centennium Datawarehouse Methodology A methodology, including DWH-tools Build or migrate datawarehouses fast, with high qualityand low cost Automatic generation of datawarehouse based on descriptivemetadata CDM includes: Template Based Development Best practices Quality control mechanism Knowledge partnership Best Practices: Data Vault, Kimball, Reference Architecture Qualitycontrol mechanism: extensive checklists anddocumentation Knowledge transfer by training, certificationandlearning on-the-job 4
  • 5. 5 Centennium Datawarehouse Methodology Knowledge Partnership Modelling Structuring Generating
  • 6. Knowledge partnership Goal: Self-supporting, CDM-awarecustomers Training options BI and DWH fundamentals CDM designer / TBD developer training Data Vaultfundamentals Data Vaultcertification(Genesee Academy) Dimensionalmodelling More info on www.bi-opleidingen.nl On-the-job coaching, learningbydoing Centennium supports it’scustomersby(management) consultancy, assessments, projects, training andsourcing 6
  • 8. Template Based Development:Generating TBD generatesall ETL and objects for Registrationlayer Presentation/Publicationlayer Repositoryand scripts are free of charge Creating the staginglayer is not part of TBD but canbeautomated per customer 8
  • 9. TBD databases 9 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
  • 10. TBD repository 10 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
  • 11. Functionality TBD repositorylogtables (1) Functionality: Registration of all ETL processes Insight in data logistics Historic profile of loadingprocesses Goal: Traceability Maintenance: localisationandidentification of all data/transactions (wrong andcorrect) Linkage to source systems 11 REP PUB STA CDW
  • 12. Functionality TBD repositorytemplate tables Functionality: Central storage of template scripts forgeneratingtablesand ETL procedures Easy promotion to other environments (OTAP) Transparant Goal: Version management of templates Object (re-)creationand maintenance 12 REP PUB STA CDW
  • 13. Functionality TBD repositorymeta data tables Functionality: Definesregistrationandpublicationlayer Contains object and ETL definitions Contains data logistics Goal: Generating: Registrationlayer(Data Vaulttables) Publicationlayer (Star schemes) ETL processes via ‘stored procedures’ Object creationand maintenance 13 REP PUB STA CDW
  • 14. Functionality TBD repositorymeta data tables Table ‘Repos_Table’: Contains object namesfor Staging Registration Publication Table is filledbyanapplication or Excel sheet Table ‘Repos_Mapping’: Contains mapping of staging, registration and presentation Table is filledbyanapplication or Excel sheet 14 REP PUB STA CDW
  • 15. Registrationlayer 15 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
  • 16. Registrationlayer Script ‘CDW 01a table hub’ Generatesall hub tables in the CDW, as defined in repository Every hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Hubs generated according to Data Vault Standards! 16 REP PUB STA CDW
  • 17. Registrationlayer Script ‘CDW 01a tabel hub’ Generatesall hub tables in the CDW, as defined in repository Every hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Hubs generated according to Data Vault Standards! 17 REP PUB STA CDW
  • 18. Registrationlayer Script ‘CDW 02a table sat’ Generatesallsatellitetables in the CDW, as defined in repository Everysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Satsgenerated according to Data Vault Standards! 18 REP PUB STA CDW
  • 19. Registrationlayer Script ‘CDW 02a tabel sat’ Generatesallsatellitetables in the CDW, as defined in repository Everysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_id Variables fromrepository are applied Once the hubs are generated, the repository is updated Satsgenerated according to Data Vault Standards! 19 REP PUB STA CDW
  • 20. Registrationlayer Script‘CDW 03a table link’ Generatesalllink tables in the CDW, as defined in repository Everylink is connected to two or more hubs Variables fromrepository are applied Once the links are generated, the repository is updated Links generated according to Data Vault Standards! 20 REP PUB STA CDW
  • 21. Registrationlayer Script‘CDW 03a table link’ Generatesalllink tables in the CDW, as defined in repository Everylink is connected to two or more hubs Variables fromrepository are applied Once the links are generated, the repository is updated Links generated according to Data Vault Standards! 21 REP PUB STA CDW
  • 22. Registrationlayer Script‘CDW 04a table link satellite’ Generatesalllink satellitetables in the CDW, as defined in repository Variables fromrepository are applied Once the link satellites are generated, the repository is updated Link satellites are generated according to Data Vault Standards! 22 REP PUB STA CDW
  • 23. Registrationlayer Script‘CDW 04a table link satellite’ Generatesalllink satellitetables in the CDW, as defined in repository Variables fromrepository are applied Once the link satellites are generated, the repository is updated Link satellites are generated according to Data Vault Standards! 23 REP PUB STA CDW
  • 24. Publicationlayer 24 Repository (REP) Central Data Warehouse (CDW) Staging (STA) Publication (PUB)
  • 25. Publicationlayerdimensionsandfacts Fedonlyanddirectlyfrom CDW Conforms to Kimball’sstar scheme standards Or anyother format Business rulescanbeapplied Currentlyusing views Business rule editor in next release Type SCD I, II, etc. Conformeddimensionswhenneeded 25 REP PUB STA CDW
  • 26. CDM: process Incremental approach Timeboxes of 4-6 weeks 26
  • 27.
  • 28. Centennium role changes from LEAD to FOLLOW
  • 29. Customer is fully CDM-aware at the end of the increment
  • 30.
  • 32.
  • 33.
  • 35.
  • 36. Strategic, tactic and operational
  • 37.
  • 40. EducationSome of our clients: Woonbron, Albron, NZa, CAK, OBR, Vopak, several Dutch Municipalities, Aegon, Nutreco, TNO,Genzyme, Tata Steel, KPN, DELTA, IKEA, Accell, TomTom, KAS BANK, LeasePlan, Brabant Water
  • 41. Certification and Industry Recognition All Centennium consultants are Certified BI Professionals (CBIP) awarded by TDWI. CBIP, backed by TDWI's reputation as the leader in BI education and research, makes a strong statement that we are among the leaders in the industry—up to date with BI technologies, knowledgeable about best practices and state-of-the-art solutions, and informed of emerging trends All Centennium Data Vault specialists and architects are certified on Data Vault modeling by Dan Linstedt, Data Warehouse industry thought leader and founder of the Data Vault modeling standard. Centennium is the business partner of Genesee Academy in organizing the Data Vault Modeling & Certification – Europe Course. We are accredited to perform the Genesee Academy Data Vault trainings. Centennium’s customer Nutreco was awarded the world’s first Certified Data Vault Data Model in 2009 for their corporate financial and procurement data warehouse model.
  • 42. 32
  • 43. Centennium BI expertisehuis Lange Voorhout 43 2514 EC 's-Gravenhage Telefoon 070 31 20 370 Fax 070 31 20 371 URL www.centennium.nl www.bi-opleidingen.nl www.data-vault.nl