SlideShare a Scribd company logo
1 of 47
Data  Warehousing Hennie de Nooijer
Data Warehousing Position ,[object Object]
Definition
Expert debate
Architecture
Methodology
Technology
TrendsLet’s dig in
Informationprovisioning
Controlled informationprovisioning Information provisioning DWH
Business Intelligence Data warehouse ETL Hardware RDBMS
Data Warehousing Definition ,[object Object]
Definition
Expert debate
Architecture
Methodology
Technology
Trends,[object Object]
A Data Warehouse is a subject-oriented, integrated, time-variant, non-updatablecollection of dataused in support of decision-making processes
Subject oriented
Integrated
Time variant
Non updatable
Data Warehousing Expert debate ,[object Object]
Definition
Expert debate
Architecture
Methodology
Technology
Trends,[object Object]
Ralph Kimball Dimensional modeling Business subject focus Bottom up Data bus
Dan Linstedt Data modeling All data, all the time Method of design Data Vault
Data Warehousing Architecture Architecture (Latinarchitectura, from the Greek ἀρχιτέκτων – arkhitekton, from ἀρχι- "chief" and τέκτων "builder, carpenter") can mean: The art and science of designing and erecting buildings and other physical structures. The practice of an architect, where architecture means to offer or render professional services in connection with the design and construction of a building, or group of buildings and the space within the site surrounding the buildings, that have as their principal purpose human occupancy or use.[1] A general term to describe buildings and other structures. A style and method of design and construction of buildings and other physical structures. A wider definition may comprise all design activity, from the macro-level (urban design, landscape architecture) to the micro-level (construction details and furniture). Architecture is both the process and product of planning, designing and constructing form, space and ambience that reflect functional, technical, social, and aesthetic considerations. It requires the creative manipulation and coordination of material, technology, light and shadow. Architecture also encompasses the pragmatic aspects of realising buildings and structures, including scheduling, cost estimating and construction administration. As documentation produced by architects, typically drawings, plans and technical specifications, architecture defines the structure and/or behavior of a building or any other kind of system that is to be or has been constructed. Architectural works are often perceived as cultural and political symbols and as works of art. Historical civilizations are often identified with their surviving architectural achievements. Architecture sometimes refers to the activity of designing any kind of system and the term is common in the information technology world. ,[object Object]
Definition
Expert debate
Architecture
Methodology
Technology
Trends,[object Object]
Conventional architecture Current Business Demands/Wishes Integration Storage Presentation D W H TRANSFORM S T A G E Business Information Model
Is geplaatst onder /betreft werkdag Bestelling op Business Information Model Ontvangt /Is geplaatst bij heeft omvang Verplicht tot /Is realisatie van Leverancier Bestaat uit /zit in Leverings condities Is bereid te leveren /kan geleverd worden door Levering Bestaat uit /komt voor in Materiaal soort Voorziet in /wordt in voorzien door werkdag omvang Komt voor in met Moet in voorzien worden voor Wordt ontvangen door /ontvangt Bestaat uit Materiaalbehoefte magazijn Betreft de bereidhied tot het levereren aan een /kan conform worden geleverd aan Magazijn
Modern architecture Integration Storage Presentation Storage Current Business Demands/Wishes S T A G E s o u r c e D W H b u s i n e s s D W H TRANSFORM ALL DATA, ALL THE TIME Current Business Information Model
Data Warehousing Methodology A methodology is instantiated and materialized by a set of methods, techniques and tools. A tool is any instrument or apparatus that is necessary to the performance of some task. A methodology does not describe specific methods; nevertheless it does specify several processes that need to be followed. These processes constitute a generic framework. They may be broken down in sub-processes, they may be combined, or their sequence may change. However any task exercise must carry out these processes in one form or another.[3] Methodology may be a description of process, or may be expanded to include a philosophically coherent collection of theories, concepts or ideas as they relate to a particular discipline or field of inquiry. Methodology may refer to nothing more than a simple set of methods or procedures, or it may refer to the rationale and the philosophical assumptions that underlie a particular study relative to the scientific method. For example, scholarly literature often includes a section on the methodology of the researchers. ,[object Object]
Definition
Expert debate
Architecture
Methodology

More Related Content

What's hot

Dataware housing
Dataware housingDataware housing
Dataware housing
work
 
04 Dimensional Analysis - v6
04 Dimensional Analysis - v604 Dimensional Analysis - v6
04 Dimensional Analysis - v6
Prithwis Mukerjee
 
Introduction to Data Warehousing
Introduction to Data WarehousingIntroduction to Data Warehousing
Introduction to Data Warehousing
Eyad Manna
 

What's hot (20)

Data warehouse system and its concepts
Data warehouse system and its conceptsData warehouse system and its concepts
Data warehouse system and its concepts
 
Data Warehousing
Data WarehousingData Warehousing
Data Warehousing
 
Dataware housing
Dataware housingDataware housing
Dataware housing
 
Data mining 3 - Data Models and Data Warehouse Design (cheat sheet - printable)
Data mining  3 - Data Models and Data Warehouse Design (cheat sheet - printable)Data mining  3 - Data Models and Data Warehouse Design (cheat sheet - printable)
Data mining 3 - Data Models and Data Warehouse Design (cheat sheet - printable)
 
Data warehousing and data mart
Data warehousing and data martData warehousing and data mart
Data warehousing and data mart
 
Data mining 2 - Data warehouse (cheat sheet - printable)
Data mining 2 - Data warehouse (cheat sheet - printable)Data mining 2 - Data warehouse (cheat sheet - printable)
Data mining 2 - Data warehouse (cheat sheet - printable)
 
DATA WAREHOUSING AND DATA MINING
DATA WAREHOUSING AND DATA MININGDATA WAREHOUSING AND DATA MINING
DATA WAREHOUSING AND DATA MINING
 
Cs1011 dw-dm-1
Cs1011 dw-dm-1Cs1011 dw-dm-1
Cs1011 dw-dm-1
 
Data Warehouse and Data Mining
Data Warehouse and Data MiningData Warehouse and Data Mining
Data Warehouse and Data Mining
 
Introduction to Data Warehousing
Introduction to Data WarehousingIntroduction to Data Warehousing
Introduction to Data Warehousing
 
DATA WAREHOUSE IMPLEMENTATION BY SAIKIRAN PANJALA
DATA WAREHOUSE IMPLEMENTATION BY SAIKIRAN PANJALADATA WAREHOUSE IMPLEMENTATION BY SAIKIRAN PANJALA
DATA WAREHOUSE IMPLEMENTATION BY SAIKIRAN PANJALA
 
Data Warehousing and Mining
Data Warehousing and MiningData Warehousing and Mining
Data Warehousing and Mining
 
11666 Bitt I 2008 Lect3
11666 Bitt I 2008 Lect311666 Bitt I 2008 Lect3
11666 Bitt I 2008 Lect3
 
04 Dimensional Analysis - v6
04 Dimensional Analysis - v604 Dimensional Analysis - v6
04 Dimensional Analysis - v6
 
Real World Business Intelligence and Data Warehousing
Real World Business Intelligence and Data WarehousingReal World Business Intelligence and Data Warehousing
Real World Business Intelligence and Data Warehousing
 
Data mining 1 - Introduction (cheat sheet - printable)
Data mining 1 - Introduction (cheat sheet - printable)Data mining 1 - Introduction (cheat sheet - printable)
Data mining 1 - Introduction (cheat sheet - printable)
 
Introduction to Data Warehousing
Introduction to Data WarehousingIntroduction to Data Warehousing
Introduction to Data Warehousing
 
IT webinar 2016
IT webinar 2016IT webinar 2016
IT webinar 2016
 
Webinar on IT Basics by IIM Rohtak for Admissions-2014
Webinar on IT Basics by IIM Rohtak for Admissions-2014Webinar on IT Basics by IIM Rohtak for Admissions-2014
Webinar on IT Basics by IIM Rohtak for Admissions-2014
 
Data warehousing
Data warehousingData warehousing
Data warehousing
 

Similar to Datawarehousing

IT_RFO10-14-ITS_AppendixA_20100513
IT_RFO10-14-ITS_AppendixA_20100513IT_RFO10-14-ITS_AppendixA_20100513
IT_RFO10-14-ITS_AppendixA_20100513
Alexander Doré
 
Evaluation of Research Tools
Evaluation of Research ToolsEvaluation of Research Tools
Evaluation of Research Tools
HATS
 

Similar to Datawarehousing (20)

Agile Testing Days 2017 Intoducing AgileBI Sustainably - Excercises
Agile Testing Days 2017 Intoducing AgileBI Sustainably - ExcercisesAgile Testing Days 2017 Intoducing AgileBI Sustainably - Excercises
Agile Testing Days 2017 Intoducing AgileBI Sustainably - Excercises
 
Brief Introduction to Digital Preservation
Brief Introduction to Digital PreservationBrief Introduction to Digital Preservation
Brief Introduction to Digital Preservation
 
IT_RFO10-14-ITS_AppendixA_20100513
IT_RFO10-14-ITS_AppendixA_20100513IT_RFO10-14-ITS_AppendixA_20100513
IT_RFO10-14-ITS_AppendixA_20100513
 
OAIS and It's Applicability for Libraries, Archives, and Digital Repositories...
OAIS and It's Applicability for Libraries, Archives, and Digital Repositories...OAIS and It's Applicability for Libraries, Archives, and Digital Repositories...
OAIS and It's Applicability for Libraries, Archives, and Digital Repositories...
 
Reference Model for an Open Archival Information Systems (OAIS): Overview and...
Reference Model for an Open Archival Information Systems (OAIS): Overview and...Reference Model for an Open Archival Information Systems (OAIS): Overview and...
Reference Model for an Open Archival Information Systems (OAIS): Overview and...
 
Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009
 
Enterprise architecture
Enterprise architectureEnterprise architecture
Enterprise architecture
 
AIS 3 - EDITED.pdf
AIS 3 - EDITED.pdfAIS 3 - EDITED.pdf
AIS 3 - EDITED.pdf
 
"Unveiling Insights: A Data Science Journey".pptx
"Unveiling Insights: A Data Science Journey".pptx"Unveiling Insights: A Data Science Journey".pptx
"Unveiling Insights: A Data Science Journey".pptx
 
Prototype Design of Open Access Institutional Repository
Prototype Design of Open Access Institutional RepositoryPrototype Design of Open Access Institutional Repository
Prototype Design of Open Access Institutional Repository
 
RDO support
RDO supportRDO support
RDO support
 
Putting Historical Data in Context: how to use DSpace-GLAM
Putting Historical Data in Context: how to use DSpace-GLAMPutting Historical Data in Context: how to use DSpace-GLAM
Putting Historical Data in Context: how to use DSpace-GLAM
 
Digital Preservation
Digital PreservationDigital Preservation
Digital Preservation
 
Hughes RDAP11 Data Publication Repositories
Hughes RDAP11 Data Publication RepositoriesHughes RDAP11 Data Publication Repositories
Hughes RDAP11 Data Publication Repositories
 
Evaluation of Research Tools
Evaluation of Research ToolsEvaluation of Research Tools
Evaluation of Research Tools
 
Introduction to the Reference Model for an Open Archival Information System (...
Introduction to the Reference Model for an Open Archival Information System (...Introduction to the Reference Model for an Open Archival Information System (...
Introduction to the Reference Model for an Open Archival Information System (...
 
Эволюция Big Data и Information Management. Reference Architecture.
Эволюция Big Data и Information Management. Reference Architecture.Эволюция Big Data и Information Management. Reference Architecture.
Эволюция Big Data и Information Management. Reference Architecture.
 
SPSRI13 - Taming Your Taxonomy in SharePoint
SPSRI13 - Taming Your Taxonomy in SharePointSPSRI13 - Taming Your Taxonomy in SharePoint
SPSRI13 - Taming Your Taxonomy in SharePoint
 
Mis module ii
Mis module iiMis module ii
Mis module ii
 
Digital Preservation
Digital PreservationDigital Preservation
Digital Preservation
 

Datawarehousing

Editor's Notes

  1. Kern punten :Agenda bespreken.Conclusie:
  2. DWH is gereedschapkistvoor BIFinancieeldirecteur is nietgeinteresseerd in ETLmatroesjkapoppetjesBI -> datawarehouse-> ETL-> RDBMS -> HardwareBusiness intelligence kan gebruik maken van een datawarehouse maar hoeft niet Andersom geldt dat een datawarehouse voor een Business intelligence gebruikt wordt.
  3. Vele definities van een datawarehouse
  4. Subject oriented: customers, patients, students, products, servicesIntegrated: consistent naming conventions, formats, encoding structures, from multiple data sourcesTime-variant: content is explicitly dependant upon time. Historic correctness Non-updatable (non volatile): read-only, periodically refreshed
  5. Data vaakgeorganiseerd per afdeling (marketing, sales)Eenklantzal in meerderesystemenaanwezigzijn.Behoeftehebtaaneen enterprise brede view van eenklant of een patient.Subject oriented: customers, patients, students, products, services
  6. Data uitverschillendebronnen en die integrerenzodatrelatieskloppenConsistentenaamgevingsconventie..FormatenNaamconventiesAls je niet kunt integreren is de waarde van je datawarehouse beperkt.
  7. Twee doelen:* Welke data was actief op eenbepaald moment* trends, complexecorrelaties en predictive analytics.Denk aan boekhoudhouden waarbij journaalposten de bewerkingen zijn. Alles herleidbaar.
  8. Data is stabiel. Data wordtnietgeupdate.versioning through timeOndersteunen van time slicingNiet auditable als we data overschrijven.
  9. Drie goeroe’s op gebied van datawarehousing: KIMBALL, INMON en Linstedt.Kimball en INMON hebben een tegenstrijdig aanpak.Linstedt zit meer op de Inmon aanpak.
  10. Corporate Information FactoryVeelboekengeschrevenwaaronder DW2.0Wanneerkies je voorInmon :DWH onderdeel van algehele BI.Eenorganisatieheeft 1 datawarehouse.Integratietussenafdelingen en divisiesbelangrijk.StrategischVaardigheden van de inhousemedewerkerszijngoed.Informatieverzoekengevarieerd.Veel AD HOC verzoeken'system of record‘ is belangrijkKostmeertijd.
  11. The Datawarehouse toolkitWanneerkies je voor Kimball: Datawarehouse is de verzameling van allerleidatamartsAltijd dimensioneel modelNood is hoog ( en je wilt snelresultaat)AfdelingsgewijzedatawarehousesInformatieverzoekenveranderenniet in grote mateVan teorennietprecieswetenwat de scope is van de datawarehouseIntegratie via conformed dimensions
  12. Alle data uit de bronnen laden.Lijkt op de Inmon aanpak.Inmon zegt ook van Datavault dat het DW2.0 compliant is.Compliance AuditabilityFlexibilityTraceabilityDDL and ETL generated.
  13. Datawarehouse is Integratie OpslagPresentatie
  14. Information model close to the business.When information model close to the source systems you need to modify or rewrite complete ETL, DDL, etc.Transform the source model into the business model.
  15. Hard hat areaEven de diepte in.
  16. Lees als de manier om historie "vast" te houden:Type 1: doet dat nietType 2: doet dat via versioning. Dus nieuwe records met nieuwe data en geldigheids meta-velden: van, tot-en-metType 3: doet dat via extra velden in zelfde rij. Dus beperkt historie, alleen huidige en vorige waarde van een beperkt aantal velden.Type 4: doet dat door afscheiding van historische data van vaste data en versioning ala type 2 op de historische data (lijkt op DV)Type 5: bestaat nietType 6: een "verzinsel" van Kimball, die niemendimplementeert
  17. Switch brains off (again)
  18. Alleen het realiseren van een Data Warehouse …BetekentNIETdatgebruikersbeterebeslissingenkunnennemen …
  19. Echtereen Data Warehouse is wel HET middelDat het mogelijkmaaktomBetereinformatievoorzieningenterealiseren