Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Partitioning your Oracle Data Warehouse - Just a simple task?

2.126 Aufrufe

Veröffentlicht am

Dani Schnider, Principal Consultant bei der Trivadis AG, an der Oracle Open World 2009 in San Francisco

Veröffentlicht in: Präsentationen & Vorträge
  • Überprüfen Sie die Quelle ⇒ www.WritersHilfe.com ⇐ . Diese Seite hat mir geholfen, eine Diplomarbeit zu schreiben.
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Partitioning your Oracle Data Warehouse - Just a simple task?

  1. 1. Partitioning Your Oracle Data Warehouse – Just a Simple Task? Dani Schnider Principal Consultant Business Intelligence dani.schnider@trivadis.com Oracle Open World 2009, San Francisco BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 1
  2. 2. Dani Schnider  Principal Consultant und DWH/BI Lead Architect bei Trivadis in Zürich  Kursleiter für Trivadis-Kurse über Data Warehousing, SQL Optimierung und Oracle Warehouse Builder  Co-Autor des Buches «Data Warehousing mit Oracle» 2014 © Trivadis 2 Partitioning Your Oracle Data Warehouse 2009
  3. 3. Mit über 600 IT- und Fachexperten bei Ihnen vor Ort 2014 © Trivadis 11 Trivadis Niederlassungen mit über 600 Mitarbeitenden 200 Service Level Agreements Mehr als 4'000 Trainingsteilnehmer Forschungs- und Entwicklungs-budget: CHF 5.0 / EUR 4 Mio. Finanziell unabhängig und nachhaltig profitabel Erfahrung aus mehr als 1'900 Projekten pro Jahr bei über 800 Kunden Stand 12/2012 Hamburg Düsseldorf Frankfurt Freiburg München Wien Basel Bern Zürich Lausanne 3 Stuttgart Partitioning Your Oracle Data Warehouse 2009 3
  4. 4. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 4
  5. 5. Partitioning – The Basic Idea  Decompose tables/indexes into smaller pieces 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 5
  6. 6. Partitioning Methods in Oracle  Additionally: Interval Partitioning, Reference Partitioning, Virtual Column-Based Partitioning 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 6
  7. 7. Benefits of Partitioning Partition Pruning  Reduce I/O: Only relevant partitions have to be accessed Partition-Wise Joins  Full PWJ: Join two equi-partitioned tables (parallel or serial)  Partial PWJ: Join partitioned with non-partitioned table (parallel) Rolling History  Create new partitions in periodical intervals  Drop old partitions in periodical intervals Manageability  Backups, statistics gathering, compressing on individual partitions 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 7
  8. 8. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 8
  9. 9. Partition Methods and Partition Keys  Important questions for Partitioning:  Which tables should be partitioned?  Which partition method should be used?  Which partition key(s) should be used?  Partition key is important for  Query optimization (partition pruning, partition-wise 2014 © Trivadis joins)  ETL performance (partition exchange, rolling history)  Typically in data warehouses  RANGE partitioning of fact tables on DATE column  But which DATE column? Partitioning Your Oracle Data Warehouse 2009 9
  10. 10. Practical Example 1: Airline Company  Flight bookings are stored in partitioned table  RANGE partitioning per month, partition key: booking date  Problem: Most of the reports are based on the flight date  Flights can be booked 11 months ahead  11 partitions must be read of one particular flight date 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 10
  11. 11. Practical Example 1: Airline Company  Solution: partition key flight date instead of booking date  Data is loaded into current and future partitions  Reports based on flight date read only one partition  Reports based on booking date must read 11 (small) partitions 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 11
  12. 12. Practical Example 1: Airline Company  Better solution: Composite RANGE-RANGE partitioning  RANGE partitioning on flight date  RANGE subpartitioning on booking date  More flexibility for reports on flight date and/or on booking date 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 12
  13. 13. Practical Example 2: International Bank  Account balance data of international customers  Monthly files of different locations (countries)  Correction files replace last delivery for the same month/country  Original approach  Technical load id for each combination of month/country  LIST partitioning on load id  Files are loaded in stage table  Partition exchange with current partition 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 13
  14. 14. Practical Example 2: International Bank  Problem: partition key load id is useless for queries  Queries are based on balance date 2014 © Trivadis  Solution  RANGE partitioning on balance date  LIST subpartitions on country code  Partition exchange with subpartitions Partitioning Your Oracle Data Warehouse 2009 14
  15. 15. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 15
  16. 16. Partitioning in Star Schema 2014 © Trivadis  Fact Table  Usually „big“ (millions to billions of rows)  RANGE partitioning by DATE column  Dimension Tables  Usually „small“ (10 to 10000 rows)  In most cases not partitioned  But how about large dimensions?  e.g. customer dimension with millions of rows Partitioning Your Oracle Data Warehouse 2009 16
  17. 17. HASH Partitioning on Large Dimension  DIM_CUSTOMER 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 17  HASH Partitioning  Partition Key: CUSTOMER_ID  FCT_SALES  Composite RANGE-HASH Partitioning  Partition Key: SALES_DATE  Subpartition Key: CUSTOMER_ID
  18. 18. HASH Partitioning on Large Dimension 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 18
  19. 19. Practical Example 3: Telecommunication Company 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 19
  20. 20. LIST Partitioning on Large Dimension  DIM_CUSTOMER 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 20  LIST Partitioning  Partition Key: COUNTRY_CODE  FCT_SALES  Composite RANGE-LIST Partitioning  Partition Key: SALES_DATE  Subpartition Key: COUNTRY_CODE (denormalized column in fact table)
  21. 21. LIST Partitioning on Large Dimension 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 21
  22. 22.  New approach for partition pruning on join conditions  A bloom filter is created based on the dimension table restriction  Dynamic partition pruning based on that bloom filter 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 22
  23. 23. AGENDA 1. Partitioning Concepts 2. The Right Partition Key 3. Large Dimensions 4. Partition Maintenance 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 23
  24. 24. Practical Example 3: Monthly Partition Maintenance  Requirements  Monthly partitions on all fact tables, daily inserts into current partitions  3 years of history (36 partitions per table)  Table compression to increase full table scan performance  Backup of current partitions only 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 24
  25. 25. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 25
  26. 26. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 26
  27. 27. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 3. Create new partition for next month 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 27
  28. 28. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 3. Create new partition for next month 4. Compress current partition 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 28
  29. 29. Practical Example 3: Monthly Partition Maintenance 1. Set next tablespace to read-write 2. Drop oldest partition 3. Create new partition for next month 4. Compress current partition 5. Set tablespace to read-only 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 29
  30. 30. Interval Partitioning  In Oracle11g, the creation of new partitions can be automated 2014 © Trivadis  Example: Partitioning Your Oracle Data Warehouse 2009 30
  31. 31. Gathering Optimizer Statistics  DBMS_STATS parameter GRANULARITY defines statistics level 2014 © Trivadis  Example: Partitioning Your Oracle Data Warehouse 2009 31
  32. 32. Problem of Global Statistics  Global statistics are essential for good execution plans  num_distinct, low_value, high_value, density, histograms  Gathering global statistics is time-consuming  All partitions must be scanned  Typical approach  After loading data, only modified partition statistics are gathered  Global statistics are gathered on regular time base (e.g. weekends) 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 32
  33. 33. Incremental Global Statistics  Synopsis-based gathering of statistics  For each partition a synopsis is stored in SYSAUX tablespace  Statistics metadata for partition and columns of partition  Global statistics by aggregating the synopses from each partition  Activate incremental global statistics: 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 33
  34. 34. Partitioning Your Data Warehouse – Core Messages 2014 © Trivadis  Oracle Partitioning is a powerful option – not only for data warehouses  The concept is simple, but the reality can be complex  Many new partitioning features added in Oracle Database 11g  New Composite Partitioning methods  Interval Partitioning  Join-Filter Pruning  Incremental Global Statistics Partitioning Your Oracle Data Warehouse 2009 34
  35. 35. Thank you. Dani Schnider Principal Consultant Business Intelligence dani.schnider@trivadis.com Oracle Open World 2009, San Francisco BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 2014 © Trivadis Partitioning Your Oracle Data Warehouse 2009 35

×