Weitere ähnliche Inhalte Ähnlich wie Data Warehousing and Bitmap Indexes - More than just some bits (20) Kürzlich hochgeladen (20) Data Warehousing and Bitmap Indexes - More than just some bits1. MORE THAN JUST SOME BITS…
Data Warehousing and Bitmap Indexes
Dani Schnider, Trivadis AG
Oracle Open World 2011,
San Francisco
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
1
2. Dani Schnider
Principal Consultant und
DWH/BI Lead Architect
bei Trivadis in Zürich
Teacher for courses about
Data Warehousing, SQL
Optimization and Oracle
Warehouse Builder
Co-author of book «Data
Warehousing mit Oracle»
2014 © Trivadis
2
Fehlertolerante Ladeprozesse
18. April 2012
3. Mit über 600 IT- und Fachexperten bei Ihnen vor Ort
2014 © Trivadis
11 Trivadis Niederlassungen mit
über 600 Mitarbeitenden
200 Service Level Agreements
Mehr als 4'000 Trainingsteilnehmer
Forschungs- und Entwicklungs-budget:
CHF 5.0 / EUR 4 Mio.
Finanziell unabhängig und
nachhaltig profitabel
Erfahrung aus mehr als 1'900
Projekten pro Jahr bei über 800
Kunden
Stand 12/2012
Hamburg
Düsseldorf
Frankfurt
Freiburg
München
Wien
Basel
Bern Zürich
Lausanne
3
Stuttgart
Data Warehouseing & Bitmap Indexes
02.10.2011
3
4. 2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
4
SOME BITS ABOUT
BITMAP INDEXES
AND THEIR USAGE IN
DATA WAREHOUSES
5. Bitmap Indexes – Concept
Oracle® Database Concepts 11g Release 2 (11.2)
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
5
6. Bitmap Indexes – Concept
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
6
7. Bitmap Indexes – Example
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
7
10. Concatenated Bitmap Indexes
Bitmap Indexes on two or more columns are possible, but not useful
2014 © Trivadis
Why?
Concatenated bitmap index can be used only when all columns are
filtered
Two or more separate bitmap indexes are more flexible and can be used
for all combinations
Data Warehouseing & Bitmap Indexes
02.10.2011
10
11. Index Strategy on Star Schema
Dimension Tables:
Unique b-tree index on primary key column
Bitmap indexes on additional columns (optional)
Fact Tables:
Bitmap indexes on all dimension key (foreign key) columns
Bitmap join indexes on often used filter columns (optional)
Usually no primary key on fact tables
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
11
12. Index Strategy on Star Schema
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
12
13. Query Optimization on Star Schemas
Typical Queries:
Filter criteria on (multiple) dimension
2014 © Trivadis
tables
Facts are selected by join with
dimensions
Problem:
Tables with restrictions should be
read first
Only two tables can be joined at a
time
No relationships between dimension
tables
Data Warehouseing & Bitmap Indexes
02.10.2011
13
15. Star Transformation – Example
What is the total revenue for product “Shorts” for all customers in Germany,
grouped by male and female customers?
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
15
17. Bitmap Join Index – Concept
Bitmap index is defined on fact
2014 © Trivadis
table
Indexed column is part of
dimension table
Possible for star and snowflake
schemas
Data Warehouseing & Bitmap Indexes
02.10.2011
17
18. Bitmap Join Index – Example
Same example with Bitmap Join Indexes
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
18
Demo
19. Bitmap Indexes and ETL
Bitmap indexes are efficient for set-based ETL
Index maintenance is deferred until end of each DML operation
Efficient index maintenance for Parallel DML
Avoid “hand-made” parallel ETL with multiple sessions
Locking behavior of bitmap indexes
For large data loads: Rebuild indexes after ETL operation
Set all indexes to UNUSABLE
Load data into table
Rebuild all indexes
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
19
20. Bitmap Indexes and Partitioning
Bitmap Indexes must always be LOCAL
Local indexes are recommended in Data Warehouse
Only purpose of global indexes in DWH: unique indexes without partition key
Restriction of local bitmap indexes is not a problem
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
20
21. Bitmap Indexes and Compression
Bitmap Indexes are always compressed
Zeros are compressed, Ones are not compressed
Bitmap contains number of zeros to next 1 bit
Bitmap Indexes need less disk space than B-tree Indexes
Even for columns with high number of distinct keys
Detail description see «Bitmap Index Internals», © Julian Dyke
http://www.juliandyke.com/Presentations/BitmapIndexInternals.ppt
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
21
22. Bitmap Indexes and Compression
Number of distinct keys has impact on size of index
Bitmap index is usually smaller than b-tree index
For high number of keys, size of bitmap index increases
Size of b-tree index is more or less constant
Sort order affects size of bitmap index, but not of b-tree index
2014 © Trivadis
350
300
250
200
150
100
50
Data Warehouseing & Bitmap Indexes
02.10.2011
22
Demo
0
10 100 1000 10000 100000
Bitmap (scattered) Bitmap (sorted) B-tree index
23. Bitmap Indexes and Compression
Enabling table compression
When a table is enabled for compression, all bitmap indexes must be disabled
Reason: Hakan factor of table
Maximum number of rows that can be stored in a table block
Hakan factor changes when table is enabled for compression (only first time)
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
23
Demo
24. Bitmap Indexes – Conclusion
Bitmap Indexes are a powerful Data Warehouse feature
Can be combined in ad hoc queries
Star Transformation
Useful for selective and non-selective columns
Bitmap Indexes are useful for large databases
Efficient for set-based ETL and Parallel DML
Local bitmap indexes on partitioned tables
Compression of bitmap indexes
There are almost no reasons to use b-tree indexes in a Data Warehouse
B-tree indexes are mainly used for Primary Key or Unique Key constraints
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
24
25. References
Oracle® Database Data Warehousing Guide 11g Release 2 (11.2)
Oracle Corporation, http://www.oracle.com/pls/db112/
Oracle® Database Concepts 11g Release 2 (11.2)
Oracle Corporation, http://www.oracle.com/pls/db112/
Christian Antognini: Troubleshooting Oracle Performance
Apress, http://www.apress.com/9781590599174
Jonathan Lewis: Cost-Based Oracle Fundamentals
Apress, http://www.apress.com/9781590596364
Julian Dyke: Bitmap Index Internals
http://www.juliandyke.com/Presentations/BitmapIndexInternals.ppt
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
25
26. THANK YOU.
Trivadis AG
Dani Schnider
Europa-Strasse 5
CH-8152 Glattbrugg/Zürich
Switzerland
Tel. +41 44 808 70 20
Fax +41 44 808 70 21
info@trivadis.com
www.trivadis.com
BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN
2014 © Trivadis
Data Warehouseing & Bitmap Indexes
02.10.2011
26