IBM DB2 Blu

1. Fabrizio DANUSSO | Client Technical Professional
16 octobre 2013
IBM DB2 with BLU Acceleration
Bois-Colombes - User Group Infosphere
© 2013 IBM Corporation

2. Acknowledgements and Disclaimers
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in
which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for
informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant.
While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without
warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this
presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or
representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use
of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have
achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended
to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other
results.
2
© 2013 IBM Corporation

3. Croissance des besoins d’analyse des données
1
The emergence of
big data
Creating new
opportunities to capture
meaningful information
from new varieties of
data and content coming
at organizations in huge
volumes and at
accelerated velocity
3
2
The shift of power to
the consumer
Creating the need for
organizations to
understand and anticipate
customer behavior and
needs based on customer
insights across all
channels
© 2013 IBM Corporation
3
Accelerating
pressure to do more
with less
Creating the need for all
parts of the organization
to optimize all of their
processes to create new
opportunities, to mitigate
risk, and to increase
efficiency

4. Accélérateur BLU: c’est quoi?

BLU combine des technologies de pointe
-

BLU augmente les fonctionnalités existantes de DB2
-

Utilise des tables organisées en colonnes qui coexistent avec les tables classique organisées en lignes
Utilise un moteur d’exécution colonne qui coexiste avec le moteur traditionnel
C’est du DB2: TBS, BP, utilitaires, SQL, ne changent pas
Les tables organisées en colonnes utilisent
-
4
25+ brevets dans 10 laboratoires sur 7 continents
Compression des données
Processus vectoriels
Optimisation de ma mémoire
Et plus
© 2013 IBM Corporation

5. Accélérateur BLU: à quoi ça sert?

BLU est conçu pour vos requêtes analytiques
-

BLU offre des avantages majeurs
-
5
Maximise l’utilisation de votre mémoire, mais…
Ne nécessite pas une solution coûteuse « in-memory only »
Performances ultra rapide (10x – 25x)
Réduit le besoin de stockage (5x – 20x)
Administration simplifié, « load-and-go »
© 2013 IBM Corporation

6. Super Fast, Super Easy – Create, Load, and Go!
Database Design and Tuning
1.
2.
3.
4.
5.
6.
7.
8.
9.
6
Decide on partition strategies
Select Compression Strategy
Create Table
Load data
Create Auxiliary Performance Structures
•
Materialized views
•
Create indexes
•
B+ indexes
Repeat
•
Bitmap indexes
Tune memory
Tune I/O
Add Optimizer hints
Statistics collection
VS
© 2013 IBM Corporation
DB2 with BLU Acceleration
1.
2.
Create Table
Load data

7. Accélérateur BLU: pourquoi est-il spéciale?
Ses caractéristiques principales:
1.
2.
3.
4.
5.
6.
7.
7
Facile à mettre en œuvre et à administrer
Extrême compression des données
Stockage des données en mode colonne
Data Skipping
Parallélisme multi processeur
Accélération CPU SIMD (Single Instruction Multi Data)
Cache mémoire intelligent
© 2013 IBM Corporation

8. Accélérateur BLU: Facile à mettre en œuvre et à administrer
 Set one registry variable: DB2_WORKLOAD=ANALYTICS
-
Then create the database to get optimal settings for an analytic environment optimized to your hardware
It’s still just DB2: use traditional storage, utilities, SQL, application interfaces, and so on
 Now just “load and go”
-
Compression automatically done and statistics automatically collected
No need for indexes, partitions, MQTs (materialized views), MDCs, hints, statistical views, etc.
Underlying structures that support BLU are created and updated automatically
 Ongoing maintenance is also minimal
-
8
REORGs (for space reclaim) and workload management is all automatic
© 2013 IBM Corporation

9. Accélérateur BLU: Extrême compression des données
 Massive compression with approximate Huffman (frequency-based) encoding
-
Most frequent values encoded with fewest bits
Extremely effective since all data in the column are the same data type
Map entire values directly to column dictionary codes by both table and page levels
 Register-friendly encoding greatly improves efficiency
-
Encoded values are packed into bits matching the CPU register length
Requires fewer I/Os and CPU cycles as well as better memory utilization
 Encoded values do not need to be decompressed during evaluation
-
9
Predicates and joins work directly on (actionable) encoded values
(Predicate examples: =, <>,<, >, >=, <=, between, etc.)
© 2013 IBM Corporation

10. Accélérateur BLU: Stockage des données en mode colonne




10
Efficient columnar storage automatically done during LOAD
Result: minimal I/O
Work performed directly on columns
Improved memory density
© 2013 IBM Corporation

11. Accélérateur BLU: Data Skipping
 Automatic detection of large sections of data that do not qualify for a query and can be ignored
 Order of magnitude savings in all of I/O, RAM, and CPU
 No DBA action to define or use – truly invisible
11
© 2013 IBM Corporation

12. Accélérateur BLU: Parallélisme multi processeur




12
BLU Acceleration builds on traditional multi-core parallelism
Multi-core parallelism is a combination of Data and Functional parallelism
Data parallelism assigns dynamic partitions to retrieve data from disk
Functional parallelism divides queries into multiple parts by function
© 2013 IBM Corporation

13. Accélérateur BLU: Accélération CPU SIMD
 “Single Instruction Multiple Data “ (SIMD) CPU design:
-
Performs the same operation on multiple data elements simultaneously
 Various CPUs have built-in instructions to perform SIMD
-
Your hardware already has this capability - BLU Acceleration simply exploits it (128-bit register usage is
maximized as well)
BLU Acceleration has been tested to perform on:
-
13
AIX on Power - best performance on Power 7
Linux 64-bit on Intel / AMD - best performance on Intel Nehalem or better
© 2013 IBM Corporation

14. Accélérateur BLU: Cache mémoire intelligent
 New algorithms cache in RAM effectively
 High percent of interesting data fits in memory
-
We leave the interesting data in memory with the new algorithms
RAM
 Data can be larger than RAM
-
No need to ensure all data fits in memory
Optimization for in memory and I/O efficiency
Near optimal caching
DISKS
14
© 2013 IBM Corporation

15. Accélérateur BLU : En conclusion








Technologie intégrée au moteur DB2
Column-organized table approach
Orienté pour les workloads hybrides et analytiques
Transparent pour les applications
Implémentation et administration simplifiés
Multi-core and SIMD parallelism
Data skipping
Meilleur taux de compression pour les données
Super Fast, Super Easy— Create, Load and Go!
No Indexes, No Aggregates, No Tuning, No SQL changes, No schema
changes, No REORG/RUNSTATS
15
© 2013 IBM Corporation

16. Fabrizio Danusso
Client Technical Professional
fabrizio.danusso@fr.ibm.com
16
© 2013 IBM Corporation

17. 17
© 2013 IBM Corporation

18. © Copyright IBM Corporation 2013. All rights reserved.
•
•
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
IBM, the IBM logo, ibm.com, Informix, are trademarks or registered trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information
with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this
information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
18
© 2013 IBM Corporation