Exadata X7-2 POC with OVM

1. 2018 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG KOPENHAGEN MUNICH STUTTGART VIENNA
2018 © Trivadis
Exadata X7-2 POC with OVM
Jacques Kostic
Principal Consultant IMS Lausanne
Emiliano Fusaglia
Principal Consultant IMS Lausanne
TechEvent September 2018
Exadata X7-2 POC with OVM
1

2. 2018 © Trivadis
Exadata X7-2 POC with OVM
2
TechEvent September 2018
Experience:
• Initially C/C++ developer
• In touch with Oracle since 1990 from version 4 on SCO Unix!
• High Availability and Backup & Recovery Architect
• SQL and Instance Performance & Tuning
• License Audit and Consolidation
Certifications:
• Oracle Certified Master 11g & 12c
• Oracle 11g Performance Tuning Certified Expert
• Oracle RAC 11g and Grid Infrastructure Administration
• Oracle Exadata Administrator Certified Expert
• Oracle Certified SQL Expert 11g
Teaching Courses at Trivadis:
• Oracle 11g & 12c Grid Infrastructure & RAC
• Oracle 11g & 12c Data Guard
• Oracle 11g & 12c Performance & Tuning
• Oracle 11g & 12c Administration
• SQL & PL-SQL
• OEM – 12 & 13
About me…
@JKOFR

3. 2018 © Trivadis
Exadata X7-2 POC with OVM
3
TechEvent September 2018
Specialties:
• Database Cloud computing (DBaaS)
• Oracle RAC
• Grid Infrastructure (CRS, ASM)
• Data Guard
• Instance and SQL Performance & Tuning
• Linux & Virtualization
Certifications:
• Oracle Certified Professional 9i, 10g, 11g & 12c
• Oracle Exadata Administrator X3 –X4 Certified Expert
Teaching Courses at Trivadis:
• Oracle 11g & 12c Grid Infrastructure & RAC
• Oracle 11g & 12c Data Guard
• Oracle Exadata
• Oracle 12c New Features
About me…
@EFusaglia

4. 2018 © Trivadis
AGENDA
1. Customer Introduction
2. Trivadis Proposal
3. POC Execution
4. Conclusion
5. Q&A
TechEvent September 2018
Exadata X7-2 POC with OVM
6

5. 2018 © Trivadis
Customer Introduction
TechEvent September 2018
Exadata X7-2 POC with OVM
7

6. 2018 © Trivadis
Customer Overview
The name will not be disclosed but the most relevant
characteristics to the project are reported below.
 Major player of the banking sector
 In the process to choose the next DWH platform able to guarantee:
 Optimal Performance
 Scalability
 Licensing Optimization
 Consolidation
Customer
Environment
TechEvent September 2018
Exadata X7-2 POC with OVM
8

7. 2018 © Trivadis
Additional Information & Requirements
 Current Production database size of 18TB, annually increasing of 15%.
 Guarantee an RTO of 24h and an RPO 6h.
 Increase the number of DWH environments from 3 to 6.
 QA database should be a full production copy, while the remaining environments a
data subset.
 The sub-setting procedure should be developed by the supplier.
 The new architecture should offer fast cloning procedure.
TechEvent September 2018
Exadata X7-2 POC with OVM
9

8. 2018 © Trivadis
Current Oracle architecture
 IBM AIX P7, two Production LPAR and one QA all with capped CPUs
 PROD: 9 VCPU, 148 GB of RAM
 QA: 6 VCPU, 148 GB of RAM
 SMT4
 Distributed on two data centers, maintained by IBM Storage SVC replication
 Oracle Licenses 10 CPU Enterprise Edition with:
 Partitioning
 Diagnostic Pack
 Tuning packs
 Major Performance problems:
 Poor IO performances
 CPU bound
TechEvent September 2018
Exadata X7-2 POC with OVM
10

9. 2018 © Trivadis
Trivadis proposal
TechEvent September 2018
Exadata X7-2 POC with OVM
11

10. 2018 © Trivadis
Trivadis Proposal
After a careful evaluation Trivadis was convinced that the Exadata X7-2 was the best option in
term of:
 Customer satisfaction.
 Agility to integrate a new DWH platform inside the customer ecosystem.
Than the next question to answer: Exadata Bare Metal or Virtualized?
TechEvent September 2018
Exadata X7-2 POC with OVM
12

11. 2018 © Trivadis
Exadata X7-2: Bare Metal
Pros.
 Use the entire machine capacity
 Less servers to manage
 Pay-as-you-grow approach (COD) for software licensing is another way in which Exadata
helps to align costs with business growth
 Minimum 14 cores per DB nodes (8 for Eighth Rack)
 All additional options must follow the same allocation
 https://docs.oracle.com/cd/E80920_01/DBMLI/exadata-capacity-on-demand.htm#DBMLI147
Cons.
 No physical isolation between environments
 License costs
TechEvent September 2018
Exadata X7-2 POC with OVM
13

12. 2018 © Trivadis
Exadata X7-2: Virtualized
Pros.
 Physical isolation between environments
 OVM Hard partitioning facilitate licensing optimization
 Minimum 14 cores per DB nodes (8 cores for Eighth Rack) must be licensed for
Enterprise Edition
 For other options, it’s linked to the CPU allocation of each VM
 Two cores per database node reserved to dom0 (no license required)
 Flexible and dynamic vCPU allocation
 IO Resource Management between databases accros all VMs.
Db_unique_name must be unique across the entire Exadata
Cons.
 More complex to manage
TechEvent September 2018
Exadata X7-2 POC with OVM
14

13. 2018 © Trivadis
Trivadis Architecture based on Exadata X7-2 Virtualized
TechEvent September 2018
Exadata X7-2 POC with OVM
15
PRD PRD’
passive
Cell 1 Cell 2 Cell 3
NAS Backup
STB STB’
passive
INT’
passive
INT
Cell 1 Cell 2 Cell 3
NAS Backup
QA’
passive
QA
Site 1 Site 2
Data Gard
Replication
AD’
passive
AD
HM’
passive
HM

14. 2018 © Trivadis
24 vCPU PRD Passive24 vCPU PRD Active
Our Go Live Proposal Data Center 1
data
fra
data
fra
free
StorageServer1 StorageServer2 StorageServer3
HD1 HD2 HD3 HD4 HD5 HD6 HD1 HD2 HD3 HD4 HD5 HD6 HD1 HD2 HD3 HD4 HD5 HD6
DBServer1
8 vCPU AD Passive 8 vCPU AD Active
IO Resource Manager: Category, Inter-Database, Intra-Database (db_unique_name unique across all VClusters)
DBServer2
TechEvent September 2018
Exadata X7-2 POC with OVM
16
4 vCPU HM Passive 4 vCPU HM Active
data
fra

15. 2018 © Trivadis
8 vCPU STB Passive8 vCPU STB Active
Our Go Live Proposal Data Center 2
data
fra
data
fra
data
fra
free
StorageServer1 StorageServer2 StorageServer3
HD1 HD2 HD3 HD4 HD5 HD6 HD1 HD2 HD3 HD4 HD5 HD6 HD1 HD2 HD3 HD4 HD5 HD6
DBServer1
24 vCPU INT Passive 24 vCPU INT Active
4 vCPU QA Passive 4 vCPU QA Active
IO Resource Manager: Category, Inter-Database, Intra-Database (db_unique_name unique across all VClusters)
DBServer2
TechEvent September 2018
Exadata X7-2 POC with OVM
17

16. 2018 © Trivadis
 Dynamic host cpu reconfiguration using: xm vcpu-set
 Dynamic oracle CPU_COUNT adjustment as of Oracle Oracle 12c
- Dynamic resource management update
Exadata X7-2: Elastic Capacity on Demand
Elasticity on demand: up to 34 vCPUs per VM
DEVPRD
34 vCPUs
18 vCPUs
2 vCPUs
TechEvent September 2018
Exadata X7-2 POC with OVM
18

17. 2018 © Trivadis
Database Fast Clone
 Use ASM Sparse Disk Group for DEV environment
 Allow fast snapshot cloning to provisions new DEV database
 Smart Scan is supported!
 Need a standby from production to manage the clones
 Apply Off  Clone  Apply On
 IO Performance degradation:
 100 time slower
 35 microsecond vs 3.5 millisecond
 More info's here: https://emilianofusaglia.net/tag/asm-sparse-disk-group/
TechEvent September 2018
Exadata X7-2 POC with OVM
19

18. 2018 © Trivadis
Licensing Optimization
 Cold Failover mode
 Oracle Active/Passive 10-days-per-year
 http://www.oracle.com/us/corporate/pricing/data-recovery-licensing-070587.pdf
 18 CPU Licenses required including:
 Enterprise Edition
 Partitioning
 Diagnostic and Tuning Packs
 Single instance databases on Oracle 12.2.0.1
TechEvent September 2018
Exadata X7-2 POC with OVM
20

19. 2018 © Trivadis
POC Execution and Result
TechEvent September 2018
Exadata X7-2 POC with OVM
21

20. 2018 © Trivadis
POC Execution: the context
Our Competitor
 IBM P8
 Full Flash Storage
 Max 16 Cores with SMT8
 Tests done with 10, 12,14 cores SMT8
 Corresponding CPU licenses: 10, 12, 14
 1 database 18 TB with 90 GB of SGA
 Oracle 12.2.0.1
TechEvent September 2018
Exadata X7-2 POC with OVM
22

21. 2018 © Trivadis
POC Execution: the context
Our Environment
 Exadata X7-2 ¼ rack
 OVM Configuration
 Single instance mode
 Two-node cluster with various vCPUs configurations
 36, 28, 24, 20, 16
 Corresponding CPU licenses: 9, 7, 6, 5, 4
 1 database 18 TB with 90 GB of SGA
 Oracle 12.2.0.1
TechEvent September 2018
Exadata X7-2 POC with OVM
23

22. 2018 © Trivadis
POC Execution: the context
In Summary
 Trivadis is proposing a complete change of architecture
 IBM is just replacing P7 by P8 and adding Full Flash Storage
TechEvent September 2018
Exadata X7-2 POC with OVM
24

23. 2018 © Trivadis
POC Execution: the setup
 We had a lot of problems to setup the stuff
 We had to use October 2017 Image
 Thanks a lot to Arrow for the help!
TechEvent September 2018
Exadata X7-2 POC with OVM
25

24. 2018 © Trivadis
POC Execution : the setup
TechEvent September 2018
Exadata X7-2 POC with OVM
26
We had network problems with
the management switch!
In reality the step was failing
because the Switch was OFF

25. 2018 © Trivadis
POC Execution: the initial load
 Import took more than 54 hours for IBM
 It took around 48 hours on Exadata
 We used Multitenant to facilitate iterations during the POC
 Pluggable database snapshots
TechEvent September 2018
Exadata X7-2 POC with OVM
27
We get finally
ready to start!

26. 2018 © Trivadis
POC Execution: the result
 IBM was able to increase the load speed by a factor of four.
 But it was achieved by:
 using the 14 cores (SMT8) configuration
 Setting the optimizer to 11.2.0.4 features!
- Many ORA-00600 on stats export/imports during the load processing
 High CPU usage during the processing
 Runs with 12 and 10 cores were CPU bound
 But still performing around 2.5 better than the current state
 Because run using 14 cores was not CPU bond, they stopped at that level.
 But they did not tried to run using optimizer_feature=’12.2.0.1’!
TechEvent September 2018
Exadata X7-2 POC with OVM
28

27. 2018 © Trivadis
POC Execution: the result
 We start our first run with 36 vCPUs and we achieved a speed increase with a
factor of two.
 But it was achieved with:
 Leaving the optimize to the default 12.2.0.1 value.
 Low CPU usage
 Average IO wait time of 35 microseconds!
 Some jobs were running very badly and we discovered that the optimizer setting
was not the same used by our competitor 
 We decided to fix the underlying queries!
TechEvent September 2018
Exadata X7-2 POC with OVM
29

28. 2018 © Trivadis
POC Execution: the result
 Some queries were hinted to use optimizer_feature=‘12.1.0.2’
 Some queries were hinted to use optimizer_feature=’11.2.0.4’
 Some queries were hinted to avoid view merge
 Some queries where hinted to avoid materialize of a particular factoring
clause
TechEvent September 2018
Exadata X7-2 POC with OVM
30

29. 2018 © Trivadis
POC Execution: the result
 We were finally able to achieve the same performance result obtain by
IBM!
 We decided then to start downsizing the vCPU configuration to see what
we can get from this beast!
 Runs with 28 and 24 did not change the performances at all!
 We got 3% less performance with 20 vCPU and around 8% less with 16
vCPU!
 CPU usage was high but acceptable with the 16 vCPU configuration
TechEvent September 2018
Exadata X7-2 POC with OVM
31

30. 2018 © Trivadis
POC Execution : Conclusions
Following our different runs
 We decide to adjust our final offer to 24 vCPU
 There are still lot of optimizations to be done!
TechEvent September 2018
Exadata X7-2 POC with OVM
32

31. 2018 © Trivadis
Conclusion
TechEvent September 2018
Exadata X7-2 POC with OVM
33

32. 2018 © Trivadis
Conclusion
 We fully addressed all customer needs
 The scalability of our platform (measured with the ratio
between the number of vCPUs and the jobs execution time)
was a key success
TechEvent September 2018
Exadata X7-2 POC with OVM
34

33. Jacques Kostic, Principal Consultant
Tel. +41-79-909 7263 Jacques.Kostic@trivadis.com
Emilian Fusaglia, Principal Consultant
Tel. +41-79-909 7213 Emiliano.Fusaglia@trivadis.com
35 TechEvent September 201814.09.2018

34. Session Feedback – now
TechEvent September 201836 14.09.2018
Please use the Trivadis Events mobile app to give feedback on each session
Use "My schedule" if you have registered for a session
Otherwise use "Agenda" and the search function
If the mobile app does not work (or if you have a Windows smartphone), use your
smartphone browser
– URL: http://trivadis.quickmobileplatform.eu/
– User name: <your_loginname> (such as "svv")
– Password: sent by e-mail...