Making the most of ssd in oracle11g

Making the most of Solid State Disk in
Oracle
Guy Harrison
VP R&D Database Management

© 2012 Quest Software Inc. All rights reserved.

Making the most of Solid State
Disk in Oracle 11g
Guy Harrison
Executive
Director, R&D
Business Intelligence
Software

Introductions

www.guyharrison.net
guy.harrison@quest.com
http://twitter.com/guyharrison

© 2012 Quest Software Inc. All rights reserved. Pg. 3

Agenda
• Brief History of Magnetic Disk
• Solid State Disk (SSD) technologies
• SSD internals
• Oracle DB flash cache architecture
• Performance comparisons
• SSD in Exadata
• Recommendations and Suggestions


A brief history of disk


5MB HDD circa 1956


28MB HDD - 1961
1800 RPM


The more that things change....


Moore‟s law
• Transistor density doubles every 18
months
• Exponential growth is observed in most
electronic components:
−CPU clock speeds
−RAM
−Hard Disk Drive storage density

• But not in mechanical components
−Service time (Seek latency) – limited by actuator arm
speed and disk circumference
−Throughput (rotational latency) – limited by speed of
rotation, circumference and data density


Solid State Disk


SSD to the rescue?

SSD DDR-RAM
15

SSD PCI flash
25

SSD SATA Flash
80

Magnetic Disk
4,000

0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500
Seek time (us)

Economics of SSD
$/GB
0.00 10.00 20.00 30.00 40.00 50.00 60.00

0.06
FusionIO PCI SLC SSD 53.44

0.06
FusionIO PCI MLC Duo SSD 24.92

0.05
Intel SLC SATA SSD 21.88
$/IOP
0.05 $/GB
Intel MLC SATA SSD 6.88

Seagate SAS HDD 1.00 1.53

Seagate SATA HDD 0.09 2.38

0.00 0.50 1.00 1.50 2.00 2.50
$/IOP

Tiered storage management

Main Memory

DDR SSD

Flash SSD

$/IOP
$/GB

Fast Disk (SAS, RAID 0+1)

Slow Disk (SATA, RAID 5)

Tape, Flat Files, Hadoop

12c Automatic Data Placement

• Segment on SSD tablespace ALTER TABLE …. ADD ILM
Active POLICY

• Segment on SAS tablespace TIER TO …. Tablespace AFTER 6
Frequent
Access
months of no access

• OLTP compression on SATA
Occassional
Access
tablespace COMPRESS FOR QUERY LOW
AFTER 12 months of no access
• Archive Compressed on RAID5
Dormant SATA

SSD technology and internals


Flavours of Flash SSD
 DDR RAM Drive
 SATA flash drive
 PCI flash drive
 SSD storage Server

PCI SSD vs SATA SSD

 PCI vs SATA
− SATA was designed for traditional disk drives with high latencies
− PCI is designed for high speed devices
− PCI SSD has latency ~ 1/3rd of SATA

Dell Express Flash

Higher
performance, durabili
ty, flexibility
Up to 1000x more IOPs than traditional HDD
Front loading, hot swappable
Maximum read and write lifespan
Improves workload processing
Enhances virtual environments

Express Flash PCIe-SSD Drives
The power to do more

PCIe SSD Solution Framework

PCIe SSS 0
PCIe SSS 1
PCIe SSS 2
PCIe SSS 3
SAS 0
SAS 1
SAS 2
SAS 3
SAS 4
SAS 5
SAS 6
SAS 7

Media Bay & Control
Panel

Extender Card (Adapter FF)

− Frees Up Valuable PCIe Slot
Real Estate
− 1X16 Slot Supports 4X4 PCIe
SSD Backplane
− Modular, scalable

4x Drives w/ BP
− Storage Device
− HDD form factor – fits in 2.5” carrier
− SLC: 175G & 350G

Flash SSD Technology
Storage Hierarchy:

• Cell: One (SLC) or Two (MLC) bits
• Page: Typically 4K
• Block: Typically 128-512K

Writes:

• Read and first write require single page IO
• Overwriting a page requires an erase & overwrite of the block

Write endurance:

• 100,000 erase cycles for SLC before failure
• 5,000 – 10,000 erase cycles for MLC


Flash Disk write degradation

Empty

Partially Full

 All Blocks empty:
Write time=250 us
 25% part full:
− Write time= ( ¾ * 250 us + 1/4 * 2000 us) = 687 us
 75% part full
− Write time = ( ¼ * 250 us + ¾ * 2000 us ) = 1562 us

Data Insert Free Block Pool

Insert
SSD Controller

Used Block Pool

Empty Data Page

Valid Data Page

InValid Data Page

Free Block Pool

Data Update

Update
SSD Controller

Used Block Pool

Empty Data Page

Valid Data Page

Invalid Data Page

Free Block Pool

Garbage Collection

SSD Controller

Used Block Pool

Empty Data Page

Valid Data Page

Invalid Data Page

11g DB flash Cache


Oracle DB flash cache

• Introduced in 11gR2 for OEL
and Solaris only
• Secondary cache
maintained by the
DBWR, but only when idle
cycles permit
• Architecture is tolerant of
poor flash write
performance


Buffer cache and Free buffer waits

Buffer
Read from buffer cache Free cache
Oracle process Buffer
Write to buffer cache
Waits

Free buffer waits often occur when DBWR

reads are much faster than writes....

Read from disk

Database
files Write dirty blocks to disk

Flash Cache
Buffer
Read from buffer cache cache
Oracle process
Write to buffer cache

Read from flash
cache Flash Cache
DBWR
Write clean
blocks (time
permitting)

DB Flash cache architecture is designed to accelerate
buffered reads

Read from disk Write dirty blocks to disk
Database
files

Configuration
• Create filesystem from flash device
• Set DB_FLASH_CACHE_FILE and DB_FLASH_CACHE_SIZE.
• Consider Filesystemio_options=setall


Flash KEEP pool
• You can prioritise blocks for important objects using the FLASH_CACHE
clause:


Oracle Db flash cache statistics

http://guyharrison.squarespace.com/storage/flash_insert_stats.sql


Flash Cache Efficiency

http://guyharrison.squarespace.com/storage/flash_time_savings.sql

Flash cache Contents

http://guyharrison.squarespace.com/storage/flashContents.sql

Performance tests


Test systems
•Low end system:
−Dell Optiplex dual-core 4GB RAM
−2xSeagate 7500RPM Baracuda SATA HDD
−Intel X-25E SLC SATA SSD

•Higher end system:
−Dell R510 2xquad core, 32 GB RAM
−4x300GB 15K RPM,6Gbps Dell SAS HDD
−1xFusionIO ioDrive SLC PCI SSD


Buffer Cache bottlenecks
• Flash cache architecture
avoids „free buffer waits‟ due
to flash IO, but write
complete waits can still
occur on hot blocks.
• Free buffer waits are still
likely against the database
files, due to high physical
read rates created by the
flash cache


Sorting – what we expect

Multi-pass
Disk Sort
Time

Single Pass
Disk Sort

Memory Sort

PGA Memory available (MB)

Table/Index IO CPU Time Temp Segment IO 50 50

Disk Sorts – temporary tablespace

4000

3500

3000
Multi-pass
Disk Sort 2500

Elapsed time (s)
2000

1500

Single Pass 1000
Disk Sort
500

0
300 250 200 150 100 50 0
Sort Area Size

SAS based TTS SSD based TTS
51

Redo performance – Fusion IO

Flash based redo log 291.93

CPU
Log IO

SAS based redo log 292.39

0 50 100 150 200 250 300 350
Elapsed time (s)

Concurrent redo workload (x10)

Flash based redo log 1,637 331 1,681

CPU
Other
Log File IO

SAS based redo log 1,605 397 1,944

0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500
Elapsed time (s) 53 53

Write degradation
• In theory, high sustained write IO can lead to SSD degradation
when GC fails to cope with the block erase/update cycle
• In practice, this is rarely noticeable from Oracle:
−Oracle write IO is largely asynchronous (DBWR)
−Almost all write activity has at least an equal amount of read activity
− Garbage collection and wear levelling algorithms are sophisticated in decent SSD
drives


OS level direct cache

File System/ Raw File System/ Raw
Devices/ ASM Devices/ ASM
•Temp
Tablespace Caching Block Device
• Hot Segments Regular Block Device Read-
• Hot Partitions directCache intensive, pot
• DB Flash entially
Cache ioMemory VSL ioMemory VSL massive
tablespaces
(limited to the
size of the SSD)
LUN

Exadata flash storage
• 4x96GB PCI Flash drives on each storage server (X2)
• Flash can be configured as:
− Exadata Smart Flash Cache (ESFC)
− Solid State Disk available to ASM disk groups

• ESFC is not the same as the DB flash cache:
− Maintained by cellsrv, not DBWR
− DOES support full table scans
− DOES NOT support smart scans
− Unless CELL_FLASH_CACHE= KEEP,
− Statistics accessed via the cellcli program

• Considerations for cache vs. SSD are similar


Exadata Smart FlashLog


Smart Flash Log

 Designed to reduce “outlier”
redo log sync waits DB Node

 Redo is written simultaneously
to disk and flash CellSrv
 First write to complete wins
 Introduced in Exadata storage
software 11.2.2.4
Disk Flash


All Redo log writes (16M log writes)

Flash Min Median Mean 99% Max
Log
ON 1.0 650 723 1,656 75,740
OFF 1.0 627 878 4,662 291,800


Redo log outliers

WAIT #47124064145648: nam='log file sync' ela= 710 buffer#=129938 sync scn=1266588258 p3=0 obj#=-1 tim=1347583167579790


Top 10,000 waits


Exadata 12c Smart Flash Cache Write-back
 Database writes go to flash cache
− LRU aging to mag disk
− Reads serviced by flash prior to age out
− Similar restrictions to flach cache (smart scans, etc)
 Performance claims:
− On X3 system, 1M random single block write IOPS
− On X3 system, 1.5M random single block read IOPS
− 500K write IOPS claimed for Exadata V2
− Note: random IO writes are less problematic for flash than sequential
writes.


Summary


Recommendations
• Don‟t wait for SSD to become as cheap as HDD
−Magnetic HDD will always be cheaper per GB, SSD cheaper per IO
−Oracle‟s Exadata strategy suggests that the era of SSD for Oracle is here

• Consider a mixed or tiered storage strategy
−Using DB flash cache, selective SSD tablespaces or partitions
−Use SSD where your IO bottleneck is greatest and SSD advantage is significant
−Oracle 12c ILM offers a perfect solution for data tablespaces

• DB flash cache offers an easy way to leverage SSD for OLTP
workloads, but has fewer advantages for OLAP or Data
Warehouse


How to use SSD
• Database flash cache
−If your bottleneck is single block (indexed reads) and you are on OEL or Solaris 11GR2

• Flash tablespace
−Optimize read/writes against “hot” segments or partitions

• Flash temp tablespace
−If multi-pass disk sorts or hash joins are your bottleneck

• Exadata (esp X3)
−Provides most of the advantages without much setup

• 12c ILM
−Should provide an ideal solution for DIY tiering

© 2012 Quest Software Inc. All rights reserved. 73
Pg. 73

References
• Latest version of this presentation:
http://www.slideshare.net/gharriso/ssd-and-the-db-flash-cache

• Quest whitepaper:
• http://www.quest.com/documents/landing.aspx?id=15423

• Guy‟s SSD guide
• http://guyharrison.squarespace.com/ssdguide/


Thank You
guy.harrison@quest.com
www.guyharrison.net
@guyharrison


Making the most of ssd in oracle11g

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Making the most of ssd in oracle11g

Ähnlich wie Making the most of ssd in oracle11g (20)

Mehr von Guy Harrison

Mehr von Guy Harrison (16)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Making the most of ssd in oracle11g

Hinweis der Redaktion