9. Main Memory Data Rate Trends
7000
NGM Diff
6000
Data Rate (Mtps)
5000
4000
3000 NGM SE
2000
DDR2
1000 DDR
SDRAM DDR3
0
1995 2000 2005 2010
Year
DRAM bandwidth requirements typically double
every 3 years
10. Memory Trends
But latency is actually getting WORSE
And power is a problem
What drives memory evolution today?
21. DRAM Can Be Fast
Random 16-Byte Transfers Max Envelope
4,000
3,500
Bandwidth per device (MB/s / dev)
3,000
2,500
2,000
1,500
1,000
DDR2
500
DDR3
GDDR3
0
RLDRAM®
Access pattern: 8 READS followed by 8 WRITES
26. Storage Demand
161 exabytes of digital data were
generated in 2006
That’s about 168 million terabytes, or roughly the equivalent of:
36
1 43
billion
million copies trillion
digital
of every book digital
movies songs
in the Library
of Congress
Sources: IDC, UC Berkeley, CIA World Fact Book, USA TODAY Research
27. DRAM-to-Disk Evolution
“Flash is Disk, Disk is Tape”
Performance, not capacity, is the issue
Disk will continue as the $/bit leader
NAND pricing is on a steep decline
28. SSDs vs. HDDs
• Based on recent
advances in NAND
SSD HDD
lithography, SSD
Capacity
densities have
reached capacities
Performance
for mass market
Reliability
appeal
Endurance
Power • SSD offers many
features that lead
Size
to improved user
Weight
experiences
Shock
• Early shortcomings
Temperature
for reliability and
Cost per bit
endurance have
been overcome
NAND solid state storage devices are ready for deployment in many applications
NAND solid state storage devices are ready for deployment in many applications
30. NAND in Notebooks and
Consumer
Hard Disk Solid State Hard Disk Hybrid Hard
Drive Drive Drive Drive
Average
1.8” HDD SSD (1.8”/2.5”) 2.5” HDD 2.5” HHD
Specifications
Capacity 30-80GB 4-32GB 40-160GB Up to 160GB
Data Rate (max
sustain)
25 MB/s 57 MB/s 44 MB/s
Read
Write 25 MB/s 32 MB/s 44 MB/s
Spindle Speed 4200 RPM None 5400 RPM 5400 RPM
Seek 15ms None 12ms 12.5ms
Non op shock 1500G 2000G 900G 900G
SSD and HHD both provide power savings in various applications,
but the exact power savings fluctuate from application to application
In a test of a 32GB SSD drive, the power savings of the SSD was 1
watt better than the closest HDD tested
Source: Web-Feet Research, Seagate, Tom’s Hardware
31. What to Look for in an SSD
SSD-optimized controller
Parallel NAND channels
NAND
NAND
NAND
NAND
NAND
NAND
NAND
NAND
SATA-II Controller
NAND
NAND
NAND
NAND
NAND
NAND
NAND
NAND
32. SSDs in the Enterprise
CPU
Relative Relative
Latency Cost/bit
1 L1 Cache 200
2.5 L2 Cache 140
35 L3 Cache 120
300 DRAM 8
NAND 250,000 SSD 3
25,000,000 HDD 0.7
NAND Flash closes the latency gap
Cost/bit data as of Aug ‘06
33. Data Center Issues
Observed Failure Rates
Fails/
Part
Fails
System Source Type
Year
Years
SCSI
Power 858 24 2.8%
10 krpm
TerraServer
Barclay
controllers 72 2 2.8%
SAN
Reliability san switch 9 1 11.1%
Space SATA
TerraServer
138 10 7.2%
Barclay
7 krpm
Brick
Performance SCSI
15,805 972 6.0%
10 krpm
Web
anon
Property 1
Controllers 900 139 15.4%
PATA
22,400 740 3.3%
Web 7 krpm
anon
Property 2
motherboard 3,769 66 1.7%
Source: Microsoft Research
34. Reliability and Endurance
Effect Description Observed as… Management
Cells not being
Increased read
programmed
ECC and Block
Program
errors immediately
receive charge
Management
Disturb
after programming
via elevated
voltage stress
Cells not being
Increased read
read receive
Reliability Read ECC and Block
error at high
charge via
Management
Disturb number of reads
elevated voltage
stress
ECC and Block
Increased read
Charge loss over
Data
Management
errors with time
time
Retention
Failed
Endurance/ Cycles cause
Retire Block
program/erase
charge trapped in
Endurance
Cycling status
dielectric
NAND failure mechanisms are well understood and managed
NAND failure mechanisms are well understood and managed
35. Management Stack
Application Flash Translation Layer
Interfaces to traditional HDD File System
Enables sector I/O to Flash
Operating System Wear leveling
Bad block management
Automatic reclamation of erased blocks
File System Power loss protection
Manages multiple NAND devices
HDD SSD
Flash Controller
Controller
Translation Manages physical protocol
ECC Layer (FTL) NAND command encoding
High speed data transport (DMA/FIFO)
Controller
ECC Error Control
Algorithm to control sectorlevel bit reliability
sector
Implemented in hardware with software control
NAND Algorithm depends upon Flash technology
36. SSD and HDD Reliability
Hard Disk Drive Solid State Drive
Application Data
Application Data
Application Error Rate < 10-15
Error Rate < 10-15
Bad Block
Bad Block
Bad Block
Bad Block
Management
Management
Management
Data Mgmt. Management
Channel and
Channel and Block Coding
Block Coding
Block Coding
Block Coding
Typical Typical Raw
Raw Media Raw Error Error Rate
Rate >10-4 <10-5
37. SSD Quality and Reliability
NAND
Extended operation of
NAND
Ongoing production
management to ensure
reliability
Management
NANDvalidated error
correction
Static and dynamic wear
leveling
Garbage collection
Bad block remapping
Other proprietary
SSD Specs
NAND Specs schemes
10year operating life
100K P/E Cycles
10-15 bit error rate
Applications
1bit ECC
1E+6 hours MTBF
Limited READ cycles Optimizations based
10year data upon management and
retention NAND
38. Endurance: Usage Example
Continuous write at MAX bus speed of 100 MB/s with a 5:1 R/W ratio
Continuous write at MAX bus speed of 100 MB/s with a 5:1 R/W ratio
30 minutes 50 years before 1M
to fill up I/O cycle limit is
disk exceeded
250 years before 1M I/O
Capacity: 32GB
cycle limit is exceeded
At 20%
utilization
Opportunities for
improvement, i.e., new
coding, will further extend
time to cycle limit
Micron flash drives are ready for deployment for various applications
Micron flash drives are ready for deployment for various applications
39. Wear Leveling
Wear leveling is a plus on SLC devices where blocks
can support up to 100,000 PROGRAM/ERASE cycles
Wear leveling is imperative on MLC devices where
blocks typically support less than 10,000 cycles
If you erased and reprogrammed a block every
minute, you would exceed the 10,000 cycle limit in
just 7 days!
60 x 24 x 7 = 10,080
Rather than cycling the same block, wear leveling
involves distributing the number of blocks that are
cycled
40. Wear Leveling (continued)
An 8Gb MLC device contains 4,096 independent blocks
If we took the previous example and distributed the
cycles over all 4,096 blocks, each block would have
been programmed less than 3 times (versus the 10,800
cycles when you cycle the same block)
If you provided perfect wear leveling on a 4,096 block
device, you could erase and program a block every
minute, every day for 77 years!
10,000 X 4,096 40,960,00
= = 28,444 days = 77.9 years
60 X 24 1,440
41. ECC Code Selection is
Becoming More Important
1.0E-25
t=5
4
t=6
3
1
2
1.0E-23
t=
t=
t=
t=
1.0E-21
Application Bit Error Rate
1.0E-19
For SLC
1.0E-17
A code with a correction
1.0E-15
=0
threshold of 1 is sufficient
t
1.0E-13
1.0E-11
t = 4 required (as a
1.0E-09
minimum) for MLC
1.0E-07
1.0E-05
1.0E-03
1.0E-01
1.0E-01 1.0E-03 1.0E-05 1.0E-07 1.0E-09 1.0E-11 1.0E-13 1.0E-15
Raw NAND Bit Error Rate
As the raw NAND Flash BER increases, matching the
ECC to the application’s target BER becomes more
important
42. Meaningful Cycling Metrics
Practical, testable solutions are needed
Simply stating “the drive must meet 1
million complete READ and WRITE cycles”
is not realistic 1M
100K
Years for
one complete
Cycles
pass
10K
13
1K
Capacity (%) 100%
43. Cycling for CE Applications
1M Years for
one complete
pass
0.93
100K
Cycles
10K
1K
5% 20% 75%
Capacity (%)
44. Cycling for Servers
1M Years for
one complete
pass
3.02
100K
Cycles
10K
1K
20% 30% 50%
Capacity (%)
45. Cycling for Enterprise
10M Years for
one complete
pass
6.78
1M
100K
Cycles
10K
1K
5% 20% 75%
Capacity (%)
46. Call to Action
Close the gaps!
Innovation opportunities exist close to the
CPU with DRAM-based caches
Innovation opportunities are being
enabled by rapid NAND scaling for
NAND-based storage
47. Additional Resources
Web Resources
Specs: http://www.micron.com/winhec07
Whitepapers: http://www.micron.com/winhec07
Related Sessions
Main Memory Technology Direction
Flash Memory Technology Direction
E-Mail Address
daklein@micron.com
