Slideshow I presented at MIT Lincoln Labs on Thursday, March 20, 2014.

1. Solid State Drive
Technology

2. Me:
• Matt Simmons
• Network &Virtualization @ NEU-CCIS
• Blog: Standalone SysAdmin
• Email: msimmons@ccs.neu.edu
• Twitter: @standaloneSA
• This Slideshow: slideshare.net/standaloneSA

3. Spinning Disks
!
Key Topics

4. Voltron Force Assemble!

5. Variables Affecting
Spinning Disk IO Rate
Platter Rotational Speed
Seek Speed
Data Density
Controller Cache

6. Logical Block Addressing
• First introduced as an abstraction layer
• Replaced CHS addressing
• Address Space is Linear (block 0 - n)
• Size of address space depends on the
standard at time of manufacture.
Disk
Geometry

7. Solid State Drive
Technology

8. NOR Flash
• Reads and writes are
atomic single-bit
• Expensive
• Small specific use cases
NAND Flash
• Reads are based on pages or
“read blocks” (4k)
• Writes are based on “erasure
blocks”
• Cheap (and getting cheaper)
• Broad use cases

9. The Magic
Insulating
Barrier
Pure
Silicon
Doped silicon capable of
holding an electrical charge

10. Quantum Tunneling

11. Doped Silicon
Single Layer Cell (SLC)
Multi-Layer Cell (MLC)
Triple-Layer Cell (TLC)

12. Gradual Destruction
Energy increases
with cell layers
Multiple cells need
multiple writes
Barrier accumulates
electrons
Electrical potential
difference of barrier
and cells disappears

13. Difficulty Going Forward
TLC
000 100
001 101
010 110
011 111
SLC
0
1
MLC
00
01
10
11
4LC
0000 0100 1000 1100
0001 0101 1001 1101
0010 0110 1010 1110
0011 0111 1011 1111

14. Read / Write Profiles
• Logical addresses abstracted from LBA
• That abstraction can cause complexity
• No seek time
• Reads are generally very fast
• Writes are comparatively slow

15. Density
• 3-Dimensional
• Charge levels
• Size of cells
• “Dot Pitch” (Cells Per Inch)
• 5nm, 3nm, 2nm
• Varies with “level” count

16. SLC / ESLC
• Low Density, Quick (25µs
Read / 200-300µ Write)
• Write endurance near 100,000
cycles
MLC / EMLC
• Reasonably High Density, Slower
(50µs Read / 600-900µs Write)
• Write endurance near 3,000
cycles
TLC
• Very High Density, Even Slower
(75µs Read / 900-1350µs Write)
• Write endurance ~ 1,000 cycles

17. Write Amplification and
Garbage Collection

18. Block Sizes
• Read Block
• 4k (aka “page”)
• Erasure Block
• (Large) multiple of 4k
• aka “block”
256KB
erasure
block size

19. Write Amplification
Written Data
Empty Cell

20. Write Amplification
Written Data
Empty Cell
New Data
Old Data

21. Write Amplification
Written Data
Empty Cell
Old Data
New Data

22. Write Amplification
Written Data
Empty Cell
New data written
over old cell

23. Garbage Collection

24. Garbage Collection

25. Garbage Collection

26. Garbage Collection

27. Garbage Collection

28. SSD Performance
Overview
• Depends on
• Number of flash chips in use
• Number of busses from the processor
• Performance of controller CPU
• Contention
• Bus speed
• Number of erasure blocks used
• Number of previous writes to flash cells

29. • Chips
• IO Busses
• CPU Cores

30. Causes of Contention
• Legitimate use
• Garbage collection
• Legitimate (but latent) useage
• IO Blender!
(Bender Blender: http://bit.ly/10vc7Sf)

31. Solid State in Practice

32. Solid State Form Factors
Removable Media Drives
PCI

33. Parts of an SSD

34. Controller
Main
Processor
I/O Bus Lanes
RAM
Cache
Battery /
SuperCapacitor
Interface

35. Flash Chips

36. Flash Controllers
• Flash Translation Layer (FTL)
• Stripe Writes
• Interpret bus instructions
• Wear Leveling
• Garbage Collection

37. Flash Translation Layer
LBA (0...n blocks)
F L A S H C H I P S

38. SSD Aspects & Concerns

39. Longevity
• Primarily determined by the class of flash
• (e)SLC, (e)MLC,TLC
• Related to wear-leveling
• Under-reported capacity
• Short-stroking improves lifetime (not speed)

40. Partition Alignment
• Performance and longevity
• As big (or bigger) issue than it was in
spinning disks
• Native 4k read blocks
• Far larger erasure blocks
• larger than is practical for block-size matching

41. TRIM/Discard
• As a command,TRIM refers to ATA-8 spec
• SCSI equivalent is UNMAP, but both are
often referred to as TRIM.
• Does not immediately delete unused blocks
• Allows for GC

42. Linux TRIM Support
• EXT4 / XFS / JFS / BTRFS - Native using
‘discard’ option
• Consider NOOP or Deadline IO scheduler
• Userland:
• fstrim (part of util-linux) for
R/W vols
• zerofree for R/O vols

43. OSX Trim Support
• Comes by default on factory-installed SSDs
• Trim-Enabler
• http://www.groths.org/trim-enabler/

44. ZFS and SSDs
• ZFS Intent Log (ZIL)
• Adaptive Replacement Cache (ARC)
• arc_summary can help you decide

45. Windows
• Native TRIM
• Win 7, Server 2012+
• ATA TRIM
• Server 2012 R2
• Manufacturer Utilities Only
• < Win 7, Server 2012

46. Monitor Health w/
S.M.A.R.T.
• S.M.A.R.T. information
• vendor-specific
• Includes flash erase count
• smartctl on Linux and Mac
• Dozens of tools on Windows (check wiki)

47. Forensics
(http://bit.ly/fast11-wei-paper)
...Our results lead to three conclusions:
!
First, built-in commands are effective, but manufacturers
sometimes implement them incorrectly.
!
Second, overwriting the entire visible address space of an SSD
twice is usually, but not always, sufficient to sanitize the drive.
!
Third, none of the existing hard drive-oriented
techniques for individual file sanitization are effective on
SSDs
Reliably Erasing Data From Flash-Based Solid State Drives
Michael Wei∗, Laura M. Grupp∗, Frederick E. Spada†, Steven Swanson∗
∗Department of Computer Science and Engineering, University of California, San Diego
†Center for Magnetic Recording and Research, University of California, San Diego

48. Questions?