Our presentation at the South African Oracle User Group Conference on the 23rd July 2013 on ZFS. ZFS is an open source file system combined with a pool manager that provides features and capabilities that surpass those of any solutions in the market today. Despite this complexity zfs is easy to setup and maintain, sometimes deceptively so. This presentations introduces zfs to new users.

1. Jumping Bean Jumping Bean
Introduction to ZFS

2. Jumping Bean Jumping Bean
What is ZFS?
● ZFS is the next generation enterprise file
system and volume manger,
● Robust, scalable and simple to administer.
● Self healing, transactional
● Two components
– Pool manager
– File system manager

3. Jumping Bean Jumping Bean
Who Am I?
● Mark Clarke
– work at Jumping Bean, an solutions integration
company,
– Social Media
● Twitter - @mxc4
● G+ - MClarke4@gmail.com
● LinkedIn
● http://www.jumpingbean.co.za

4. Jumping Bean Jumping Bean
Where did it come from?
● Released by Sun Microsystems now owned by
Oracle
● Announced September 2004
● Work started in 2001
● Open source - Licensed under the CDDL (Common
Development and Distributions License)
● Latest versions:
– ZFS File System -5/6
– ZFS Pool Version Number 28/34

5. Jumping Bean Jumping Bean
ZFS Components
ZFS File System
ZPool (storage)
storage/home storage/share

6. Jumping Bean Jumping Bean
ZFS Features
● Combined file system and logical volume
manager,
● Features:
– Protection against data corruption,
– Snapshots, clones
– Automatic repair and
– Continuous integrity checking

7. Jumping Bean Jumping Bean
ZFS Features
● 128bit file system
– Can address 1.84 × 10^19 more data than 64
bit system,
– No practical limit to
● File size,
● Directory entries
● Disk drives

8. Jumping Bean Jumping Bean
Introduction to ZFS
● Only two commands
– zpool – for creating/managing storage pools
– zfs – for creating/managing file-systems

9. Jumping Bean Jumping Bean
ZPOOL

10. Jumping Bean Jumping Bean
ZPool
● Zpool handles the storage pool,
● Responsible for:
– Data integrity
– Self healing
– Check summing
– Performance
– Vdev creation, management

11. Jumping Bean Jumping Bean
ZPool
● Stripes data across vdevs
● ZPOOL components
– ARC
– L2ARC
– ZIL (ZFS Intent Log)
– COW (copy on writes)
– Transaction groups

12. Jumping Bean Jumping Bean
Why Do we Have Data
Integrity Issues?
● Data faults/corruption occur because:
– Bit rot,
– Current spikes,
– Firmware bugs
– Phantom writes,
– Misdirected read/writes
– Raid “Write holes”

13. Jumping Bean Jumping Bean
Data Integrity
● Silent data corruption
– Errors undetected by firmware and/or operating
system
– Netapp study found 1 in 90 SATA drives had
silent software corruption,
– Faster disks/raid controllers + larger capacity =
problem
– Jeff Bonwick estimates silent corruption every
15 minutes at Greenplum

14. Jumping Bean Jumping Bean
How ZFS handles Integrity
● ZFS Data Integrity handle by COW
transactional data writes
– Uses Hash Tree (Merkle Tree)
● Each block checksummed, stored in pointer to block,
● Each pointer checksums stored in pointer, etc – up
to root block,
● Uber block has check sum,
● Checksum compared on block access,

15. Jumping Bean Jumping Bean
ZFS Copy on Write

16. Jumping Bean Jumping Bean
ZFS Auto Healing
● On check sum check error , zpool will
rebuild data from redundant storage and
repair bad blocks,

17. Jumping Bean Jumping Bean
Zpool Vdevs
● Zpool works with virtual devices (vdevs)
● Vdevs can be:
– Single disks
– N-way mirrors
– Raidz1,
– Raidz2,
– RaidDz3

18. Jumping Bean Jumping Bean
ZPool Vdevs
● RAID-Z
– RAID-Z1 - one disk failure ~ Raid 5
– RAID-Z2 - up to two disk failures ~ Raid 6
– Raid-Z3 – allows three disk failures
– No write whole problem due to transaction
group/cow

19. Jumping Bean Jumping Bean
Zpool - Storage Design
● Zpool uses dynamic striping across vdevs,
● Can mix any combination of vdevs
● Design storage pool with redundancy,
performance and maintainability in mind.

20. Jumping Bean Jumping Bean
Zpool - ARC
● ARC is the adaptive replacement cache
● In memory cache,
● Zpool will use all memory -1G,
● Frees up memory when requested by other apps
● More ram = better performance
● ARC uses:
– MRU,
– MFU
– MRU Ghosts – evicted pages
– MFU Ghost – evicted pages

21. Jumping Bean Jumping Bean
ZPOOL – Hybrid Pools
● Hybrid storage pools
– Use SSD for
● ZIL (write cache)
● L2Arc cache (read cache)
– Use disk for mass storage capacity
● MLC – multi layer cell (L2Arc)
● SLC – single layer cell (SLOG)

22. Jumping Bean Jumping Bean
ZFS Hybrid Pools
SLOG L2Arc Raid-z1 Mirror
ZFS File System
Read
Cache
ZIL
Write
Cache
ZPool (storage)
Adaptive Replacement Cache
storage/home storage/share

23. Jumping Bean Jumping Bean
Zpool Demo

24. Jumping Bean Jumping Bean
ZFS Datasets

25. Jumping Bean Jumping Bean
What does ZFS File System
Provide?
● ZFS file system provides
– Compression
– Encryption
– Shares
– De-duplication,
– Quotas
– Reservation,
– Snapshot,
– Clone,
– Properties

26. Jumping Bean Jumping Bean
ZFS - File Systems
● ZFS creates datasets,
● ZFS can create and mount file system with
a single command,
● File systems mounted by default under pool
name,
● Block devices can also be create on a Zpool
and formatted with ext4 etc,

27. Jumping Bean Jumping Bean
ZFS Properties
● Using ZFS properties one can:
– Enable compression,
– Enable CIFs/NFSv4 shares
– Change mount point
– Enable de-duplication – requires lots of memory

28. Jumping Bean Jumping Bean
ZFS Snapshots & Clones
● Snapshot are efficient and cheap to create,
● Can rollback to snapshots easily
● Snapshots can be access via .zfs hidden
directory
● Snapshots read only,
● Clones read & write

29. Jumping Bean Jumping Bean
ZFS Send/Receive
● ZFS stream snapshots over stdin/stdout,
● ZFS send/receive can be done over WAN,
no special hardware required,
● Can send/receive incremental snapshots

30. Jumping Bean Jumping Bean
ZFS Dataset Demo

31. Jumping Bean Jumping Bean
Q & A

32. Jumping Bean Jumping Bean
The End