EXT4 and BTRFS from an operational point of view (autumn/fall 2011)

1. Quo vadis Linux File Systems: An
operations point of view on
EXT4 and BTRFS
Udo Seidel

2. Agenda
● Introduction/motivation
● EXT4 – the new member of the extfs family
● Facts, specs
● Operational considerations
● BTRFS – the newbie .. the hope
● Facts, specs
● Operational considerations
● Summary
LinuxCon Europe 26-28/10 2011 2

3. Me ;-)
● Teacher of mathematics and physics
● PhD in experimental physics
● Started with Linux in 1996
● Linux/UNIX trainer
● Solution engineer in HPC and CAx environment
● Joined Amadeus in 2006
● Head of a international team with people in
Erding, Miami and Sydney
LinuxCon Europe 26-28/10 2011 3

4. File systems challenges
● Speed
● Size/growth
● Flexibility
● Sustainability
LinuxCon Europe 26-28/10 2011 4

5. Linux file systems
● More than 50 file systems shipped with Linux
kernel
● Local
● Remote
● Cluster
● ...
● A few as standard for root directory
● EXT2, EXT3
● XFS
LinuxCon Europe 26-28/10 2011 5

6. Operational challenges with Linux
file systems
● ReiserFS
● sun-setted
● Big installation base
● EXT3
● Stable, robust
● Slow
● XFS
● Additional costs?
● Changes in recent Enterprise distributions
LinuxCon Europe 26-28/10 2011 6

7. Chances, possibilities, choices for
the next Linux file systems
● New version of the extfs family -> EXT4
● Marked as stable
● Shipped with Enterprise distributions
● New approach with BTRFS
● Still experimental in vanilla kernel
● Shipped as production ready by some Enterprise
distributions
LinuxCon Europe 26-28/10 2011 7

8. th
4 extended file system
● Shipped since 2.6.19
● Stable since 2.6.28
● To overcome limits of EXT3
● Size
● Performance
LinuxCon Europe 26-28/10 2011 8

9. EXT4 - facts
● Max volume size: 1 EB = 1024 PB (EXT3: 16
TB)
● Max file size: 16 TB (EXT3: 2 TB)
● Max length of file name: 256 Bytes
● Support of extended attributes
● No encryption
● Not really compression
● Partially 64bit
LinuxCon Europe 26-28/10 2011 9

10. EXT4 – operational first glance
● Looks promising
● OPROC/procedure changes?
● Behaviour changes?
● Migration
LinuxCon Europe 26-28/10 2011 10

11. EXT4 – OPROCs/procedures
● Known tools
● mkfs
● fsck
● tune2fs
● e2label
=> Easy to re-use or adapt of existing :-)
LinuxCon Europe 26-28/10 2011 11

12. EXT4 – Behaviour changes ...
● Extents
● Improved block allocation
● Journaling
● File system check
LinuxCon Europe 26-28/10 2011 12

13. EXT4 – from blocks to extents
● Common addressing for modern file systems
● Overdue
● Contiguous area of blocks
– Less management information needed
– Less meta data operations
– Less “fragmentation”
● Requires change of on-disk format :-(
● Migration paths?
LinuxCon Europe 26-28/10 2011 13

14. EXT4 – delayed allocation
● Use cache information for placement
● Less fragmentation
● Risk of data loss in early versions => improved
since 2.6.30
● Easy switch on/off
● No change of on-disk format! :-)
LinuxCon Europe 26-28/10 2011 14

15. EXT4 – “clever” allocation
● Support of system call fallocate()
● Application reserves blocks ahead
● File system ensures disk space availability
● Allocation information in extent structure
● Only usable with extents!
LinuxCon Europe 26-28/10 2011 15

16. EXT4 – consistency via JBD2
● Transactions have checksums
● 64 bit ready
● Deactivation possible
● Keep other EXT4 features
● Comparison: EXT3 ... -> EXT2
● Reduction of file systems in use :-)
LinuxCon Europe 26-28/10 2011 16

17. EXT4 – repair
● Improved fsck()
● No check of unused blocks
– information stored in block group header
– Information secured via checksums
– (de)activation possible at any time
● First run possibly as slow like in EXT3
LinuxCon Europe 26-28/10 2011 17

18. EXT4 – other news
● Nano second precision time stamps
● Unix millennium bug shifted to 2514
● More subdirectories
● Up to 65000
● More than 65000 ... with limitation
LinuxCon Europe 26-28/10 2011 18

19. EXT4 – background for migration
● 2 kind of changes compared to EXT3
● change of on-disk format:
– Extents
– Only enabled for new files via tune2fs
– Additional tasks needed
● On-disk format not relevant:
– block allocation
– Immediately enabled via tune2fs
LinuxCon Europe 26-28/10 2011 19

20. EXT4 – general migration paths
● mkfs() and backup/restore
● Clean new file system structure
● Only way for file systems other than EXT2/3
● Extended outage
● Conversion via tune2fs
● Partial only
● Possible for EXT2/3 family only
● Faster/easier
LinuxCon Europe 26-28/10 2011 20

21. EXT4 – migration strategy
● It depends
● For us: both options
● Significant installation base of non-EXT2/3
● Big landscape (2500+ Linux servers)
LinuxCon Europe 26-28/10 2011 21

22. EXT4 – migration via mkfs()
● For all new deployments
● for existing file systems
● On demand
● By chance
LinuxCon Europe 26-28/10 2011 22

23. EXT4 – migration via tune2fs
● Results in mix of EXT2/3 and EXT4 structure
● Access via EXT2/3 driver impossible
● fsck() needed
parameter description
extent Extent based block allocation
flex_bg Flexible placement of meta data
uninit_bg Flag uninitialized blocks for faster fsck
dir_nlink Infinite number of sub directories
extra_isize Timestamps with nano seconds
LinuxCon Europe 26-28/10 2011 23

24. EXT4 – migration hints
● fsck() recommended
● /boot – booting from EXT4 possible?
● Rescue media enabled for EXT4?
● Backup/restore of EXT4?
LinuxCon Europe 26-28/10 2011 24

25. EXT4 – summary
● Good successor of EXT3
● Manages higher amount of data
● Faster
● Performance
● recovery
● Safer
● Sufficient migration options from/to EXT2/3
=> GO!
LinuxCon Europe 26-28/10 2011 25

26. Better/b-tree file system
● Shipped since 2.6.29
● Still experimental
● In vanilla kernel
● But .... production ready in some Enterprise
distributions
● Supposed to replace EXT3/4
● New storage management approach
LinuxCon Europe 26-28/10 2011 26

27. BTRFS - facts
● Max volume/file size: 16 EB
● Max length of file name: 256 Bytes
● Support of
● Extended attributes
● No encryption yet
● Snapshot
● Compression
● Copy-on-Write
LinuxCon Europe 26-28/10 2011 27

28. BTRFS – operational first glance
● OPROC/procedure changes?
● Behaviour changes?
● Migration
LinuxCon Europe 26-28/10 2011 28

29. BTRFS – OPROCs/procedures
● Known tools
● mkfs ... with limitation
● fsck ... with limitation
● tune2fs -> nope
● XXlabel -> nope
=> No easy to re-use or adapt of existing :-(
LinuxCon Europe 26-28/10 2011 29

30. BTRFS – behaviour changes ...
● Device management
● Snapshots
● Compression
LinuxCon Europe 26-28/10 2011 30

31. BTRFS – device management
● Included volume manager
● RAID-0, RAID-1, ...
● Add/remove devices
● ...
● New tool
● change of OPROC's/procedures :-(
● Change to CLI/GUI or new CLI/GUI tools :-(
LinuxCon Europe 26-28/10 2011 31

32. BTRFS – snapshots
● Not really new ... but not used so far
● Integration into package management
● automatic - no additional tasks
● Faster roll-back
● Post change tracking
● New tool
● New OPROC's anyway :-|
● Minimal human intervention :-)
LinuxCon Europe 26-28/10 2011 32

33. BTRFS – compression
● Transparent
● Flexible:
● different algorithms possible
● Easy to switch on/off
● Operations:
● Minimal changes on admin side
● No changes on other levels :-)
LinuxCon Europe 26-28/10 2011 33

34. BTRFS – file system check
● Yes ... there is something ....
● Reliable?
● Use of alternatives?!
● Backup/restore
– Possible longer outage :-(
– covered in OPROC's :-)
● Snapshots -> new at all :-(
LinuxCon Europe 26-28/10 2011 34

35. BTRFS – what else
● Support of POSIX ACL's
● Online grow/shrink
● Online add/removal of disks
● SSD-aware
● Management tool evolution (btrfsctl -> btrfs)
● du/df not fully BTRFS-aware
LinuxCon Europe 26-28/10 2011 35

36. BTRFS – migration paths
● mkfs() and backup/restore
OR
● In place from EXT3/4 via tool btrfs-convert
● Via libe2fs
● BTRFS meta data location flexible
● Old EXT3/4 organized in snapshot
● Roll-back possible to date/time of conversion
LinuxCon Europe 26-28/10 2011 36

37. BTRFS – migration strategy
● In place via btrfs-convert
● Fast
● EXT3/4 a given in data centre by then
● Change management friendly
LinuxCon Europe 26-28/10 2011 37

38. BTRFS – migration hints
● New file system – check everything!
● General knowledge/awareness
● /boot – booting from BTRFS possible?
● Rescue media enabled for BTRFS?
● Backup/restore of BTRFS?
LinuxCon Europe 26-28/10 2011 38

39. BTRFS summary
● Still experimental
● Meets standard file systems requirements
● Bridges existing gaps
● Easy migration from EXT3/4 possible
● New approach to storage management
● Prospects
● SSD
● Compression
LinuxCon Europe 26-28/10 2011 39

40. Summary
● Improvement moving to EXT4
● Safe switching to EXT4
● In place migration from EXT3 possible
● Future is BTRFS
● In place migration from EXT3/4 to BTRFS
possible
LinuxCon Europe 26-28/10 2011 40

41. References
● http://ext4.wiki.kernel.org
● http://btrfs.wiki.kernel.org
LinuxCon Europe 26-28/10 2011 41

42. Thank you!
LinuxCon Europe 26-28/10 2011 42