SlideShare a Scribd company logo
1 of 40
Download to read offline
Graphical ZFS
Management Tools
Dru Lavigne
Documentation Lead, iXsystems
Texas LinuxFest, June 14, 2014
Outline
Discuss ZFS features and describe the available
management utilities for the following FreeBSD-
based operating systems:
- FreeNAS 9.2.1.5: open source NAS (Network
Attached Storage)
- PC-BSD 10.0.2: open source desktop (GUI) or
server (CLI)
History of ZFS
Modern filesystem specifically designed to add
features not available in traditional filesystems
Originally developed at Sun with the intent to
open source
After the Oracle acquisition, open source
development continued and the original engineers
founded OpenZFS (open-zfs.org) which is under
active development
OpenZFS uses feature flags instead of versions
What is ZFS?
128-bit COW (Copy on Write) filesystem and
logical volume manager with a maximum pool/file
size of 16 exabytes
In a traditional Unix filesystem, you need to define
the partition size and mount point at filesystem
creation time
In ZFS, you instead feed disks to a “pool” and
create filesystems from the pool as needed
Pool
Root (parent) volume which can be logically sub-
divided as needed
The number of disks added at a time is known as
a “vdev”
To optimize performance and resilvering time,
number of disks per vdev is limited
As more capacity is needed, add identical vdevs--
these will be striped into the pool
RAIDZ
RAIDZ* levels designed to overcome hardware
RAID limitations such as the write-hole and
corrupt data written over time before the controller
provides an alert
Designed for commodity disks so no RAID
controller is needed
Can also be used with a RAID controller, but it
typically should be put into JBOD mode
RAIDZ1
Parity blocks are distributed across all disks
Up to one disk can fail per vdev without losing
pool
Pool can be lost if second disk in a vdev fails
before resilver completes
RAIDZ2
Double-parity solution similar to RAID6
Parity blocks are distributed across all disks
Up to two disks can fail per vdev without losing
pool, with no restrictions on which disks can fail
RAIDZ3
Triple-parity solution
Parity blocks are distributed across all disks
Up to three disks can fail per vdev without losing
pool, with no restrictions on which disks can fail
Create Pool on FreeNAS
Create Pool on PC-BSD
ZIL
ZFS Intent Log
Effectively a filesystem journal that stores sync
writes until they are committed to the pool
A dedicated SSD as a secondary log device
(SLOG) can increase synchronous write
performance, will have no effect on asynchronous
writes
FreeNAS includes the zilstat CLI utility to help
determine if system would benefit from a SLOG
ARC and L2ARC
ARC refers to read cache in RAM. Takes time for
ARC to populate with hits; if high misses continue
for cached reads, the system needs to be tuned.
Freenas adds ARC stats to top(1) and includes
arc_summary.py and arcstat.py tools for ARC
monitoring
Optional, secondary ARC can be installed on SSD
or disk in order to increase random read
performance. Always add as much RAM as
possible first.
Adding SLOG/L2ARC on FreeNAS
Adding SLOG/L2ARC
During PC-BSD Install
Adding SLOG/L2ARC
Using PC-BSD Disk Manager
Datasets
As needed, pool can be divided into additional,
dynamically sized filesystems known as datasets
Permissions and properties such as quotas and
compression can be set on a per-dataset level
A well thought out design can optimize storage for
the type of data being stored
Properties
Dozens of configurable properties such as: atime
(access time), canmount, compression, copies,
dedup, exec, quota, userquota, groupquota,
readonly, recordsize, reservation, setuid, etc.
Descriptions can be found at
http://www.freebsd.org/cgi/man.cgi?query=zfs
Adding Dataset on FreeNAS
Adding Dataset During
PC-BSD Installation
Adding Dataset Using PC-BSD
Disk Manager
Zvols
Pool can also be divided into zvols
Essentially, a virtual, raw block device
Ideal for iSCSI device extents or for hosting
foreign file systems
Regardless of the filesytem the zvol is formatted
with by the iSCSI initiator, the underlying disk
blocks still benefit from all of the features provided
by ZFS
Creating Zvols on FreeNAS
Snapshots
Provide low cost, instantaneous, read-only, point-
in-time image of the specified pool, dataset, or
zvol
Snapshots can be recursive (atomic inclusion of
all child datasets)
Initial size is 0 bytes as COW, snapshot increases
in size as changes are written to disk
Can be replicated to another system
Create Snapshot on FreeNAS
Create Snapshot on PC-BSD
Using Warden
Automating Snapshots on PC-BSD
Using Life Preserver
Snapshot Restore
In PC-BSD, the Life Preserver utility provides a
snapshot browser for finding and restoring copies
of earlier versions of files
It can also automate the replication of local
snapshots to another system or to a FreeNAS
system over SSH
A remote snapshot can be used to perform an
operating system restore from a PC-BSD install
media, should the system become unusable
Restoring Data from
a PC-BSD Snapshot
Restoring the OS From
a Remote Snapshot
Scrubs
ZFS was designed to be self-healing; it creates
and verifies checksums as data is written to disk
A scrub verifies the checksum in each disk block
and attempts to correct data as necessary
I/O intensive, so should be scheduled
appropriately
Reading the scrub results can provide an early
indication of possible disk failure
Scrubs
In FreeNAS, a scrub is automatically scheduled to
run every Sunday at midnight whenever a
pool/volume is created (this can be edited)
The results of the last scrub can be viewed from
Volume Status or by typing “zpool status”, and a
scrub can be started now from View Volumes
In PC-BSD, a scrub can be started from Disk
Manager or Life Preserver
Scheduling Scrubs on FreeNAS
Starting a Scrub on PC-BSD
Deduplication
ZFS property which avoids writing duplicate data
Can improve storage efficiency at the price of
performance—compression is often the better
choice
Dedup tables must fit into L2ARC, rule of thumb
is at least 5 GB RAM/L2ARC per TB of storage to
be deduplicated
PC-BSD Boot Environments
A snapshot of the dataset the operating system
resides on is automatically taken before a system
or package upgrades
This saved “boot environment” is automatically
added to the GRUB boot manager
Should the upgrade fail, simply reboot and select
the previous boot environment from the boot
menu
Managing PC-BSD Boot
Environments
Managing PC-BSD Boot
Environments
Additional Resources
PC-BSD Users Handbook: wiki.pcbsd.org
FreeNAS User Guide: doc.freenas.org
ZFS Best Practices Guide: http://ow.ly/oHtP3
Becoming a ZFS Ninja:
https://blogs.oracle.com/video/entry/becoming_a_
zfs_ninja
Questions
Contact:
dru@freebsd.org
URL to Slides:
http://slideshare.net/dlavigne/tlf2014

More Related Content

What's hot

Lavigne bsdmag march12
Lavigne bsdmag march12Lavigne bsdmag march12
Lavigne bsdmag march12Dru Lavigne
 
Lavigne bsdmag apr13
Lavigne bsdmag apr13Lavigne bsdmag apr13
Lavigne bsdmag apr13Dru Lavigne
 
Lavigne bsdmag sept12
Lavigne bsdmag sept12Lavigne bsdmag sept12
Lavigne bsdmag sept12Dru Lavigne
 
Scale 2010: BSD for Linux Users
Scale 2010: BSD for Linux UsersScale 2010: BSD for Linux Users
Scale 2010: BSD for Linux UsersDru Lavigne
 
Dru lavigne servers-tutorial
Dru lavigne servers-tutorialDru lavigne servers-tutorial
Dru lavigne servers-tutorialDru Lavigne
 
101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2Acácio Oliveira
 
Lavigne bsdmag may13
Lavigne bsdmag may13Lavigne bsdmag may13
Lavigne bsdmag may13Dru Lavigne
 
Recipe of a linux Live CD (archived)
Recipe of a linux Live CD (archived)Recipe of a linux Live CD (archived)
Recipe of a linux Live CD (archived)Bud Siddhisena
 
2.1 design hard disk layout v2
2.1 design hard disk layout v22.1 design hard disk layout v2
2.1 design hard disk layout v2Acácio Oliveira
 
Installing Linux: Partitioning and File System Considerations
Installing Linux: Partitioning and File System ConsiderationsInstalling Linux: Partitioning and File System Considerations
Installing Linux: Partitioning and File System ConsiderationsKevin OBrien
 

What's hot (20)

Lavigne bsdmag march12
Lavigne bsdmag march12Lavigne bsdmag march12
Lavigne bsdmag march12
 
Ilf2012
Ilf2012Ilf2012
Ilf2012
 
Asiabsdcon15
Asiabsdcon15Asiabsdcon15
Asiabsdcon15
 
Flourish16
Flourish16Flourish16
Flourish16
 
Fossetcon14
Fossetcon14Fossetcon14
Fossetcon14
 
Lavigne bsdmag apr13
Lavigne bsdmag apr13Lavigne bsdmag apr13
Lavigne bsdmag apr13
 
Olf2013
Olf2013Olf2013
Olf2013
 
Scale2014
Scale2014Scale2014
Scale2014
 
Lavigne bsdmag sept12
Lavigne bsdmag sept12Lavigne bsdmag sept12
Lavigne bsdmag sept12
 
Flourish11
Flourish11Flourish11
Flourish11
 
Fsoss2011
Fsoss2011Fsoss2011
Fsoss2011
 
ZFS Talk Part 1
ZFS Talk Part 1ZFS Talk Part 1
ZFS Talk Part 1
 
Scale 2010: BSD for Linux Users
Scale 2010: BSD for Linux UsersScale 2010: BSD for Linux Users
Scale 2010: BSD for Linux Users
 
Oclug 2010
Oclug 2010Oclug 2010
Oclug 2010
 
Dru lavigne servers-tutorial
Dru lavigne servers-tutorialDru lavigne servers-tutorial
Dru lavigne servers-tutorial
 
101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2101 2.1 design hard disk layout v2
101 2.1 design hard disk layout v2
 
Lavigne bsdmag may13
Lavigne bsdmag may13Lavigne bsdmag may13
Lavigne bsdmag may13
 
Recipe of a linux Live CD (archived)
Recipe of a linux Live CD (archived)Recipe of a linux Live CD (archived)
Recipe of a linux Live CD (archived)
 
2.1 design hard disk layout v2
2.1 design hard disk layout v22.1 design hard disk layout v2
2.1 design hard disk layout v2
 
Installing Linux: Partitioning and File System Considerations
Installing Linux: Partitioning and File System ConsiderationsInstalling Linux: Partitioning and File System Considerations
Installing Linux: Partitioning and File System Considerations
 

Similar to Tlf2014

OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...NETWAYS
 
Zettabyte File Storage System
Zettabyte File Storage SystemZettabyte File Storage System
Zettabyte File Storage SystemAmdocs
 
Zettabyte File Storage System
Zettabyte File Storage SystemZettabyte File Storage System
Zettabyte File Storage SystemAmdocs
 
Root file system for embedded systems
Root file system for embedded systemsRoot file system for embedded systems
Root file system for embedded systemsalok pal
 
Network Storage dan Filesystem.pdf
Network Storage dan Filesystem.pdfNetwork Storage dan Filesystem.pdf
Network Storage dan Filesystem.pdfTaseigerKu
 
Xen server storage Overview
Xen server storage OverviewXen server storage Overview
Xen server storage OverviewNuno Alves
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File SystemNtu
 
ZFS Tutorial USENIX LISA09 Conference
ZFS Tutorial USENIX LISA09 ConferenceZFS Tutorial USENIX LISA09 Conference
ZFS Tutorial USENIX LISA09 ConferenceRichard Elling
 
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)Hari Shankar Sreekumar
 
The basic concept of Linux FIleSystem
The basic concept of Linux FIleSystemThe basic concept of Linux FIleSystem
The basic concept of Linux FIleSystemHungWei Chiu
 
I/O System and Case study
I/O System and Case studyI/O System and Case study
I/O System and Case studyLavanya G
 

Similar to Tlf2014 (20)

Nycbsdcon14
Nycbsdcon14Nycbsdcon14
Nycbsdcon14
 
Posscon2013
Posscon2013Posscon2013
Posscon2013
 
ZFS
ZFSZFS
ZFS
 
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...
OSDC 2016 - Interesting things you can do with ZFS by Allan Jude&Benedict Reu...
 
Zettabyte File Storage System
Zettabyte File Storage SystemZettabyte File Storage System
Zettabyte File Storage System
 
Zettabyte File Storage System
Zettabyte File Storage SystemZettabyte File Storage System
Zettabyte File Storage System
 
Root file system for embedded systems
Root file system for embedded systemsRoot file system for embedded systems
Root file system for embedded systems
 
Network Storage dan Filesystem.pdf
Network Storage dan Filesystem.pdfNetwork Storage dan Filesystem.pdf
Network Storage dan Filesystem.pdf
 
Xen server storage Overview
Xen server storage OverviewXen server storage Overview
Xen server storage Overview
 
Distributed File System
Distributed File SystemDistributed File System
Distributed File System
 
Sweden11
Sweden11Sweden11
Sweden11
 
FreeBSD Portscamp, Kuala Lumpur 2016
FreeBSD Portscamp, Kuala Lumpur 2016FreeBSD Portscamp, Kuala Lumpur 2016
FreeBSD Portscamp, Kuala Lumpur 2016
 
ZFS Tutorial USENIX LISA09 Conference
ZFS Tutorial USENIX LISA09 ConferenceZFS Tutorial USENIX LISA09 Conference
ZFS Tutorial USENIX LISA09 Conference
 
Magnetic disk - Krishna Geetha.ppt
Magnetic disk  - Krishna Geetha.pptMagnetic disk  - Krishna Geetha.ppt
Magnetic disk - Krishna Geetha.ppt
 
Hadoop Architecture
Hadoop ArchitectureHadoop Architecture
Hadoop Architecture
 
os
osos
os
 
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
 
The basic concept of Linux FIleSystem
The basic concept of Linux FIleSystemThe basic concept of Linux FIleSystem
The basic concept of Linux FIleSystem
 
I/O System and Case study
I/O System and Case studyI/O System and Case study
I/O System and Case study
 
Fsoss12
Fsoss12Fsoss12
Fsoss12
 

More from Dru Lavigne (20)

Olf2018
Olf2018Olf2018
Olf2018
 
Olf2017
Olf2017Olf2017
Olf2017
 
FreeBSD System Administration Using SysAdm
FreeBSD System Administration Using SysAdmFreeBSD System Administration Using SysAdm
FreeBSD System Administration Using SysAdm
 
Asiabsdcon2017
Asiabsdcon2017Asiabsdcon2017
Asiabsdcon2017
 
Olf2016
Olf2016Olf2016
Olf2016
 
Tlf2016
Tlf2016Tlf2016
Tlf2016
 
Knoxbug2016
Knoxbug2016Knoxbug2016
Knoxbug2016
 
Lfnw2016
Lfnw2016Lfnw2016
Lfnw2016
 
Scale2016
Scale2016Scale2016
Scale2016
 
Fossetcon15
Fossetcon15Fossetcon15
Fossetcon15
 
Lfnw15
Lfnw15Lfnw15
Lfnw15
 
Scale2015
Scale2015Scale2015
Scale2015
 
Olf2014
Olf2014Olf2014
Olf2014
 
Ghc14
Ghc14Ghc14
Ghc14
 
Dru lavigne oss-sysadmin
Dru lavigne oss-sysadminDru lavigne oss-sysadmin
Dru lavigne oss-sysadmin
 
Dru lavigne oss-desktop_apps
Dru lavigne oss-desktop_appsDru lavigne oss-desktop_apps
Dru lavigne oss-desktop_apps
 
Bsd ss
Bsd ssBsd ss
Bsd ss
 
Fosscon2013
Fosscon2013Fosscon2013
Fosscon2013
 
Self2013
Self2013Self2013
Self2013
 
Tlf2013
Tlf2013Tlf2013
Tlf2013
 

Recently uploaded

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 

Recently uploaded (20)

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 

Tlf2014

  • 1. Graphical ZFS Management Tools Dru Lavigne Documentation Lead, iXsystems Texas LinuxFest, June 14, 2014
  • 2. Outline Discuss ZFS features and describe the available management utilities for the following FreeBSD- based operating systems: - FreeNAS 9.2.1.5: open source NAS (Network Attached Storage) - PC-BSD 10.0.2: open source desktop (GUI) or server (CLI)
  • 3. History of ZFS Modern filesystem specifically designed to add features not available in traditional filesystems Originally developed at Sun with the intent to open source After the Oracle acquisition, open source development continued and the original engineers founded OpenZFS (open-zfs.org) which is under active development OpenZFS uses feature flags instead of versions
  • 4. What is ZFS? 128-bit COW (Copy on Write) filesystem and logical volume manager with a maximum pool/file size of 16 exabytes In a traditional Unix filesystem, you need to define the partition size and mount point at filesystem creation time In ZFS, you instead feed disks to a “pool” and create filesystems from the pool as needed
  • 5. Pool Root (parent) volume which can be logically sub- divided as needed The number of disks added at a time is known as a “vdev” To optimize performance and resilvering time, number of disks per vdev is limited As more capacity is needed, add identical vdevs-- these will be striped into the pool
  • 6. RAIDZ RAIDZ* levels designed to overcome hardware RAID limitations such as the write-hole and corrupt data written over time before the controller provides an alert Designed for commodity disks so no RAID controller is needed Can also be used with a RAID controller, but it typically should be put into JBOD mode
  • 7. RAIDZ1 Parity blocks are distributed across all disks Up to one disk can fail per vdev without losing pool Pool can be lost if second disk in a vdev fails before resilver completes
  • 8. RAIDZ2 Double-parity solution similar to RAID6 Parity blocks are distributed across all disks Up to two disks can fail per vdev without losing pool, with no restrictions on which disks can fail
  • 9. RAIDZ3 Triple-parity solution Parity blocks are distributed across all disks Up to three disks can fail per vdev without losing pool, with no restrictions on which disks can fail
  • 10. Create Pool on FreeNAS
  • 11. Create Pool on PC-BSD
  • 12. ZIL ZFS Intent Log Effectively a filesystem journal that stores sync writes until they are committed to the pool A dedicated SSD as a secondary log device (SLOG) can increase synchronous write performance, will have no effect on asynchronous writes FreeNAS includes the zilstat CLI utility to help determine if system would benefit from a SLOG
  • 13. ARC and L2ARC ARC refers to read cache in RAM. Takes time for ARC to populate with hits; if high misses continue for cached reads, the system needs to be tuned. Freenas adds ARC stats to top(1) and includes arc_summary.py and arcstat.py tools for ARC monitoring Optional, secondary ARC can be installed on SSD or disk in order to increase random read performance. Always add as much RAM as possible first.
  • 17. Datasets As needed, pool can be divided into additional, dynamically sized filesystems known as datasets Permissions and properties such as quotas and compression can be set on a per-dataset level A well thought out design can optimize storage for the type of data being stored
  • 18. Properties Dozens of configurable properties such as: atime (access time), canmount, compression, copies, dedup, exec, quota, userquota, groupquota, readonly, recordsize, reservation, setuid, etc. Descriptions can be found at http://www.freebsd.org/cgi/man.cgi?query=zfs
  • 19. Adding Dataset on FreeNAS
  • 21. Adding Dataset Using PC-BSD Disk Manager
  • 22. Zvols Pool can also be divided into zvols Essentially, a virtual, raw block device Ideal for iSCSI device extents or for hosting foreign file systems Regardless of the filesytem the zvol is formatted with by the iSCSI initiator, the underlying disk blocks still benefit from all of the features provided by ZFS
  • 23. Creating Zvols on FreeNAS
  • 24. Snapshots Provide low cost, instantaneous, read-only, point- in-time image of the specified pool, dataset, or zvol Snapshots can be recursive (atomic inclusion of all child datasets) Initial size is 0 bytes as COW, snapshot increases in size as changes are written to disk Can be replicated to another system
  • 26. Create Snapshot on PC-BSD Using Warden
  • 27. Automating Snapshots on PC-BSD Using Life Preserver
  • 28. Snapshot Restore In PC-BSD, the Life Preserver utility provides a snapshot browser for finding and restoring copies of earlier versions of files It can also automate the replication of local snapshots to another system or to a FreeNAS system over SSH A remote snapshot can be used to perform an operating system restore from a PC-BSD install media, should the system become unusable
  • 29. Restoring Data from a PC-BSD Snapshot
  • 30. Restoring the OS From a Remote Snapshot
  • 31. Scrubs ZFS was designed to be self-healing; it creates and verifies checksums as data is written to disk A scrub verifies the checksum in each disk block and attempts to correct data as necessary I/O intensive, so should be scheduled appropriately Reading the scrub results can provide an early indication of possible disk failure
  • 32. Scrubs In FreeNAS, a scrub is automatically scheduled to run every Sunday at midnight whenever a pool/volume is created (this can be edited) The results of the last scrub can be viewed from Volume Status or by typing “zpool status”, and a scrub can be started now from View Volumes In PC-BSD, a scrub can be started from Disk Manager or Life Preserver
  • 34. Starting a Scrub on PC-BSD
  • 35. Deduplication ZFS property which avoids writing duplicate data Can improve storage efficiency at the price of performance—compression is often the better choice Dedup tables must fit into L2ARC, rule of thumb is at least 5 GB RAM/L2ARC per TB of storage to be deduplicated
  • 36. PC-BSD Boot Environments A snapshot of the dataset the operating system resides on is automatically taken before a system or package upgrades This saved “boot environment” is automatically added to the GRUB boot manager Should the upgrade fail, simply reboot and select the previous boot environment from the boot menu
  • 39. Additional Resources PC-BSD Users Handbook: wiki.pcbsd.org FreeNAS User Guide: doc.freenas.org ZFS Best Practices Guide: http://ow.ly/oHtP3 Becoming a ZFS Ninja: https://blogs.oracle.com/video/entry/becoming_a_ zfs_ninja