SlideShare ist ein Scribd-Unternehmen logo
1 von 50
Downloaden Sie, um offline zu lesen
High Performance, Reliable Secondary Storage

Uğur Tılıkoğlu
Gebze Institute of Technology
Overview
 Introduction
 Background
 Disk Terminology
 Data Paths
 Technology Trends
 Disk Array Basics
 Data Striping and Redundancy
 Basic Raid Organizations
 Performance and Cost Comparisons
 Reliability
 Implementation Considerations
2
Overview
 Advanced Topics
 Improving Small Write Performance for RAID Level 5
 Declustered Parity
 Exploiting On-Line Spare Disks
 Data Striping in Disk Arrays
 Performance and Reliability Modeling
 Opportunities for Future Research
 Experience with Disk Arrays
 Interaction among New Organizations
 Scalability, Massively Parallel Computers and Small Disks
 Latency
3
Introduction
 RAID: Redundant Arrays of Inexpensive / Independent

Disks
 Improvements in microprocessors and memory
systems require larger, higher-performance secondary
storage systems
 Microprocessors performance increase rate > Disk
performance increase rate
 Disk arrays: multiple, independent disks  large,
high-performance logical disk
4
Background
 Disk Terminology
 Data Paths
 Technology Trends

5
Disk Terminology

6
Data Paths

7
Technology Trends

8
Disk Array Basics
 Data Striping and Redundancy
 Basic Raid Organizations
 Performance and Cost Comparisons
 Reliability

 Implementation Considerations

9
Data Striping and Redundancy
 Data Striping
 Distribute data over multiple disks
 Service in parallel
 More disks  More performance

10
Data Striping and Redundancy
 More disks  More unreliable
 100 disks  1/100 reliability of a single disks
 Redundancy
 Two categories
 Granularity of data interleaving
 Method of computing redundant information and
distribute accross the disk array

11
Data Striping and Redundancy
 Data interleaving
 Fine grained
 Advantages:



access all the disks
high transfer rate

 Disadvantages



only one I/O request serviced at any time
All disks waste time positioning for every request

12
Data Striping and Redundancy
 Data interleaving
 Coarse grained
 Advantages:



Multiple small requests serviced simultaneously
Large requests can access all the disks

13
Data Striping and Redundancy
 Redundancy
 Two main problems
 Computing the redundant information: Parity
 Selecting a method for distributing the redundant
information accross the disk array

14
Basic Raid Organizations
 Nonredundant (RAID Level 0)
 Lowest cost
 Best write performance
 No best read performance
 Any single disk failure result data loss

15
Basic Raid Organizations
 Mirrored (RAID Level 1)
 Twice number of disks
 Data also written to redundant disk
 If a disk fails, the other copy is used

16
Basic Raid Organizations
 Memory Style ECC (RAID Level 2)
 Contain parity disks
 Parity disk proportional to data disks
 Efficiency increases when data disk number increases
 Multiple parity disks are needed to identify the failed
disk, but only one is needed to recover

17
Basic Raid Organizations
 Bit-Interleaved Parity (RAID Level 3)
 Bit-wise data is used
 Disk controller can identify which disk has failed
 A single parity disk is used
 Read  all disks, Write  all disks + parity disk

18
Basic Raid Organizations
 Block-Interleaved Parity (RAID Level 4)
 Same as Level 3 but blocks (striping units) are used
 Read & write < striping unit  one disk
 Parity calculation  xor new data with old data
 Four I/O: write new data, read old data and old parity,
write new parity
 Bottleneck at parity disk

19
Basic Raid Organizations
 Block-Interleaved Distributed-Parity (RAID Level 5)
 Solves bottleneck problem at Level 4
 Best small read, larger read and large write performance
 Small writes are inefficient because of read-modifywrite

20
Basic Raid Organizations
 P + Q Redundancy (RAID Level 6)
 Have stronger codes to solve multiple failures
 Operate in much the same manner as Level 5
 Small writes are inefficient because of 6 I/O requests
due to update both P and Q information

21
Performance and Cost
Comparisons
 Ground Rules and Observations
 Reliability, performance and cost
 Disk arrays are throughput oriented
 I/Os per second per dollar
 Configuration
 RAID 5 can operate as RAID 1 and RAID 3 by configuring
striping unit

22
Performance and Cost
Comparisons
 Comparisons – Small Read & Writes

23
Performance and Cost
Comparisons
 Comparisons – Large Read & Writes

24
Performance and Cost
Comparisons
 Comparisons – RAID 3 & 5 & 6

25
Reliability
 Basic Reliability
 RAID 5




MTTF: mean time to failure, MTTR: mean time to repair
N: total number of disks, G: parity group size
100 disks each had MTTF of 200.000 hours, MTTR of 1 hour,
partiy group size 16  mean time to failure of the system is
about 3000 years !!!

26
Reliability
 Basic Reliability
 RAID 6




MTTF: mean time to failure, MTTR: mean time to repair
N: total number of disks, G: parity group size
100 disks each had MTTF of 200.000 hours, MTTR of 1
hour, partiy group size 16  mean time to failure of the
system is about 38.000.000 years !!!

27
System Crashes and Parity
Inconsistency
 System crash: power failure, operator error, hardware

breakdown, software crash etc.
 Causes parity inconsistencies in both bit-interleaved
and block-interleaved disk arrays
 System crash may occur more frequently than disk
failures
 To avoid the loss of parity on system
crashes, information sufficient to recover parity mus
be logged on a non-volatile storage (nvram) before
each write operation.
28
Uncorrectable Bit Errors
 What is bit error? It is unclear
 Data is incorrectly written or magnetic media

gradually damaged
 Some manifacturers developed an approach; monitors
the warnings given by disks and notifies an operator
when it feels the disk is about to fail.

29
Correlated Disk Failures
 Environmental and manufacturing factors
 Example: earthquake

30
Reliability Revisited
 Double disk failure
 System crash followed by a disk failure
 Disk failure followed by an uncorrectable bit error

during reconstruction

31
Reliability Revisited

32
Reliability Revisited

33
Reliability Revisited

34
Implementation Considerations
 Avoiding Stale Data
 When a disk fails, failed disk must be marked as invalid.
Invalid mark prevents user from reading corrupted data
on the failed disk
 When an invalid logical sector is reconstructed to a
spare disk, the logical sector must be marked as valid.

35
Implementation Considerations
 Regenerating Parity after a System Crash
 Before servicing any write request, the corresponding
parity sectors must be marked inconsistent
 When bringing a system up from a system crash, all
inconsistent parity sectors must be regenerated

36
Implementation Considerations
 Operating with a Failed Disk
 Demand reconstruction: access to a parity stripe with an
invalid sector triggers reconstruction of the appropriate
data immediately onto a spare disk. A background
process scans the entire disk.
 Parity sparing: before servicing a write request, the
invalid sector is reconstructed and relocated to
overwrite its corresponding parity sector

37
Implementation Considerations
 Orthagonal Raid

38
Advanced Topics
 Improving Small Write Performance for RAID Level 5
 Declustered Parity
 Exploiting On-Line Spare Disks
 Data Striping in Disk Arrays

 Performance and Reliability Modeling

39
Improving Small Write
Performance for RAID Level 5
 Buffering and Caching
 Write buffering (async writes): Collect small writes in a
buffer and write as a large data
 Read caching: reduce four I/O access to three, old data is
read from cache

40
Improving Small Write
Performance for RAID Level 5
 Floating Parity
 Shortens the read-modify-write time
 Many free blocks
 New parity block is writted rotationally nearest
unallocated block following the old parity block
 Implemented on disk controller

41
Improving Small Write
Performance for RAID Level 5
 Floating Parity
 Shortens the read-modify-write time
 Many free blocks
 New parity block is writted rotationally nearest
unallocated block following the old parity block
 Implemented on disk controller

42
Improving Small Write
Performance for RAID Level 5
 Parity Logging
 Delaying the read of old parity and write of the new
parity
 Difference is temporarily logged
 Logs are grouped together and large contiguous blocks
are updated more efficiently

43
Declustered Parity
 Distributes the increased load uniformly over all disks

44
Exploiting On-Line Spare Disks
 Distributed Sparing

45
Exploiting On-Line Spare Disks
 Parity Sparing

46
Data Striping in Disk Arrays
 Disk positioning time is wasted work
 Idle times are same as disk positioning
 Data striping or interleaving is, distributing data

among multiple disks.
 Researchers work on data striping unit size to
maximize the throughput

47
Data Striping in Disk Arrays
 P: average disk positioning time
 X: average disk transfer rate
 L: concurrency
 Z: request size

 N: array size in disks

48
Performance and Reliability
Modeling
 Performance
 Kim: response time equations
 Kim & Tantawi: approximate service time equations
 Chen & Towsley
 Lee & Katz
 Reliability
 Markov

49
Opportunities for Future Research
 Experience with Disk Arrays
 Interaction among New Organizations
 Scalability, Massively Parallel Computers and Small

Disks
 Latency

50

Weitere ähnliche Inhalte

Was ist angesagt?

Raid data recovery Tips
Raid data recovery TipsRaid data recovery Tips
Raid data recovery TipsHone Software
 
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer ArchitectureRaid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer ArchitectureAiman Hafeez
 
Coal presentationt
Coal presentationtCoal presentationt
Coal presentationtfika sweety
 
DBMS Unit IV and V Material
DBMS Unit IV and V MaterialDBMS Unit IV and V Material
DBMS Unit IV and V MaterialArthyR3
 
Raid : Redundant Array of Inexpensive Disks
Raid : Redundant Array of Inexpensive DisksRaid : Redundant Array of Inexpensive Disks
Raid : Redundant Array of Inexpensive DisksCloudbells.com
 
Storage Devices And Backup Media
Storage Devices And Backup MediaStorage Devices And Backup Media
Storage Devices And Backup MediaTyrone Turner
 
Native erasure coding support inside hdfs presentation
Native erasure coding support inside hdfs presentationNative erasure coding support inside hdfs presentation
Native erasure coding support inside hdfs presentationlin bao
 
Mass storage structurefinal
Mass storage structurefinalMass storage structurefinal
Mass storage structurefinalmarangburu42
 
Less is More: 2X Storage Efficiency with HDFS Erasure Coding
Less is More: 2X Storage Efficiency with HDFS Erasure CodingLess is More: 2X Storage Efficiency with HDFS Erasure Coding
Less is More: 2X Storage Efficiency with HDFS Erasure CodingZhe Zhang
 
Unit3 ppt3 hard drive
Unit3 ppt3 hard driveUnit3 ppt3 hard drive
Unit3 ppt3 hard driveFarhanMalik93
 

Was ist angesagt? (20)

Raid data recovery Tips
Raid data recovery TipsRaid data recovery Tips
Raid data recovery Tips
 
RAID
RAIDRAID
RAID
 
RAID
RAIDRAID
RAID
 
Raid_intro.ppt
Raid_intro.pptRaid_intro.ppt
Raid_intro.ppt
 
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer ArchitectureRaid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
 
Coal presentationt
Coal presentationtCoal presentationt
Coal presentationt
 
Disk structure
Disk structureDisk structure
Disk structure
 
DBMS Unit IV and V Material
DBMS Unit IV and V MaterialDBMS Unit IV and V Material
DBMS Unit IV and V Material
 
Raid : Redundant Array of Inexpensive Disks
Raid : Redundant Array of Inexpensive DisksRaid : Redundant Array of Inexpensive Disks
Raid : Redundant Array of Inexpensive Disks
 
Disk management
Disk managementDisk management
Disk management
 
Storage Devices And Backup Media
Storage Devices And Backup MediaStorage Devices And Backup Media
Storage Devices And Backup Media
 
Native erasure coding support inside hdfs presentation
Native erasure coding support inside hdfs presentationNative erasure coding support inside hdfs presentation
Native erasure coding support inside hdfs presentation
 
Mass storage structurefinal
Mass storage structurefinalMass storage structurefinal
Mass storage structurefinal
 
Less is More: 2X Storage Efficiency with HDFS Erasure Coding
Less is More: 2X Storage Efficiency with HDFS Erasure CodingLess is More: 2X Storage Efficiency with HDFS Erasure Coding
Less is More: 2X Storage Efficiency with HDFS Erasure Coding
 
Raid in SNA
Raid in SNA Raid in SNA
Raid in SNA
 
Raid levels
Raid levelsRaid levels
Raid levels
 
Unit3 ppt3 hard drive
Unit3 ppt3 hard driveUnit3 ppt3 hard drive
Unit3 ppt3 hard drive
 
Ch10
Ch10Ch10
Ch10
 
File Fragmentation
File FragmentationFile Fragmentation
File Fragmentation
 
Storage memory
Storage memoryStorage memory
Storage memory
 

Ähnlich wie RAID: High-Performance, Reliable Secondary Storage

disk structure and multiple RAID levels .ppt
disk structure and multiple  RAID levels .pptdisk structure and multiple  RAID levels .ppt
disk structure and multiple RAID levels .pptRAJASEKHARV10
 
Raid Levels Technology
Raid Levels TechnologyRaid Levels Technology
Raid Levels TechnologyIshwor Panta
 
Storage systems reliability
Storage systems reliabilityStorage systems reliability
Storage systems reliabilityJuha Salenius
 
Storage, San And Business Continuity Overview
Storage, San And Business Continuity OverviewStorage, San And Business Continuity Overview
Storage, San And Business Continuity OverviewAlan McSweeney
 
Various raid levels pros &amp; cons
Various raid levels pros &amp; consVarious raid levels pros &amp; cons
Various raid levels pros &amp; consIT Tech
 
Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)
Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)
Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)Raid Data Recovery
 
3 configuring basic and dynamic disks
3 configuring basic and dynamic disks3 configuring basic and dynamic disks
3 configuring basic and dynamic disksHameda Hurmat
 
Data center core elements, Data center virtualization
Data center core elements, Data center virtualizationData center core elements, Data center virtualization
Data center core elements, Data center virtualizationMadhuraNK
 
Present of Raid and Its Type
Present of Raid and Its TypePresent of Raid and Its Type
Present of Raid and Its TypeUsama ahmad
 

Ähnlich wie RAID: High-Performance, Reliable Secondary Storage (20)

Raid
Raid Raid
Raid
 
disk structure and multiple RAID levels .ppt
disk structure and multiple  RAID levels .pptdisk structure and multiple  RAID levels .ppt
disk structure and multiple RAID levels .ppt
 
Raid Levels Technology
Raid Levels TechnologyRaid Levels Technology
Raid Levels Technology
 
RAID Levels
RAID LevelsRAID Levels
RAID Levels
 
Raid
RaidRaid
Raid
 
Database 3
Database 3Database 3
Database 3
 
Storage systems reliability
Storage systems reliabilityStorage systems reliability
Storage systems reliability
 
Class2
Class2Class2
Class2
 
Storage, San And Business Continuity Overview
Storage, San And Business Continuity OverviewStorage, San And Business Continuity Overview
Storage, San And Business Continuity Overview
 
SQL 2005 Disk IO Performance
SQL 2005 Disk IO PerformanceSQL 2005 Disk IO Performance
SQL 2005 Disk IO Performance
 
raid technology
raid technologyraid technology
raid technology
 
Various raid levels pros &amp; cons
Various raid levels pros &amp; consVarious raid levels pros &amp; cons
Various raid levels pros &amp; cons
 
Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)
Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)
Understanding RAID Levels (RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5)
 
UNIT III.pptx
UNIT III.pptxUNIT III.pptx
UNIT III.pptx
 
3 configuring basic and dynamic disks
3 configuring basic and dynamic disks3 configuring basic and dynamic disks
3 configuring basic and dynamic disks
 
Raid and its levels
Raid and its levelsRaid and its levels
Raid and its levels
 
Raid
RaidRaid
Raid
 
Understanding RAID Controller
Understanding RAID ControllerUnderstanding RAID Controller
Understanding RAID Controller
 
Data center core elements, Data center virtualization
Data center core elements, Data center virtualizationData center core elements, Data center virtualization
Data center core elements, Data center virtualization
 
Present of Raid and Its Type
Present of Raid and Its TypePresent of Raid and Its Type
Present of Raid and Its Type
 

Kürzlich hochgeladen

Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 

Kürzlich hochgeladen (20)

Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 

RAID: High-Performance, Reliable Secondary Storage

  • 1. High Performance, Reliable Secondary Storage Uğur Tılıkoğlu Gebze Institute of Technology
  • 2. Overview  Introduction  Background  Disk Terminology  Data Paths  Technology Trends  Disk Array Basics  Data Striping and Redundancy  Basic Raid Organizations  Performance and Cost Comparisons  Reliability  Implementation Considerations 2
  • 3. Overview  Advanced Topics  Improving Small Write Performance for RAID Level 5  Declustered Parity  Exploiting On-Line Spare Disks  Data Striping in Disk Arrays  Performance and Reliability Modeling  Opportunities for Future Research  Experience with Disk Arrays  Interaction among New Organizations  Scalability, Massively Parallel Computers and Small Disks  Latency 3
  • 4. Introduction  RAID: Redundant Arrays of Inexpensive / Independent Disks  Improvements in microprocessors and memory systems require larger, higher-performance secondary storage systems  Microprocessors performance increase rate > Disk performance increase rate  Disk arrays: multiple, independent disks  large, high-performance logical disk 4
  • 5. Background  Disk Terminology  Data Paths  Technology Trends 5
  • 9. Disk Array Basics  Data Striping and Redundancy  Basic Raid Organizations  Performance and Cost Comparisons  Reliability  Implementation Considerations 9
  • 10. Data Striping and Redundancy  Data Striping  Distribute data over multiple disks  Service in parallel  More disks  More performance 10
  • 11. Data Striping and Redundancy  More disks  More unreliable  100 disks  1/100 reliability of a single disks  Redundancy  Two categories  Granularity of data interleaving  Method of computing redundant information and distribute accross the disk array 11
  • 12. Data Striping and Redundancy  Data interleaving  Fine grained  Advantages:   access all the disks high transfer rate  Disadvantages   only one I/O request serviced at any time All disks waste time positioning for every request 12
  • 13. Data Striping and Redundancy  Data interleaving  Coarse grained  Advantages:   Multiple small requests serviced simultaneously Large requests can access all the disks 13
  • 14. Data Striping and Redundancy  Redundancy  Two main problems  Computing the redundant information: Parity  Selecting a method for distributing the redundant information accross the disk array 14
  • 15. Basic Raid Organizations  Nonredundant (RAID Level 0)  Lowest cost  Best write performance  No best read performance  Any single disk failure result data loss 15
  • 16. Basic Raid Organizations  Mirrored (RAID Level 1)  Twice number of disks  Data also written to redundant disk  If a disk fails, the other copy is used 16
  • 17. Basic Raid Organizations  Memory Style ECC (RAID Level 2)  Contain parity disks  Parity disk proportional to data disks  Efficiency increases when data disk number increases  Multiple parity disks are needed to identify the failed disk, but only one is needed to recover 17
  • 18. Basic Raid Organizations  Bit-Interleaved Parity (RAID Level 3)  Bit-wise data is used  Disk controller can identify which disk has failed  A single parity disk is used  Read  all disks, Write  all disks + parity disk 18
  • 19. Basic Raid Organizations  Block-Interleaved Parity (RAID Level 4)  Same as Level 3 but blocks (striping units) are used  Read & write < striping unit  one disk  Parity calculation  xor new data with old data  Four I/O: write new data, read old data and old parity, write new parity  Bottleneck at parity disk 19
  • 20. Basic Raid Organizations  Block-Interleaved Distributed-Parity (RAID Level 5)  Solves bottleneck problem at Level 4  Best small read, larger read and large write performance  Small writes are inefficient because of read-modifywrite 20
  • 21. Basic Raid Organizations  P + Q Redundancy (RAID Level 6)  Have stronger codes to solve multiple failures  Operate in much the same manner as Level 5  Small writes are inefficient because of 6 I/O requests due to update both P and Q information 21
  • 22. Performance and Cost Comparisons  Ground Rules and Observations  Reliability, performance and cost  Disk arrays are throughput oriented  I/Os per second per dollar  Configuration  RAID 5 can operate as RAID 1 and RAID 3 by configuring striping unit 22
  • 23. Performance and Cost Comparisons  Comparisons – Small Read & Writes 23
  • 24. Performance and Cost Comparisons  Comparisons – Large Read & Writes 24
  • 25. Performance and Cost Comparisons  Comparisons – RAID 3 & 5 & 6 25
  • 26. Reliability  Basic Reliability  RAID 5    MTTF: mean time to failure, MTTR: mean time to repair N: total number of disks, G: parity group size 100 disks each had MTTF of 200.000 hours, MTTR of 1 hour, partiy group size 16  mean time to failure of the system is about 3000 years !!! 26
  • 27. Reliability  Basic Reliability  RAID 6    MTTF: mean time to failure, MTTR: mean time to repair N: total number of disks, G: parity group size 100 disks each had MTTF of 200.000 hours, MTTR of 1 hour, partiy group size 16  mean time to failure of the system is about 38.000.000 years !!! 27
  • 28. System Crashes and Parity Inconsistency  System crash: power failure, operator error, hardware breakdown, software crash etc.  Causes parity inconsistencies in both bit-interleaved and block-interleaved disk arrays  System crash may occur more frequently than disk failures  To avoid the loss of parity on system crashes, information sufficient to recover parity mus be logged on a non-volatile storage (nvram) before each write operation. 28
  • 29. Uncorrectable Bit Errors  What is bit error? It is unclear  Data is incorrectly written or magnetic media gradually damaged  Some manifacturers developed an approach; monitors the warnings given by disks and notifies an operator when it feels the disk is about to fail. 29
  • 30. Correlated Disk Failures  Environmental and manufacturing factors  Example: earthquake 30
  • 31. Reliability Revisited  Double disk failure  System crash followed by a disk failure  Disk failure followed by an uncorrectable bit error during reconstruction 31
  • 35. Implementation Considerations  Avoiding Stale Data  When a disk fails, failed disk must be marked as invalid. Invalid mark prevents user from reading corrupted data on the failed disk  When an invalid logical sector is reconstructed to a spare disk, the logical sector must be marked as valid. 35
  • 36. Implementation Considerations  Regenerating Parity after a System Crash  Before servicing any write request, the corresponding parity sectors must be marked inconsistent  When bringing a system up from a system crash, all inconsistent parity sectors must be regenerated 36
  • 37. Implementation Considerations  Operating with a Failed Disk  Demand reconstruction: access to a parity stripe with an invalid sector triggers reconstruction of the appropriate data immediately onto a spare disk. A background process scans the entire disk.  Parity sparing: before servicing a write request, the invalid sector is reconstructed and relocated to overwrite its corresponding parity sector 37
  • 39. Advanced Topics  Improving Small Write Performance for RAID Level 5  Declustered Parity  Exploiting On-Line Spare Disks  Data Striping in Disk Arrays  Performance and Reliability Modeling 39
  • 40. Improving Small Write Performance for RAID Level 5  Buffering and Caching  Write buffering (async writes): Collect small writes in a buffer and write as a large data  Read caching: reduce four I/O access to three, old data is read from cache 40
  • 41. Improving Small Write Performance for RAID Level 5  Floating Parity  Shortens the read-modify-write time  Many free blocks  New parity block is writted rotationally nearest unallocated block following the old parity block  Implemented on disk controller 41
  • 42. Improving Small Write Performance for RAID Level 5  Floating Parity  Shortens the read-modify-write time  Many free blocks  New parity block is writted rotationally nearest unallocated block following the old parity block  Implemented on disk controller 42
  • 43. Improving Small Write Performance for RAID Level 5  Parity Logging  Delaying the read of old parity and write of the new parity  Difference is temporarily logged  Logs are grouped together and large contiguous blocks are updated more efficiently 43
  • 44. Declustered Parity  Distributes the increased load uniformly over all disks 44
  • 45. Exploiting On-Line Spare Disks  Distributed Sparing 45
  • 46. Exploiting On-Line Spare Disks  Parity Sparing 46
  • 47. Data Striping in Disk Arrays  Disk positioning time is wasted work  Idle times are same as disk positioning  Data striping or interleaving is, distributing data among multiple disks.  Researchers work on data striping unit size to maximize the throughput 47
  • 48. Data Striping in Disk Arrays  P: average disk positioning time  X: average disk transfer rate  L: concurrency  Z: request size  N: array size in disks 48
  • 49. Performance and Reliability Modeling  Performance  Kim: response time equations  Kim & Tantawi: approximate service time equations  Chen & Towsley  Lee & Katz  Reliability  Markov 49
  • 50. Opportunities for Future Research  Experience with Disk Arrays  Interaction among New Organizations  Scalability, Massively Parallel Computers and Small Disks  Latency 50