The Rise of Open Storage
•Download as PPTX, PDF•
1 like•494 views
My presentation to the Nexenta European User Conference, May 20th 2011, Amsterdam
Recommended
More Related Content
What's hot
What's hot (19)
Similar to The Rise of Open Storage
Similar to The Rise of Open Storage (20)
Recently uploaded
Recently uploaded (20)
The Rise of Open Storage
- 2. The Rise of Open Storage …and the benefits of storage virtualisation 2
- 3. Agenda Early Storage Storage Arrays - the last 20 years The move to standardisedhardware It’s all about the software Parallels with Server Virtualisation Storage virtualisation and hardware independence Future speculation 3
- 4. Early Storage Pioneered by IBM IBM 350 Disk Storage Unit Released in 1956 1.52m x 1.72m x 0.74m 50 magnetic disks 5MB capacity 600ms access time IBM 350 Disk Storage Unit Image courtesy of IBM Archives 4
- 5. Early Storage “Winchester” Drives Named after 30-30 rifle Released in 1973 Smaller & lighter 70MB capacity 25ms access time IBM 3340 DASF (courtesy of IBM archives) 5
- 6. Early Storage Large, monolithic “refrigerator” units No hardware recovery Slow & expensive Cumbersome CKD format Each LUN/volume still a physical disk IBM 3380 Model CJ2 (courtesy of IBM archives) 6
- 7. Early Storage Seagate ST-506 Released in 1980 5MB Capacity 5.25” form factor No onboard controller Adopted for IBM PC Over 24 years, for the same capacity, drive sizes reduced by 800 times 7
- 8. Early Storage Disk Drives Today 3TB+ Capacity Integrated controllers Small Form Factor (2.5”) 6Gb/s interfaces Very high reliability Low cost per GB Drives are now Commodity Components 8
- 9. Early Storage 50 years of development… …from cargo to pocket! 9
- 10. Storage – The Last 20 Years EMC set the standard Symmetrix released 1990 Integrated Cache Disk Array Dedicated hardware components RAID-1 Replication (SRDF in 1994) Support for non-mainframe (1995) 10
- 11. Storage – The Last 20 Years Integrated storage arrays separated control and management from the host Custom hardware design More functionality pushed to the array Cache I/O Replication Snapshots Logical LUNs 11
- 12. Storage – The Last 20 Years Rapid development in features Many vendors – IBM, Hitachi, HP New product categories Midrange/modular (e.g. CLARiiON) NFS Appliances – Filers De-duplication devices 12
- 13. Storage – The Last 20 Years Storage has Centralised Storage Area Networks Started with ESCON & SCSI Fibre Channel (1997 onwards) NAS (early 1990s) iSCSI (1999 onwards) FCoE 13
- 14. The Move to Standarisation Hardware components have become more reliable More features moved into software RAID Replication Some bespoke features remaining in silicon 3PAR dedicated ASIC Hitachi VSP virtual processors 14
- 15. The Move to Standardisation Reduced Cost Cheaper components No custom design Reusable by generation Higher Margins 15
- 16. The Move to Standardisation New breed of products EMC VMAX Hitachi VSP HP P9500 New Companies Compellent 3PAR Lefthand Equallogic Isilon IBRIX It’s no surprise that these companies have been acquired for their software assets 16
- 17. It’s all About Software Storage arrays look like servers Common components Generic physical layer Independence from hardware allows: Reduced cost Design hardware to meet requirements Quicker to market with new hardware More scalability Quicker/Easier upgrade path Deliver new features without hardware upgrade 17
- 18. It’s all About Software Many vendors have produced VSAs Netapp – simulator (not strictly a VSA), Lefthand/HP, Gluster, Falconstor, Openfiler, OPEN-E, StorMagic, NexentaStor, Sun Amber Road Most of these run exactly the same codebase as the physical storage device As long as reliability & availability are met, then the hardware is no longer significant 18
- 19. Parallels with Server Virtualisation Server virtualisation was successful due to power of Intel processors & Linux Enabled x86 work to be used for Windows and Open Systems Windows platform almost needs 1 server per application, forcing consolidation Wave 1 server virtualisation reduced costs, improved hardware utilisation – the consolidation phase. Wave 2 implemented mobility features; vMotion, Storage vMotion, HA, DRS. 19
- 20. Parallels with Server Virtualisation Virtualisation enables disparate operating systems to be supported on the same hardware Workload can be balanced to meet demand Hardware can be added/removed non-disruptively – transparent upgrade Server virtualisation has enabled high scalability 20
- 21. Storage Virtualisation & Hardware Independence VSAs show closely coupled hardware/software is no longer required Software can be developed and released independently Feature release not dependent on hardware 21
- 22. Storage Virtualisation & Hardware Independence Hardware can be designed to meet performance, availability & throughput, leveraging server hardware development Branches with smaller hardware Core data centres with bigger arrays Both using same features/functionality 22
- 23. Future Speculation LUN virtualisation rather than array virtualisation is the key to future LUNs must be individually addressable Ability to move a LUN between physical infrastructures LUN owned/managed by an array Transparent migration, failover Increased availability Delivers data mobility – an absolute requirement as data quantities increase (especially PB+ arrays) 23
- 24. Future Speculation New addressing schema necessary Remove restrictions of Fibre Channel Address LUN independently of physical location Allow LUN to move around infrastructure Allow LUN to be addressed through multiple locations More granular sub-object access Better load balancing Better mobility 24
- 25. Future Speculation VMFS are LUNs – which are binary objects VMFS divides into VMDKs for independent access Virtual machine becomes the object to move around the infrastructure Sub-LUN access and locking enables read & write everywhere approach Storage and Virtualisation will be inextricably linked to each other 25
- 26. Questions? 26
Editor's Notes
- The first disk drive was invented by IBM in 1956. It had a capacity of only 5MB and as can be seen from the later picture, was huge, consisting of 50 magnetic disks and being housed in something the size of a large refrigerator.
- The technology quickly moved on, with smaller size, larger capacity drives. Winchester drives were introduced, so called because they were intended to have 2 30MB spindles.
- However storage still suffered from being large, expensive, had no built in recovery technology and used CKD. Each “LUN” or volume was still a physical disk.
- Revolution came with the arrival of the first 5.25 form factor drive from Seagate, out of Shugart Associates. This had the same capacity of the drive from 24 years earlier, but was 800 times smaller. Although this drive had no onboard controller, it was the shape of drives to come. Very quickly drive controls were integrated onto the drive itself (hence the term IDE, integrated drive elecronics).
- Today we have drives that are stand-alone commodity devices. They have reduced to 2.5”, become highly reliable and can transfer data in/out incredibly quickly. We’re likely to see more hybrid flash/HDD models too and SSDs are in use where performance wins over cost.
- 50 years of development has brought us from a cargo plane to an envelope – from 5MB to 32GB.
- What about the storage array? Until drives became commodity, we couldn’t deliver arrays. EMC integrated commodity drives with cache and bespoke components to create an array that worked stand-alone from the processor.
- The I/O subsystem that the mainframe previously needed to control disks was pushed down into the array. This meant new features could be created – replication, snapshots and RAID – all without host control.
- Quickly we saw rapid development in features; new participants in the market arrived. New product categories developed – Midrange, filers, deduplication devices. These devices were charged at a premium – growing these companies substantially.
- Storage became centralised with the advent of Fibre Channel and Storage Area Networks. Today we have FCoE, FC, iSCSI, NFS, CIFS as connectivity protocols – all of which are still based on SCSI and the first SAN - ESCON
- More recently vendors have been able to take advantage of increased processor power, reliable components and faster backplanes to move away from dedicated ASIC and component development. They’ve moved from manufacturers to assemblers and software developers. Only two companies haven’t shaken off this; 3PAR and Hitachi still use dedicated ASICs, but the market is too expensive for any new competitors.
- This new breed of products has seen competition from companies using commodity hardware – Compellent being the latest to fall to Dell, all of which have used commodity hardware and pushed the intelligence to hardware.
- So the hardware has moved to become standardised. Storage arrays now look like servers – common components – generic physical hardware. This commoditisation means the developments in the array aren’t tied to hardware developments – both can develop independently.
- This can easily be witnessed by the fact that there are so many VSAs – virtual appliances available. Some are fully functional, some development only. But either way, they show that the features are all now software based.
- We can see parallels here with virtualisation. Server virtualisation existed 38 years ago with the release of VM in 1972 but had a new least of life with X86 virtualisation of Windows & Linux. Wave 1 was initial adoption – the consolidation phase. Phase 2 moves the technology mainstream by improving scalability with new features.
- Server virtualisation removed the tie to the hardware. Workload exists independent of the hardware to the extend that it can be used more efficiently, with a more highly available delivery.
- VSAs for storage show hardware and software don’t need to be tightly coupled. Software can be developed independently and will be the future of storage deployments.
- So hardware can be designed to meet requirements – small for branches, clustered for high availability, Both use the same software layer – so interoperability exists.
- What about the future – where are we headed? Data needs independence from the array. It needs to be addressed as an independent object – similar to how DNS and NFS/CIFS addressing works.
- We’re too tied to WWN/IP address. We need to be able to federate access to LUNs or objects as they move around the infrastructure.
- In the future the boundaries between storage and virtualisation will continue to be blurred.