Slides from webinar demonstrating the disaster recovery and storage management capabilities of Microsoft Azure and StoreSimple.
The webinar was hosted on Friday 14th November 2014 and the recording can be viewed here:
http://1drv.ms/1vovwKF
Azure and StorSimple for Disaster Recovery and Storage Management - SoftwareONE - November 2014
1. Transform your Storage Management with Azure and StorSimple
Jeremy Lloyd
Solutions Architect
Jeremy.Lloyd@softwareone.com
@jeremylloydIT
2. Agenda
• Introducing Azure Storage
• Storage challenges
• StorSimple 8000 series
- Overview
- Cloud integration, tiering, deduplication, compression, and encryption
- StorSimple Manager in Microsoft Azure Management Portal
- StorSimple Virtual Appliance
- Data protection and disaster recovery
• Question & Answers
3. More than 35 trillion stored objects
3.5+ Million requests/sec on average
4.
5.
6. Storage challenges
Rapid data growth - 40% YoY*
Storage cost
and
infrastructure
sprawl
Complex data
protection &
recovery
Resource
constraints
Storage Today = Complex & Expensive
* Source: EMC Digital Universe with Research and Analysis by IDC, 2014
7. Addressing storage challenges with StorSimple
Storage Today Microsoft Azure StorSimple
Primary Storage
Archival Storage
Disk-based Backup
Remote Replication
Tape backup and DR
Manage
data growth
Lower storage
costs
Simplify data
protection and
disaster recovery
Increase business
agility
StorSimple
Reduce storage costs by 40-60%
8. Series 8000 models
Models
Model Number 8100 8600
Total usable capacity 15TB 40TB
Usable SSD Capacity 800GB 2TB
Effective Local Capacity 15-45TB 40-120TB
Max Volume Capacity 64TB 64TB
Max Capacity (including
cloud)
200TB 500TB
Network Interface Cards 4 x 1Gbps and 2 x 10Gbps
Enclosure Form Factor 1 X 2U 2 X 2U
9. Fault tolerance of enterprise class storage
Software
Feature
Protects
Against?
How is works?
Controller
failover
Protection
against the failure
of a controller.
The dual controllers are
active/passive. In the event
the active controller fails
the passive controller
automatically takes over.
NVRAM mirrored across
controllers via NTB.
Storage Spaces
Protection in the
event of one or
more drive
failures.
Disk insert into and
removal from pools.
Nondisruptive
software
upgrades (NDU)
Allows updates to
be made to the
operating system
without taking
downtime.
The software is
downloaded and updated
on each controller. One
controller is up at all times
to continue processing
and serving data.
Redundant
Power Supplies
Dual Controllers
Network
Interfaces
10. Local redundant and geo redundant storage
Primary Location
Secondary Location
11. Automatic tiering
Low
Threshold
High
Threshold
Time
Tier is empty in the beginning
New data is written at start of free space
Data exceeds high watermark
Old data spills to next tier
Tier returns to low watermark level
DATA
DATA
DATA
DATA
12. 12
SSD
A B C Linear Tier A B D E
SSD
Deduplicated
SAS
Deduplicated
Compressed
Cloud
Deduplicated
Compressed
Encrypted
E
C D E
D E
13.
14. Consolidated management of appliances
StorSimple_PhysicalAppliance1
Microsoft Confidential
StorSimple_PhysicalAppliance2
StorSimple_VirtualAppliance1
Azure StorSimple Manager
All device will need
the Registration Key
and Service Data
Encryption Key to be
added to the Service
(Service Data Encryption Key is generated, on device console, when first device is registered)
21. Local and Cloud snapshots
SSD
SAS
Local
Snapshot
Cloud
Snapshot
Example Group Of Data Protection Policies
Backup Type Snapshot
Type
Frequency Retention
Period
Short-term
backup
Local &
Cloud
Daily ~1
week
Medium-term
backup
Cloud Weekly ~4
weeks
Long-term
Backup
Cloud Monthly ~52
weeks
22. Local snapshot
Instantaneous
and provide
short term
data recovery
Volume Data
Volume Metadata
Snapshot 1
Snapshot 2
Snapshot 3
Reference to
data chunk
23. Cloud snapshot
Metadata is
preserved in
its entirety
Only changed
blocks are
copied to
Cloud
SSD
SAS
24. StorSimple Snapshot Manager
MMC UI snap-in
that lives on
application host
and
communicates
with appliance(s)
Orchestrates
application
consistent
snapshots
Manages backup
policies
Restores
volumes and
mounts them as
NTFS
folders/drives
25. Disaster recovery
• Thin restores provides fast recovery
of data, downloading a small
subset of data necessary data bring
applications online. Not dependent
on volume size.
• Recover data anyway: a secondary
datacenter or a StorSimple Virtual
Appliance in Azure
• A volume container is failed over to
the destination appliance. Time to
recover is the same for physical
and virtual appliance
26. DR to secondary datacenter or Virtual Appliance
Datacenter 1
Filer Server
Physical Appliance
(Appliance1)
File Server
Azure
Storage
Virtual Appliance
(Appliance3)
Filer Server
Physical Appliance
(Appliance2)
Datacenter 2
StorSimpleManager1
Recover data to Appliance2 (physical) or
Appliance3 (virtual) by rehydrating data
from a Cloud snapshot stored in Azure BLOB
storage.
28. Any general questions on the content can be posted to
Jeremy.Lloyd@softwareone.com
David.Bloxham@softwareone.com
29. THANK
YOU
Jeremy Lloyd
Solutions Architect
Jeremy.Lloyd@softwareone.com
@jeremylloydIT
Info.uk@softwareone.com
+44 (0)203 5300 270
@SoftwareONEuk
Hinweis der Redaktion
BLOBS:
Simple REST interface (Put, Get, Delete)
Data sharing – share documents, pictures, video, music, etc.
Big Data – store raw data/logs and compute/map reduce over data
Backups – data and device backups
DISKS:
Persistent disks for VMs in Azure
Disks are VHDs stored in Azure Page Blobs
Page blobs are optimized for random I/O
VM see the VHD/Blob as a disk
Reads translated to GETs, writes to PUTs
Blob protected by write-lease
Reads from the blob (and snapshots) still allowed
TABLES:
Tables – Massively scalable NoSQL cloud store
Key/Attribute(s) store at scale
Store user, device or any type of metadata for your service
Auto load balances partitions to meet traffic needs
Supports the OData protocol (AtomPub or JSON)
QUEUES:
Queues – Reliable messaging system
Reliable, low latency, high throughput messaging system
Decouple components/roles
Web role to worker role communication
Allows roles to scale independently
Implement scheduling of asynchronous tasks
Building process/work flows
FILES:
Move on-premises applications to cloud
VMs can net use an SMB share using standard file APIs and semantics
Supports the SMB 2.1 protocol
VM and storage account within same region
Supports REST and SMB protocol access to same file share
Build Internet Scale applications, some example of applications that are running on Azure are Bing, O365, Xbox Live, Aston Martin’s website. For Aston Martin the platform gives them the ability to very quickly and easily scale up their website ahead of peak demands such as the launch of a new car.
Tools such a Azure Storage explorers and SDKs are available to help you better interact with the storage. There is a healthy community out there building and refining tools that you can freely use. Check out NuGet or
Works in conjunction with the new D series virtual machines available from the gallery.
Surveys show that on average your data growth is around 40% year on year. That increase in data and storage infrastructure behind supporting that data growth is time consuming. Add in the cost of managing this and it makes todays storage complex and expensive.
On the left is an example of today’s typical enterprise customers with primary storage (netapp etc), archival storage, disk-based backup, remote replication and tap backup and DR.
Storsimple device dramatically simplifies this but reducing the requirement for separate physical storage infrastructures by means of storage tiering. Which enable you to manage your data growth, simplify disaster recovery while lowering your storage costs.
Doesn’t use RAID as instead it uses MetaData map which I will explain later, which it how it can make effective use of the physical disks
Device performs deduplication and compression hence the larger effective local capacity.
Does now support 10GBPS nics in the 8000 series.
Built using Seagate hardware
Built to enable you to hot swap any components. Disks in separate raid groups. Dual Controllers in active/passive state with automatic failover
Can do nondisruptive software upgrade but upgrading just one controller at a time and then failing over before doing the next.
For the cloud tier of the solution you have options for locally redundant storage and geo redundant storage.
Locally redundant means all your data will reside in a single Azure datacentre, Dublin or Amsterdam etc. However there will be x3 copies of the data held within that datacentre.
Geo-redundant will provide an asynchronous replication to another Azure datacentre. Where by you get another x3 copies of your data. For standard blob storage this service ususally costs extra however the StorSimple SKU provides geo-redundant storage without additional costs
So lets look at how the tiering aspect works. This method applies to both SSD and SAS arrays.
You start with the empty tier
Then users start writing data to their volumes filling up the tier.
Once the data exceeds the high threshold the oldest data is moved automatically to the next tier, be that SAS or Azure.
It will move data until you are back at the low threshold again. The reason it does this is to ensure any large data migrations have sufficient space in the tier.
Ok so we’ve looked at the automatic tiering so now we know how data moves but how does that work and when does my data go into the cloud?
Storsimple works at the block layer, treating everything in 64kb blocks. The letter in this diagram represent a block. The Storsimple appliance is made up of two physical tiers, the first tier is SSD storage. This is what we call our working tier, all active read/writes take places here. Inside the same physical tier is our SSD deduplicated tier, here all the blocks that can be deduplicated are.
Once this tier reaches the thresholds we look at previously that blocks will move to the SAS layer whereupon compression will take place. So the SAS layer contains deduplicated and compressed blocks.
Once the SAS layer reaches it’s threshold the data in encrypted before being moved to your Azure Storage account. The encryption is high standard AES-256 algorithm.
You maintain the security keys and these are stored securely on the appliance.
In azure the data would simply look like the fingerprint and nothing tangible.
So where do I do my day to day management of the appliance?
The StorSimple 8000 series can be managed from your Azure portal that you may have for other cloud services. StorSimple manager enables you to manage multiple physical storsimple devices and multiple storsimple VA’s from one portal/one login. It’s a very simple process to register a device into your Azure subscription.
Once the appliance is registered you can create your volumes but first you need to create a volume container. First give it a name, then link it to the storage account that you would have previously created when registering the appliance to Azure. Then use your encryption key for the data encryption.
Now we have at the top of the topology our Storage account, then here in this example we have two volume containers, one for file shares and one for sharepoint. These are isolated containers. Under the containers we can now create our volume to present the available storage to the end users.
To backup the device we can perform local snapshots which are pointers to the data and kept locally on the appliance or we can perform cloud snapshots of volumes inside a volume container. This puts the entire volume into azure giving you additional redundancy in a disaster scenario
The VA is a software version of the physical storsimple appliance. You build a VM in Azure, then connect to the VA as an iSCSI target. You will need to place your VMs and the VA in a vnetwork in azure which gives you the ability to place virtual machines on a private address range that you’ve specified when you create an Azure virtual network.
You use protection policies to define your frequency and type of snapshot. There is flexibility in the local & cloud snapshots, you can create local snapshots for short-term backups. Or just use cloud for this, or use both for additional resiliency.
The table shows example protection policies.
Local snapshots work by looking at the blocks on the volume data. The metadata references the blocks, example A in metadata references block A and so on.
We treat the metadata as a data chunk.
If you create a snapshot with no changes, there is nothing new to snapshot.
If block A changes and the reference to it in the metadata is updated then the next snapshot will contain the updated pointer to the original A block as shown.
The first cloud snapshot will take all of the data from your SSD and SAS and put this into Azure.
The next snapshot will only put the new blocks and the metadata into Azure, it’s an incremental backup.
From a cloud snapshot would could restore your entire data sets to a new storsimple physical appliance or restore into Azure.
Use the snapshot manager to create application consistent backups. It works with applications that integrate with VSS, such as SQL or use it for Hyper-V snapshots and it based upon an MMC snapin.
Once you have your cloud snapshots be they scheduled weekly or monthly or done manually and have a secondary datacentre with a StorSimple appliance or use the StorSimple Virtual Appliance in Azure, Testing DR simply means you restore volume containers to their the physical DR storsimple appliance or to the virtual appliance. It’s really very easy. This is done by using the same registration key from your primary appliance which enables the device to access to your encryption key, the storage account’s password, and the metadata map in Azure.
So with that in mind if we lose our primary datacentre we can failover to another physical appliance as shown or use an Azure Virtual Appliance. You can create multiple Azure VA’s to separate out workloads or departments data when recovering.
Recovery is quicker than traditional DR solutions because when mounting cloud snapshots it only pulls down the metadata information.
So take the scenario that you’ve performed a restore and have 100TB in your cloud snapshot. The metadata map has been restored to the secondary physical appliance, so your users now see their data/shares in their network drives or sharepoint. When they access those documents then it will actually pull down the document and this would be to the SSD tier. We do not have to restore the entire 100TB dataset.
