Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (8)
Similar to Cassandra on Azure - "Tel-Aviv-Cassandra-Users" meetup 2015
Similar to Cassandra on Azure - "Tel-Aviv-Cassandra-Users" meetup 2015 (20)
Recently uploaded
Recently uploaded (20)
Cassandra on Azure - "Tel-Aviv-Cassandra-Users" meetup 2015
- 1. CASSANDRA ON AZURE? REALLY?! Oded Peer, RSA Security
- 2. 2 © Copyright 2015 EMC Corporation. All rights reserved. THE ELEPHANT IN THE ROOM
- 3. 3 © Copyright 2015 EMC Corporation. All rights reserved. • Only two leaders in Gartner Magic Quadrant for Cloud IaaS (2015) • Only two leaders in Gartner Magic Quadrant for Public Cloud Storage Services (2014) • Computing capacity: AWS has X10 the combined competition, Microsoft has X2 the non-Amazon combined competition • Cloud platform revenues: AWS 6bn revenue, Azure 1.5bn AZURE AND AWS
- 4. 4 © Copyright 2015 EMC Corporation. All rights reserved. AZURE IAAS
- 5. 5 © Copyright 2015 EMC Corporation. All rights reserved. Type CPU RAM (GB) Temp Storage (GB) IOPS $/mo A7 8 56 605 16*500 655 A8 8 56 382 16*500 1091 VM TYPES
- 6. 6 © Copyright 2015 EMC Corporation. All rights reserved. Type CPU RAM (GB) Temp Storage (GB) IOPS $/mo A7 8 56 605 16*500 655 A8 8 56 382 16*500 1091 D13 8 56 400 (SSD) 16*500 521 D14 16 112 800 (SSD) 32*500 1032 D13_v2 8 56 400 (SSD) 16*500 599 VM TYPES
- 7. 7 © Copyright 2015 EMC Corporation. All rights reserved. Type CPU RAM (GB) Temp Storage (GB) IOPS $/mo A7 8 56 605 16*500 655 A8 8 56 382 16*500 1091 D13 8 56 400 (SSD) 16*500 521 D14 16 112 800 (SSD) 32*500 1032 D13_v2 8 56 400 (SSD) 16*500 599 DS13 8 56 112 (SSD) 25,600 256 MB/S D13++ VM TYPES
- 8. 8 © Copyright 2015 EMC Corporation. All rights reserved. Type CPU RAM (GB) Temp Storage (GB) IOPS $/mo A7 8 56 605 16*500 655 A8 8 56 382 16*500 1091 D13 8 56 400 (SSD) 16*500 521 D14 16 112 800 (SSD) 32*500 1032 D13_v2 8 56 400 (SSD) 16*500 599 DS13 8 56 112 (SSD) 25,600 256 MB/S D13++ G4 16 224 3072 (SSD) 32*500 3274 GS4 16 224 448 (SSD) 40,000 1000 MB/S G4++ VM TYPES
- 9. 9 © Copyright 2015 EMC Corporation. All rights reserved. • Ephemeral – Fast, data loss • Standard – 16k IOPS limit (single VM) • Premium (SSD) – Per-disk IOPS limit STORAGE OPTIONS
- 10. 10 © Copyright 2015 EMC Corporation. All rights reserved. Type Size (GB) IOPS Throughput MB/S $/mo P10 128 500 100 18 P20 512 2300 150 66 P30 1024 5000 200 122 PREMIUM STORAGE PRICING
- 11. 11 © Copyright 2015 EMC Corporation. All rights reserved. • Data and logs on ephemeral SSD – 100k IOPS and 1.5 GB/sec on G5 • Data and logs on premium storage – Striped volumes (better than JBOD) – Network latency 1-3 ms – Max 64 TB, 80k IOPS and 1 GB/sec disk throughput – Use standard storage for backup snapshots • Data on premium logs on ephemeral CASSANDRA STORAGE OPTIONS
- 12. 12 © Copyright 2015 EMC Corporation. All rights reserved. • Logically isolated network • Within a VNET bandwidth is a function of VM type – Up to 20 Gbps for G5 • VNET to VNET VPN gateway – Cross region support – Standard (100 Mbps) – High performance (200 Mbps) – Latency according to distance VNET AND MULTI-REGION DEPLOYMENT
- 13. 13 © Copyright 2015 EMC Corporation. All rights reserved. • Availability Zones PLACING REPLICAS
- 14. 14 © Copyright 2015 EMC Corporation. All rights reserved. • Availability Zones • Upgrade Domain, Fault Domain PLACING REPLICAS
- 15. 15 © Copyright 2015 EMC Corporation. All rights reserved. • Availability Zones • Upgrade Domain, Fault Domain – assigned at deployment PLACING REPLICAS
- 16. 16 © Copyright 2015 EMC Corporation. All rights reserved. AUTOMATIC DEPLOYMENT
- 17. 17 © Copyright 2015 EMC Corporation. All rights reserved. • Single node sandbox • DSE cluster (BYOL) • Single region • 4, 12, 36 or 90 nodes • Choose your VM type (D12-D14, G2-G5) • Ephemeral storage • Includes OpsCenter AZURE MARKETPLACE SIMPLIFIED DEPLOYMENT
- 18. 18 © Copyright 2015 EMC Corporation. All rights reserved.
- 19. 19 © Copyright 2015 EMC Corporation. All rights reserved. • Fully managed Cassandra-as-a-Service hosted on Microsoft Azure • Apache Cassandra and DSE • Multiple Cassandra versions • OpsCenter • Uses D-series VMs • Multi region MANAGED CASSANDRA
- 20. 20 © Copyright 2015 EMC Corporation. All rights reserved. REAL WORLD USE CASES
- 21. 21 © Copyright 2015 EMC Corporation. All rights reserved. • DSE on Ubuntu • Single Azure region, 2 vDC • G4 nodes, considering D14_v2 • Data on ephemeral disks • 12 nodes: ingest 20k - 50k events/sec • 30 nodes: ingest 200k events/sec OFFICE 365
- 22. 22 © Copyright 2015 EMC Corporation. All rights reserved. • DS13 VMs (8 CPUs, 56 GB RAM) • 16 p10 premium storage disks • Single Azure region • Combination of Upgrade Domain and Fault Domain for replica placement ADAPTIVE AUTHENTICATION
- 23. 23 © Copyright 2015 EMC Corporation. All rights reserved. • Your mileage may vary • Microsoft eager to help • VPN throughput lower than expected • Commitlog flush IO spikes change in flush interval • LCS STCS to reduce compaction IO TIPS
- 24. 24 © Copyright 2015 EMC Corporation. All rights reserved. THANK YOU
Editor's Notes
- https://centerstagecma.corp.emc.com/CenterStagePro/#CenterStagePro/Brand_+_Creative_Development;v=content/0b0278cd80623579 https://upload.wikimedia.org/wikipedia/commons/thumb/5/5c/AWS_Simple_Icons_AWS_Cloud.svg/2000px-AWS_Simple_Icons_AWS_Cloud.svg.png
- Microsoft has invested heavily in Azure infrastructure and its ability to accommodate hybrid computing tasks—jobs that run partly on the Microsoft Azure public cloud and partly on customers’ data centers. Microsoft alone has more than twice as much cloud computing capacity in use than all the non-Amazon contenders combined. 3,4,5 have less than half of MS in revenue Hapoel Jerusalem and Maccabi Tel Aviv
- Cost – East US 2, Linux A8 and A9 virtual machines feature Intel® Xeon® E5 processors. Adds a 40Gbit/s InfiniBand network with remote direct memory access (RDMA) technology When configured with the necessary network drivers, the A8 and A9 instances can communicate with other A8 and A9 instances over a low-latency, high-throughput network in Azure that is based on remote direct memory access (RDMA) technology. A8 provides 10 Gb ethernet
- D-series VMs are designed to run applications that demand higher compute power and temporary disk performance. D-series VMs provide faster processors, a higher memory-to-core ratio, and a solid-state drive (SSD) for the temporary disk. For details, see the announcement on the Azure blog,New D-Series Virtual Machine Sizes. Dv2-series, a follow-on to the original D-series, features a more powerful CPU. The Dv2-series CPU is about 35% faster than the D-series CPU. It is based on the latest generation 2.4 GHz Intel Xeon® E5-2673 v3 (Haswell) processor, and with the Intel Turbo Boost Technology 2.0, can go up to 3.2 GHz. The Dv2-series has the same memory and disk configurations as the D-series.
- DS-series and GS-series VMs can use Premium Storage, which provides high-performance, low-latency storage for I/O intensive workloads. These VMs use solid-state drives (SSDs) to host a virtual machine’s disks and also provide a local SSD disk cache. Premium Storage is available in certain regions. For details, see Premium Storage: High-performance storage for Azure virtual machine workloads.
- G-series VMs offer the biggest size and best performance and run on hosts that have Intel Xeon E5 V3 family processors. G5 has double the spec values of G4
- https://upload.wikimedia.org/wikipedia/commons/a/aa/Floppy_disk_2009_G1.jpg http://www.datastax.com/2015/04/getting-started-with-azure-premium-storage-and-datastax-enterprise-dse Durability based on LRS (Locally Redundant system) – 3 replicas in the same region.
- https://upload.wikimedia.org/wikipedia/commons/a/aa/Floppy_disk_2009_G1.jpg http://www.datastax.com/2015/04/getting-started-with-azure-premium-storage-and-datastax-enterprise-dse Commitlog on ephemeral storage
- The Azure equivalent of AWS VPC ~20ms (West US, California to South-Central US, Texas) No-crypto option, get additional bandwidth AA has 2 VNETs, one for the application and another for management You can control access by using Network Security Groups (NSGs). VNets can be connected to each other, and even to your on-premises datacenter, by using a site-to-site VPN connection
- Every VM is associated with an upgrade and a fault domain. A fault domain is a rack of computers. the allocation of a fault domain is determined by Azure at VM deployment time. A service owner can not control the allocation of a fault domain. An upgrade domain is a strategy to ensure an application stays up and running, while undergoing an update of the application. Azure will distribute instances evenly into multiple upgrade domains with each upgrade domain as a logical unit of a deployment. When upgrading a deployment, it is then carried out one upgrade domain at a time. The steps are: stopping the instances running in the first upgrade domain, upgrading the application, bringing the instances back online followed by repeating the steps in the next upgrade domain.
- Every VM is associated with an upgrade and a fault domain. A fault domain is a rack of computers. the allocation of a fault domain is determined by Azure at VM deployment time. A service owner can not control the allocation of a fault domain. An upgrade domain is a strategy to ensure an application stays up and running, while undergoing an update of the application. Azure will distribute instances evenly into multiple upgrade domains with each upgrade domain as a logical unit of a deployment. When upgrading a deployment, it is then carried out one upgrade domain at a time. The steps are: stopping the instances running in the first upgrade domain, upgrading the application, bringing the instances back online followed by repeating the steps in the next upgrade domain.
- Every VM is associated with an upgrade and a fault domain. A fault domain is a rack of computers. the allocation of a fault domain is determined by Azure at VM deployment time. A service owner can not control the allocation of a fault domain. An upgrade domain is a strategy to ensure an application stays up and running, while undergoing an update of the application. Azure will distribute instances evenly into multiple upgrade domains with each upgrade domain as a logical unit of a deployment. When upgrading a deployment, it is then carried out one upgrade domain at a time. The steps are: stopping the instances running in the first upgrade domain, upgrading the application, bringing the instances back online followed by repeating the steps in the next upgrade domain.
- https://www.youtube.com/watch?v=rhqSmc9meMw
- https://vimeopro.com/user35188327/cassandra-summit-2015/video/140949184 G1 GC reduced pause times quite a bit, RF=3 in every DC
- Started with A7, tested G3