SlideShare a Scribd company logo
1 of 25
Cassandra in the Cloud
How to Build a Multi-DC
Cassandra Cluster in AWS
w/ OpsCenter LCM
Manual
- Error prone.
- Days and weeks.
- Bad auditability.
- Always waiting on someone .
- Costs time and money.
- Open security issues.
- No one legitimate should be
doing this manually. Even in
the government, you should
be automating.
Automated
- Repeatable
- Minutes and hours
- Changes stored in source
Control
- Can do CICD
- Can be super secured.
- If you are dealing with
systems that make money,
you should use it.
Setting up 3 nodes. Basic bootcamp
exercise. How long?
Add 4 more nodes. How long then?
Add 40 more nodes. How long then?
What we want:
- Setup 3 Nodes
- Add 4 Nodes
Infrastructure: AWS
Infrastructure: Terraform
Install: Ansible
Install: OpsCenter LCM
Configure: OpsCenter LCM
Maintain: OpsCenter LCM
Why not just use
one thing for
everything?
We design and build realtime
business platforms…
By mapping and connecting real-time customer experiences
with information systems on fast data & analytics platforms.
Resources
Cassandra
● cassandra.link
● https://anant.github.io/awesome-
cassandra
Datastax
● www.datastax.com
● https://docs.datastax.com/en/opscent
er/6.7/opsc/LCM/opscLCM.html
Ansible
● https://github.com/rackerlabs/ansible-
dse
Terraform
● https://hackernoon.com/introduction-
to-aws-with-terraform-7a8daf261dc0
Creating Nodes with Terraform
Benefits of AWS
● Ease of deployment
● Scalability (Availability Zones)
● Security (VPC)
● Automation (CloudFormation)
Using Terraform to put Cassandra on an EC2
● Terraform - Infrastructure as Code Software
Tool
● Can define and provision a datacenter
infrastructure
● Used today to create an EC2 instance on AWS
Terraform in Action
Automation with Ansible
What is a playbook?
● An organized unit of scripts in a YAML file
● Defines work for a server configuration
Benefits of Ansible with DSE
● An hour long process can be done in a minute
● Adding users is simplified
● Can be applied to multiple datacenters, clusters, and nodes
Playbook: Validation / Common
● Confirm credentials (SSH, pem, etc)
● Modify System Kernel Attributes
○ Network
○ Memory
● Set CPU Governor
● Set EC2 User limits
Playbook: OpsCenter
● Add the Datastax repo
● Add DSE Academy credentials if needed.
● Install the OpsCenterD service.
● Start the OpsCenterD service.
Installing Clusters with OpsCenter
Lifecycle Manager
Benefits of LCM
● Simple yet robust User Interface
● Automatic detection of package manager
○ Easy deployment in a variety of
environments and conditions
● Health, sanity checks automated during install
Installation Process
● Done in OpsCenter
● Config profiles set for each datacenter
○ Profile
○ SSH Credentials
○ Workload
● Nodes added to each datacenter
○ Rack
○ Credentials
○ 1st Node is a Seed Node
Ansible Scripts Used
Playbooks used today
● Cluster Validation
● Cluster Common
● Cluster OpsCenter
Resources
Cassandra
● cassandra.link
● https://anant.github.io/awesome-
cassandra
Datastax
● www.datastax.com
● https://docs.datastax.com/en/opscent
er/6.7/opsc/LCM/opscLCM.html
Ansible
● https://github.com/rackerlabs/ansible-
dse
Terraform
● https://hackernoon.com/introduction-
to-aws-with-terraform-7a8daf261dc0
Any Questions?
www.anant.us | solutions@anant.us | (855) 262-6826
3 Washington Circle, NW | Suite 301 | Washington, DC 20037
Data & Analytics
Cassandra, DataStax, Kafka, Spark
Customer Experience
Sitecore
Information Systems
Salesforce, Quickbooks, and more

More Related Content

What's hot

Tài liệu mô tả dự án echodientu
Tài liệu mô tả dự án echodientuTài liệu mô tả dự án echodientu
Tài liệu mô tả dự án echodientuĐổi Thay
 
Cài đặt và cấu hình dns server trên windows server 2012
Cài đặt và cấu hình dns server trên windows server 2012Cài đặt và cấu hình dns server trên windows server 2012
Cài đặt và cấu hình dns server trên windows server 2012laonap166
 
Bài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPT
Bài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPTBài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPT
Bài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPTMasterCode.vn
 
[Java] Khái niệm về RMI trong Java và cách sử dụng RMI
[Java] Khái niệm về RMI trong Java và cách sử dụng RMI[Java] Khái niệm về RMI trong Java và cách sử dụng RMI
[Java] Khái niệm về RMI trong Java và cách sử dụng RMITí Bụng Bự
 
Spring framework
Spring frameworkSpring framework
Spring frameworkAn Nguyen
 
Bài 2 : Các đối tượng trong CSDL - SQL server
Bài 2 : Các đối tượng trong CSDL - SQL serverBài 2 : Các đối tượng trong CSDL - SQL server
Bài 2 : Các đối tượng trong CSDL - SQL serverMasterCode.vn
 
Tài liệu data warehouse vietsub
Tài liệu data warehouse  vietsubTài liệu data warehouse  vietsub
Tài liệu data warehouse vietsubhoangdat1361
 
Đồ án kiểm thử phần mềm
Đồ án kiểm thử phần mềmĐồ án kiểm thử phần mềm
Đồ án kiểm thử phần mềmNguyễn Anh
 
Quy tắc thiết kế giao diện và viết code C#
Quy tắc thiết kế giao diện và viết code C#Quy tắc thiết kế giao diện và viết code C#
Quy tắc thiết kế giao diện và viết code C#An Nguyen
 
Trigger, Cursor, Function in SQL Server
Trigger, Cursor, Function in SQL ServerTrigger, Cursor, Function in SQL Server
Trigger, Cursor, Function in SQL ServerNguyễn Phúc
 
Báo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên Android
Báo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên AndroidBáo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên Android
Báo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên AndroidDlinh Truong
 
Kịch bản demo phát hiện xâm nhập sử dụng snort ids
Kịch bản demo phát hiện xâm nhập sử dụng snort idsKịch bản demo phát hiện xâm nhập sử dụng snort ids
Kịch bản demo phát hiện xâm nhập sử dụng snort idsLinh Hoang
 
Tong quan ve phan cum data mining
Tong quan ve phan cum   data miningTong quan ve phan cum   data mining
Tong quan ve phan cum data miningHoa Chu
 

What's hot (20)

firewall
firewallfirewall
firewall
 
Tài liệu mô tả dự án echodientu
Tài liệu mô tả dự án echodientuTài liệu mô tả dự án echodientu
Tài liệu mô tả dự án echodientu
 
Cài đặt và cấu hình dns server trên windows server 2012
Cài đặt và cấu hình dns server trên windows server 2012Cài đặt và cấu hình dns server trên windows server 2012
Cài đặt và cấu hình dns server trên windows server 2012
 
Bài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPT
Bài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPTBài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPT
Bài 6: Bảo mật trong SQL SERVER 2008 - Giáo trình FPT
 
Cách trích dẫn và liệt kế nguồn tài liệu tham khảo
Cách trích dẫn và liệt kế nguồn tài liệu tham khảoCách trích dẫn và liệt kế nguồn tài liệu tham khảo
Cách trích dẫn và liệt kế nguồn tài liệu tham khảo
 
Đề tài: Kiểm thử phần mềm trên thiết bị di động, HAY, 9đ
Đề tài: Kiểm thử phần mềm trên thiết bị di động, HAY, 9đĐề tài: Kiểm thử phần mềm trên thiết bị di động, HAY, 9đ
Đề tài: Kiểm thử phần mềm trên thiết bị di động, HAY, 9đ
 
[Java] Khái niệm về RMI trong Java và cách sử dụng RMI
[Java] Khái niệm về RMI trong Java và cách sử dụng RMI[Java] Khái niệm về RMI trong Java và cách sử dụng RMI
[Java] Khái niệm về RMI trong Java và cách sử dụng RMI
 
Spring framework
Spring frameworkSpring framework
Spring framework
 
Bài 2 : Các đối tượng trong CSDL - SQL server
Bài 2 : Các đối tượng trong CSDL - SQL serverBài 2 : Các đối tượng trong CSDL - SQL server
Bài 2 : Các đối tượng trong CSDL - SQL server
 
Tài liệu data warehouse vietsub
Tài liệu data warehouse  vietsubTài liệu data warehouse  vietsub
Tài liệu data warehouse vietsub
 
Đồ án kiểm thử phần mềm
Đồ án kiểm thử phần mềmĐồ án kiểm thử phần mềm
Đồ án kiểm thử phần mềm
 
Quy tắc thiết kế giao diện và viết code C#
Quy tắc thiết kế giao diện và viết code C#Quy tắc thiết kế giao diện và viết code C#
Quy tắc thiết kế giao diện và viết code C#
 
200 đề tài luận văn thạc sĩ an ninh mạng. HAY
200 đề tài luận văn thạc sĩ an ninh mạng. HAY200 đề tài luận văn thạc sĩ an ninh mạng. HAY
200 đề tài luận văn thạc sĩ an ninh mạng. HAY
 
Trigger, Cursor, Function in SQL Server
Trigger, Cursor, Function in SQL ServerTrigger, Cursor, Function in SQL Server
Trigger, Cursor, Function in SQL Server
 
Báo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên Android
Báo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên AndroidBáo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên Android
Báo Cáo Đề Tài Xây Dựng Phần Mềm Quản Lí Khách Hàng Trên Android
 
Kịch bản demo phát hiện xâm nhập sử dụng snort ids
Kịch bản demo phát hiện xâm nhập sử dụng snort idsKịch bản demo phát hiện xâm nhập sử dụng snort ids
Kịch bản demo phát hiện xâm nhập sử dụng snort ids
 
Luận văn: Công nghệ ảo hóa Docker và ứng dụng tại ĐH Hải Phòng
Luận văn: Công nghệ ảo hóa Docker và ứng dụng tại ĐH Hải PhòngLuận văn: Công nghệ ảo hóa Docker và ứng dụng tại ĐH Hải Phòng
Luận văn: Công nghệ ảo hóa Docker và ứng dụng tại ĐH Hải Phòng
 
Báo cáo snort
Báo cáo snortBáo cáo snort
Báo cáo snort
 
BÀI MẪU Tiểu luận văn hóa doanh nghiệp tại Vinamilk, HAY
BÀI MẪU Tiểu luận văn hóa doanh nghiệp tại Vinamilk, HAYBÀI MẪU Tiểu luận văn hóa doanh nghiệp tại Vinamilk, HAY
BÀI MẪU Tiểu luận văn hóa doanh nghiệp tại Vinamilk, HAY
 
Tong quan ve phan cum data mining
Tong quan ve phan cum   data miningTong quan ve phan cum   data mining
Tong quan ve phan cum data mining
 

Similar to How to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCM

NetflixOSS Meetup season 3 episode 1
NetflixOSS Meetup season 3 episode 1NetflixOSS Meetup season 3 episode 1
NetflixOSS Meetup season 3 episode 1Ruslan Meshenberg
 
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016aspyker
 
Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016Sharma Podila
 
Building a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for AnalystsBuilding a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for AnalystsAvere Systems
 
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...DataStax Academy
 
Leveraging Cassandra for real-time multi-datacenter public cloud analytics
Leveraging Cassandra for real-time multi-datacenter public cloud analyticsLeveraging Cassandra for real-time multi-datacenter public cloud analytics
Leveraging Cassandra for real-time multi-datacenter public cloud analyticsJulien Anguenot
 
Truemotion Adventures in Containerization
Truemotion Adventures in ContainerizationTruemotion Adventures in Containerization
Truemotion Adventures in ContainerizationRyan Hunter
 
Docker on Amazon ECS
Docker on Amazon ECSDocker on Amazon ECS
Docker on Amazon ECSDeepak Kumar
 
Cloud Architecture best practices
Cloud Architecture best practicesCloud Architecture best practices
Cloud Architecture best practicesOmid Vahdaty
 
Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...
Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...
Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...Amazon Web Services
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...NETWAYS
 
Infrastructure Considerations : Design : "webops"
Infrastructure Considerations : Design : "webops"Infrastructure Considerations : Design : "webops"
Infrastructure Considerations : Design : "webops"Piyush Kumar
 
Private Cloud with Open Stack, Docker
Private Cloud with Open Stack, DockerPrivate Cloud with Open Stack, Docker
Private Cloud with Open Stack, DockerDavinder Kohli
 
Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...
Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...
Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...Xiaoman DONG
 
Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28Sadique Puthen
 
Bootstrapping - Session 1 - Your First Week with Amazon EC2
Bootstrapping - Session 1 - Your First Week with Amazon EC2Bootstrapping - Session 1 - Your First Week with Amazon EC2
Bootstrapping - Session 1 - Your First Week with Amazon EC2Amazon Web Services
 
Netflix and Containers: Not A Stranger Thing
Netflix and Containers:  Not A Stranger ThingNetflix and Containers:  Not A Stranger Thing
Netflix and Containers: Not A Stranger Thingaspyker
 
Netflix and Containers: Not Stranger Things
Netflix and Containers: Not Stranger ThingsNetflix and Containers: Not Stranger Things
Netflix and Containers: Not Stranger ThingsAll Things Open
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAmazon Web Services
 
Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...
Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...
Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...Kaseya
 

Similar to How to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCM (20)

NetflixOSS Meetup season 3 episode 1
NetflixOSS Meetup season 3 episode 1NetflixOSS Meetup season 3 episode 1
NetflixOSS Meetup season 3 episode 1
 
Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016Netflix Container Scheduling and Execution - QCon New York 2016
Netflix Container Scheduling and Execution - QCon New York 2016
 
Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016Scheduling a fuller house - Talk at QCon NY 2016
Scheduling a fuller house - Talk at QCon NY 2016
 
Building a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for AnalystsBuilding a Just-in-Time Application Stack for Analysts
Building a Just-in-Time Application Stack for Analysts
 
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
 
Leveraging Cassandra for real-time multi-datacenter public cloud analytics
Leveraging Cassandra for real-time multi-datacenter public cloud analyticsLeveraging Cassandra for real-time multi-datacenter public cloud analytics
Leveraging Cassandra for real-time multi-datacenter public cloud analytics
 
Truemotion Adventures in Containerization
Truemotion Adventures in ContainerizationTruemotion Adventures in Containerization
Truemotion Adventures in Containerization
 
Docker on Amazon ECS
Docker on Amazon ECSDocker on Amazon ECS
Docker on Amazon ECS
 
Cloud Architecture best practices
Cloud Architecture best practicesCloud Architecture best practices
Cloud Architecture best practices
 
Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...
Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...
Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
 
Infrastructure Considerations : Design : "webops"
Infrastructure Considerations : Design : "webops"Infrastructure Considerations : Design : "webops"
Infrastructure Considerations : Design : "webops"
 
Private Cloud with Open Stack, Docker
Private Cloud with Open Stack, DockerPrivate Cloud with Open Stack, Docker
Private Cloud with Open Stack, Docker
 
Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...
Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...
Kubernetes Clusters At Scale: Managing Hundreds Apache Pinot Kubernetes Clust...
 
Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28Introduction openstack-meetup-nov-28
Introduction openstack-meetup-nov-28
 
Bootstrapping - Session 1 - Your First Week with Amazon EC2
Bootstrapping - Session 1 - Your First Week with Amazon EC2Bootstrapping - Session 1 - Your First Week with Amazon EC2
Bootstrapping - Session 1 - Your First Week with Amazon EC2
 
Netflix and Containers: Not A Stranger Thing
Netflix and Containers:  Not A Stranger ThingNetflix and Containers:  Not A Stranger Thing
Netflix and Containers: Not A Stranger Thing
 
Netflix and Containers: Not Stranger Things
Netflix and Containers: Not Stranger ThingsNetflix and Containers: Not Stranger Things
Netflix and Containers: Not Stranger Things
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 
Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...
Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...
Kaseya Connect 2013: Optimizing Your K Server - Best Practices in Kaseya Infr...
 

More from Anant Corporation

QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137Anant Corporation
 
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdfKono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdfAnant Corporation
 
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache PinotData Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache PinotAnant Corporation
 
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...Anant Corporation
 
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPTAutomate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPTAnant Corporation
 
Episode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapEpisode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapAnant Corporation
 
Machine Learning Orchestration with Airflow
Machine Learning Orchestration with AirflowMachine Learning Orchestration with Airflow
Machine Learning Orchestration with AirflowAnant Corporation
 
Cassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward TalksCassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward TalksAnant Corporation
 
Data Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with ArcionData Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with ArcionAnant Corporation
 
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...Anant Corporation
 
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & FutureCassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & FutureAnant Corporation
 
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...Anant Corporation
 
Data Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackData Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackAnant Corporation
 
Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache IcebergData Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache IcebergAnant Corporation
 
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOpsApache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOpsAnant Corporation
 
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache CassandraApache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache CassandraAnant Corporation
 
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Anant Corporation
 
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessData Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessAnant Corporation
 

More from Anant Corporation (20)

QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
QLoRA Fine-Tuning on Cassandra Link Data Set (1/2) Cassandra Lunch 137
 
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdfKono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
Kono.IntelCraft.Weekly.AI.LLM.Landscape.2024.02.28.pdf
 
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache PinotData Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
Data Engineer's Lunch 96: Intro to Real Time Analytics Using Apache Pinot
 
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
NoCode, Data & AI LLM Inside Bootcamp: Episode 6 - Design Patterns: Retrieval...
 
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPTAutomate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
Automate your Job and Business with ChatGPT #3 - Fundamentals of LLM/GPT
 
YugabyteDB Developer Tools
YugabyteDB Developer ToolsYugabyteDB Developer Tools
YugabyteDB Developer Tools
 
Episode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapEpisode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
Episode 2: The LLM / GPT / AI Prompt / Data Engineer Roadmap
 
Machine Learning Orchestration with Airflow
Machine Learning Orchestration with AirflowMachine Learning Orchestration with Airflow
Machine Learning Orchestration with Airflow
 
Cassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward TalksCassandra Lunch 130: Recap of Cassandra Forward Talks
Cassandra Lunch 130: Recap of Cassandra Forward Talks
 
Data Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with ArcionData Engineer's Lunch 90: Migrating SQL Data with Arcion
Data Engineer's Lunch 90: Migrating SQL Data with Arcion
 
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
Data Engineer's Lunch 89: Machine Learning Orchestration with AirflowMachine ...
 
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & FutureCassandra Lunch 129: What’s New:  Apache Cassandra 4.1+ Features & Future
Cassandra Lunch 129: What’s New: Apache Cassandra 4.1+ Features & Future
 
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
Data Engineer's Lunch #86: Building Real-Time Applications at Scale: A Case S...
 
Data Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackData Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data Stack
 
CL 121
CL 121CL 121
CL 121
 
Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache IcebergData Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg
 
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOpsApache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
Apache Cassandra Lunch 120: Apache Cassandra Monitoring Made Easy with AxonOps
 
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache CassandraApache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
Apache Cassandra Lunch 119: Desktop GUI Tools for Apache Cassandra
 
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
 
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessData Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
 

Recently uploaded

Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...apidays
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 

Recently uploaded (20)

Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 

How to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCM

  • 1. Cassandra in the Cloud How to Build a Multi-DC Cassandra Cluster in AWS w/ OpsCenter LCM
  • 2. Manual - Error prone. - Days and weeks. - Bad auditability. - Always waiting on someone . - Costs time and money. - Open security issues. - No one legitimate should be doing this manually. Even in the government, you should be automating. Automated - Repeatable - Minutes and hours - Changes stored in source Control - Can do CICD - Can be super secured. - If you are dealing with systems that make money, you should use it.
  • 3. Setting up 3 nodes. Basic bootcamp exercise. How long? Add 4 more nodes. How long then? Add 40 more nodes. How long then? What we want: - Setup 3 Nodes - Add 4 Nodes
  • 4.
  • 5. Infrastructure: AWS Infrastructure: Terraform Install: Ansible Install: OpsCenter LCM Configure: OpsCenter LCM Maintain: OpsCenter LCM Why not just use one thing for everything?
  • 6. We design and build realtime business platforms… By mapping and connecting real-time customer experiences with information systems on fast data & analytics platforms.
  • 7. Resources Cassandra ● cassandra.link ● https://anant.github.io/awesome- cassandra Datastax ● www.datastax.com ● https://docs.datastax.com/en/opscent er/6.7/opsc/LCM/opscLCM.html Ansible ● https://github.com/rackerlabs/ansible- dse Terraform ● https://hackernoon.com/introduction- to-aws-with-terraform-7a8daf261dc0
  • 9. Benefits of AWS ● Ease of deployment ● Scalability (Availability Zones) ● Security (VPC) ● Automation (CloudFormation)
  • 10. Using Terraform to put Cassandra on an EC2 ● Terraform - Infrastructure as Code Software Tool ● Can define and provision a datacenter infrastructure ● Used today to create an EC2 instance on AWS
  • 13. What is a playbook? ● An organized unit of scripts in a YAML file ● Defines work for a server configuration
  • 14. Benefits of Ansible with DSE ● An hour long process can be done in a minute ● Adding users is simplified ● Can be applied to multiple datacenters, clusters, and nodes
  • 15. Playbook: Validation / Common ● Confirm credentials (SSH, pem, etc) ● Modify System Kernel Attributes ○ Network ○ Memory ● Set CPU Governor ● Set EC2 User limits
  • 16. Playbook: OpsCenter ● Add the Datastax repo ● Add DSE Academy credentials if needed. ● Install the OpsCenterD service. ● Start the OpsCenterD service.
  • 17. Installing Clusters with OpsCenter Lifecycle Manager
  • 18. Benefits of LCM ● Simple yet robust User Interface ● Automatic detection of package manager ○ Easy deployment in a variety of environments and conditions ● Health, sanity checks automated during install
  • 19.
  • 20. Installation Process ● Done in OpsCenter ● Config profiles set for each datacenter ○ Profile ○ SSH Credentials ○ Workload ● Nodes added to each datacenter ○ Rack ○ Credentials ○ 1st Node is a Seed Node
  • 22. Playbooks used today ● Cluster Validation ● Cluster Common ● Cluster OpsCenter
  • 23. Resources Cassandra ● cassandra.link ● https://anant.github.io/awesome- cassandra Datastax ● www.datastax.com ● https://docs.datastax.com/en/opscent er/6.7/opsc/LCM/opscLCM.html Ansible ● https://github.com/rackerlabs/ansible- dse Terraform ● https://hackernoon.com/introduction- to-aws-with-terraform-7a8daf261dc0
  • 25. www.anant.us | solutions@anant.us | (855) 262-6826 3 Washington Circle, NW | Suite 301 | Washington, DC 20037 Data & Analytics Cassandra, DataStax, Kafka, Spark Customer Experience Sitecore Information Systems Salesforce, Quickbooks, and more