SlideShare ist ein Scribd-Unternehmen logo
1 von 29
Downloaden Sie, um offline zu lesen
Cassandra Native
Protocol Drivers

Tyler Hobbs, C* and C* driver engineer
About Me
CHART TITLE GOES HERE
Thrift drivers: pycassa, phpcassa,
telephus, and others
DataStax python driver (native
protocol)
Cassandra Engineer
Thrift Drivers
CHART TITLE GOES HERE
RPC Framework, machine generated
Thrift Drivers
Problems?
● Backwards & forwards compatibility
● Too many connections
● No standard interface
● Thrift overhead
● Cluster state must be polled
Problems & Solutions
Backwards/Forwards Compatibility
● Possible with Thrift, but easier with a
query language (CQL)
● Separately versioned query language
and protocol
Problems & Solutions
Too Many Connections
Operation pipelining with the native
protocol
Problems & Solutions
Operation Pipelining
● 127 in-flight ops per connection
● Improves throughput (not latency)
● Out-of-order processing
● Async, event-loop driven
Problems & Solutions
No Standard Interface
● Query language
● Standard policies for load balancing,
connection management, and
retries/failure handling
● More similar to standard RDMBS drivers
Problems & Solutions
Thrift Overhead
Custom protocol, prepared statements
Problems & Solutions
Cluster State Must be Polled
“Control connection,” register for
pushed notifications
New Driver API
Sync/Async Operations

result = session.execute(“SELECT * FROM foo”)
New Driver API
Sync/Async Operations
future = session.execute_async(“SELECT * FROM foo”)
…
result = future.result()
New Driver API
Sync/Async Operations
session.execute_async(query).add_callbacks(
    callback=process_data,
    errback=log_error
)
New Driver Architecture
Connection Pooling

● Min/max conns per remote, local nodes
● Use least busy conn
● Open and close conns as needed
New Driver Architecture
What happens during Operations?
● Nodes to query are picked by
LoadBalancingPolicy
● Failures are handled by RetryPolicy
● On errors, nodes are marked down by
ConvictionPolicy
New Driver Architecture
Load Balancing Policies
● RoundRobin
● DcAwareRoundRobin
● TokenAware wrapper
● Custom
New Driver Architecture
What happens during Operations?
● Nodes to query are picked by
LoadBalancingPolicy
● Failures are handled by RetryPolicy
● On errors, nodes are marked down by
ConvictionPolicy
New Driver Architecture
Retry Policies
● Operation type
● Consistency level
● Number (and type) of responses
● Type of failure
● Retry, raise error, or ignore error
New Driver Architecture
What happens during Operations?
● Nodes to query are picked by
LoadBalancingPolicy
● Failures are handled by RetryPolicy
● On errors, nodes are marked down by
ConvictionPolicy, reconnect with
ReconnectionPolicy
New Driver Architecture
Reconnection Policy
● Schedule for attempting reconnects to
down nodes
● Constant and Exponential backoff
New Driver Architecture
Policy Defaults
● RoundRobin load balancing (not token
or DC aware)
● Retry at most once (in a small number
of cases)
● Mark node down after one failure
● Exponential backoff on reconnection
attempts
New Driver Architecture
Prepared Statements
● Prepared against all nodes
● Cache
● Re-preparation

prepared = session.prepare(“SELECT foo FROM bar WHERE id=?”)
result = session.execute(prepared, [user_id1])
New Driver Architecture
Control Connection
● Listens for pushed updates to cluster
state and schema
● Marks nodes up and down
● Auto discovers nodes in cluster
● Updates schema metadata
New Driver Architecture
Metrics
● Count timeouts, connection errors, and
other errors
● Open connection stats
● Operation latency histogram
New Driver Architecture
Cursors
● No more manual paging over large
queries
● Works across multiple nodes
●

Paging state provided by client
New Driver Architecture
Quick Python Benchmark
● 3 nodes (local, ccm), one conn per host
● 50k individual inserts, single threaded
● Pycassa: ~1200 ops/sec
● DataStax python driver (sync,
blocking): ~950 ops/sec
● DataStax python driver (future
batching): ~3000 ops/sec
● DataStax python driver (callback
chaining): ~7300 ops/sec
New Driver Architecture
Languages Supported
● Java – 1.0 released
●

in Spring 2013

Simple object mapper under
development

● C# - 1.0 released in Summer 2013
●

LINQ integration

● Python – Beta since Summer 2013, 1.0
coming soon
●

Basic mapper available through
cqlengine

● C++ - Currently in Alpha state
● Ruby, JS, PHP – planned, but no
development so far
New Driver Architecture
Languages Supported
github.com/datastax
Questions?
@tylhobbs
thobbs on #cassandra, #datastaxdrivers

Weitere ähnliche Inhalte

Was ist angesagt?

SAP LVM Integration with SAP BPA
SAP LVM Integration with SAP BPASAP LVM Integration with SAP BPA
SAP LVM Integration with SAP BPAAliter Consulting
 
Net flix embracingfailure re-invent2014-141113085858-conversion-gate02
Net flix embracingfailure re-invent2014-141113085858-conversion-gate02Net flix embracingfailure re-invent2014-141113085858-conversion-gate02
Net flix embracingfailure re-invent2014-141113085858-conversion-gate02~Eric Principe
 
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon
 
Geode Transactions by Swapnil Bawaskar
Geode Transactions by Swapnil BawaskarGeode Transactions by Swapnil Bawaskar
Geode Transactions by Swapnil BawaskarPivotalOpenSourceHub
 
hbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architectureHBaseCon
 
Hermes Reliable Replication Protocol - Poster
Hermes Reliable Replication Protocol - Poster Hermes Reliable Replication Protocol - Poster
Hermes Reliable Replication Protocol - Poster Antonios Katsarakis
 
ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...
ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...
ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...Cask Data
 
Flink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache Flink
Flink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache FlinkFlink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache Flink
Flink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache FlinkFlink Forward
 
Inside Dolphin Test Infrastructure
Inside Dolphin Test InfrastructureInside Dolphin Test Infrastructure
Inside Dolphin Test InfrastructureTao Jiang
 
Training Slides: 102 - Basics - Tungsten Replicator - How We Move Your Data
Training Slides: 102 - Basics - Tungsten Replicator - How We Move Your DataTraining Slides: 102 - Basics - Tungsten Replicator - How We Move Your Data
Training Slides: 102 - Basics - Tungsten Replicator - How We Move Your DataContinuent
 
Akka HTTP with Scala
Akka HTTP with ScalaAkka HTTP with Scala
Akka HTTP with ScalaKnoldus Inc.
 
ChinaNetCloud Training - iptables Intro
ChinaNetCloud Training - iptables IntroChinaNetCloud Training - iptables Intro
ChinaNetCloud Training - iptables IntroChinaNetCloud
 

Was ist angesagt? (19)

SAP LVM Integration with SAP BPA
SAP LVM Integration with SAP BPASAP LVM Integration with SAP BPA
SAP LVM Integration with SAP BPA
 
Thomson Reuters
Thomson ReutersThomson Reuters
Thomson Reuters
 
Play Framework
Play FrameworkPlay Framework
Play Framework
 
Net flix embracingfailure re-invent2014-141113085858-conversion-gate02
Net flix embracingfailure re-invent2014-141113085858-conversion-gate02Net flix embracingfailure re-invent2014-141113085858-conversion-gate02
Net flix embracingfailure re-invent2014-141113085858-conversion-gate02
 
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
 
Geode Transactions by Swapnil Bawaskar
Geode Transactions by Swapnil BawaskarGeode Transactions by Swapnil Bawaskar
Geode Transactions by Swapnil Bawaskar
 
hbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
 
Hermes Reliable Replication Protocol - Poster
Hermes Reliable Replication Protocol - Poster Hermes Reliable Replication Protocol - Poster
Hermes Reliable Replication Protocol - Poster
 
Advanced LB
Advanced LBAdvanced LB
Advanced LB
 
Javantura v6 - On the Aspects of Polyglot Programming and Memory Management i...
Javantura v6 - On the Aspects of Polyglot Programming and Memory Management i...Javantura v6 - On the Aspects of Polyglot Programming and Memory Management i...
Javantura v6 - On the Aspects of Polyglot Programming and Memory Management i...
 
SAP Post Copy Automation
SAP Post Copy AutomationSAP Post Copy Automation
SAP Post Copy Automation
 
ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...
ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...
ACID Transactions in Apache Phoenix with Apache Tephra™ (incubating), by Poor...
 
EVCache Builderscon
EVCache BuildersconEVCache Builderscon
EVCache Builderscon
 
Flink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache Flink
Flink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache FlinkFlink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache Flink
Flink Forward Berlin 2017: Tzu-Li (Gordon) Tai - Managing State in Apache Flink
 
Inside Dolphin Test Infrastructure
Inside Dolphin Test InfrastructureInside Dolphin Test Infrastructure
Inside Dolphin Test Infrastructure
 
Training Slides: 102 - Basics - Tungsten Replicator - How We Move Your Data
Training Slides: 102 - Basics - Tungsten Replicator - How We Move Your DataTraining Slides: 102 - Basics - Tungsten Replicator - How We Move Your Data
Training Slides: 102 - Basics - Tungsten Replicator - How We Move Your Data
 
Akka HTTP with Scala
Akka HTTP with ScalaAkka HTTP with Scala
Akka HTTP with Scala
 
Caap presentation by me
Caap presentation by meCaap presentation by me
Caap presentation by me
 
ChinaNetCloud Training - iptables Intro
ChinaNetCloud Training - iptables IntroChinaNetCloud Training - iptables Intro
ChinaNetCloud Training - iptables Intro
 

Ähnlich wie Cassandra drivers

Scaling Magento
Scaling MagentoScaling Magento
Scaling MagentoCopious
 
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)Apache Apex
 
Prelim Slides
Prelim SlidesPrelim Slides
Prelim Slidessmpant
 
Optimizing Python
Optimizing PythonOptimizing Python
Optimizing PythonAdimianBE
 
Linux Kernel vs DPDK: HTTP Performance Showdown
Linux Kernel vs DPDK: HTTP Performance ShowdownLinux Kernel vs DPDK: HTTP Performance Showdown
Linux Kernel vs DPDK: HTTP Performance ShowdownScyllaDB
 
DrupalCon 2014: A Perfect Launch, Every Time
DrupalCon 2014: A Perfect Launch, Every TimeDrupalCon 2014: A Perfect Launch, Every Time
DrupalCon 2014: A Perfect Launch, Every TimePantheon
 
Verification Strategy for PCI-Express
Verification Strategy for PCI-ExpressVerification Strategy for PCI-Express
Verification Strategy for PCI-ExpressDVClub
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBaseMichael Stack
 
Simplified Troubleshooting through API Scripting
Simplified Troubleshooting through API Scripting Simplified Troubleshooting through API Scripting
Simplified Troubleshooting through API Scripting Network Automation Forum
 
LCE13: Test and Validation Summit: The future of testing at Linaro
LCE13: Test and Validation Summit: The future of testing at LinaroLCE13: Test and Validation Summit: The future of testing at Linaro
LCE13: Test and Validation Summit: The future of testing at LinaroLinaro
 
LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...
LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...
LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...Linaro
 
Informix HA Best Practices
Informix HA Best Practices Informix HA Best Practices
Informix HA Best Practices Scott Lashley
 
Always on high availability best practices for informix
Always on high availability best practices for informixAlways on high availability best practices for informix
Always on high availability best practices for informixIBM_Info_Management
 
Adaptive indexing throttling
Adaptive indexing throttling Adaptive indexing throttling
Adaptive indexing throttling Arpit Jain
 
Performance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei RadovPerformance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei RadovValeriia Maliarenko
 
Shared Database Concurrency
Shared Database ConcurrencyShared Database Concurrency
Shared Database ConcurrencyAivars Kalvans
 

Ähnlich wie Cassandra drivers (20)

Scaling Magento
Scaling MagentoScaling Magento
Scaling Magento
 
Optimizing Linux Servers
Optimizing Linux ServersOptimizing Linux Servers
Optimizing Linux Servers
 
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)
 
Prelim Slides
Prelim SlidesPrelim Slides
Prelim Slides
 
Optimizing Python
Optimizing PythonOptimizing Python
Optimizing Python
 
Linux Kernel vs DPDK: HTTP Performance Showdown
Linux Kernel vs DPDK: HTTP Performance ShowdownLinux Kernel vs DPDK: HTTP Performance Showdown
Linux Kernel vs DPDK: HTTP Performance Showdown
 
DrupalCon 2014: A Perfect Launch, Every Time
DrupalCon 2014: A Perfect Launch, Every TimeDrupalCon 2014: A Perfect Launch, Every Time
DrupalCon 2014: A Perfect Launch, Every Time
 
Verification Strategy for PCI-Express
Verification Strategy for PCI-ExpressVerification Strategy for PCI-Express
Verification Strategy for PCI-Express
 
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBasehbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
hbaseconasia2019 Test-suite for Automating Data-consistency checks on HBase
 
Balancing Power & Performance Webinar
Balancing Power & Performance WebinarBalancing Power & Performance Webinar
Balancing Power & Performance Webinar
 
Simplified Troubleshooting through API Scripting
Simplified Troubleshooting through API Scripting Simplified Troubleshooting through API Scripting
Simplified Troubleshooting through API Scripting
 
LCE13: Test and Validation Summit: The future of testing at Linaro
LCE13: Test and Validation Summit: The future of testing at LinaroLCE13: Test and Validation Summit: The future of testing at Linaro
LCE13: Test and Validation Summit: The future of testing at Linaro
 
LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...
LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...
LCE13: Test and Validation Mini-Summit: Review Current Linaro Engineering Pro...
 
Performance Whackamole (short version)
Performance Whackamole (short version)Performance Whackamole (short version)
Performance Whackamole (short version)
 
Informix HA Best Practices
Informix HA Best Practices Informix HA Best Practices
Informix HA Best Practices
 
Always on high availability best practices for informix
Always on high availability best practices for informixAlways on high availability best practices for informix
Always on high availability best practices for informix
 
R12.2 dba
R12.2 dbaR12.2 dba
R12.2 dba
 
Adaptive indexing throttling
Adaptive indexing throttling Adaptive indexing throttling
Adaptive indexing throttling
 
Performance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei RadovPerformance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei Radov
 
Shared Database Concurrency
Shared Database ConcurrencyShared Database Concurrency
Shared Database Concurrency
 

Kürzlich hochgeladen

UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 

Kürzlich hochgeladen (20)

UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 

Cassandra drivers

  • 1. Cassandra Native Protocol Drivers Tyler Hobbs, C* and C* driver engineer
  • 2. About Me CHART TITLE GOES HERE Thrift drivers: pycassa, phpcassa, telephus, and others DataStax python driver (native protocol) Cassandra Engineer
  • 3. Thrift Drivers CHART TITLE GOES HERE RPC Framework, machine generated
  • 4. Thrift Drivers Problems? ● Backwards & forwards compatibility ● Too many connections ● No standard interface ● Thrift overhead ● Cluster state must be polled
  • 5. Problems & Solutions Backwards/Forwards Compatibility ● Possible with Thrift, but easier with a query language (CQL) ● Separately versioned query language and protocol
  • 6. Problems & Solutions Too Many Connections Operation pipelining with the native protocol
  • 7. Problems & Solutions Operation Pipelining ● 127 in-flight ops per connection ● Improves throughput (not latency) ● Out-of-order processing ● Async, event-loop driven
  • 8. Problems & Solutions No Standard Interface ● Query language ● Standard policies for load balancing, connection management, and retries/failure handling ● More similar to standard RDMBS drivers
  • 9. Problems & Solutions Thrift Overhead Custom protocol, prepared statements
  • 10. Problems & Solutions Cluster State Must be Polled “Control connection,” register for pushed notifications
  • 11. New Driver API Sync/Async Operations result = session.execute(“SELECT * FROM foo”)
  • 12. New Driver API Sync/Async Operations future = session.execute_async(“SELECT * FROM foo”) … result = future.result()
  • 13. New Driver API Sync/Async Operations session.execute_async(query).add_callbacks(     callback=process_data,     errback=log_error )
  • 14. New Driver Architecture Connection Pooling ● Min/max conns per remote, local nodes ● Use least busy conn ● Open and close conns as needed
  • 15. New Driver Architecture What happens during Operations? ● Nodes to query are picked by LoadBalancingPolicy ● Failures are handled by RetryPolicy ● On errors, nodes are marked down by ConvictionPolicy
  • 16. New Driver Architecture Load Balancing Policies ● RoundRobin ● DcAwareRoundRobin ● TokenAware wrapper ● Custom
  • 17. New Driver Architecture What happens during Operations? ● Nodes to query are picked by LoadBalancingPolicy ● Failures are handled by RetryPolicy ● On errors, nodes are marked down by ConvictionPolicy
  • 18. New Driver Architecture Retry Policies ● Operation type ● Consistency level ● Number (and type) of responses ● Type of failure ● Retry, raise error, or ignore error
  • 19. New Driver Architecture What happens during Operations? ● Nodes to query are picked by LoadBalancingPolicy ● Failures are handled by RetryPolicy ● On errors, nodes are marked down by ConvictionPolicy, reconnect with ReconnectionPolicy
  • 20. New Driver Architecture Reconnection Policy ● Schedule for attempting reconnects to down nodes ● Constant and Exponential backoff
  • 21. New Driver Architecture Policy Defaults ● RoundRobin load balancing (not token or DC aware) ● Retry at most once (in a small number of cases) ● Mark node down after one failure ● Exponential backoff on reconnection attempts
  • 22. New Driver Architecture Prepared Statements ● Prepared against all nodes ● Cache ● Re-preparation prepared = session.prepare(“SELECT foo FROM bar WHERE id=?”) result = session.execute(prepared, [user_id1])
  • 23. New Driver Architecture Control Connection ● Listens for pushed updates to cluster state and schema ● Marks nodes up and down ● Auto discovers nodes in cluster ● Updates schema metadata
  • 24. New Driver Architecture Metrics ● Count timeouts, connection errors, and other errors ● Open connection stats ● Operation latency histogram
  • 25. New Driver Architecture Cursors ● No more manual paging over large queries ● Works across multiple nodes ● Paging state provided by client
  • 26. New Driver Architecture Quick Python Benchmark ● 3 nodes (local, ccm), one conn per host ● 50k individual inserts, single threaded ● Pycassa: ~1200 ops/sec ● DataStax python driver (sync, blocking): ~950 ops/sec ● DataStax python driver (future batching): ~3000 ops/sec ● DataStax python driver (callback chaining): ~7300 ops/sec
  • 27. New Driver Architecture Languages Supported ● Java – 1.0 released ● in Spring 2013 Simple object mapper under development ● C# - 1.0 released in Summer 2013 ● LINQ integration ● Python – Beta since Summer 2013, 1.0 coming soon ● Basic mapper available through cqlengine ● C++ - Currently in Alpha state ● Ruby, JS, PHP – planned, but no development so far
  • 28. New Driver Architecture Languages Supported github.com/datastax

Hinweis der Redaktion

  1. Maintainer of pycassa, phpcassa, and telephus. I've also worked on Hector and clj-hector a bit. The Datastax python driver was originally ported from the DataStax Java driver.