SlideShare ist ein Scribd-Unternehmen logo
1 von 58
Downloaden Sie, um offline zu lesen
LWTs in practice
Christopher Batey
@chbatey
The Last Pickle
Overview
Review of Cassandra’s consistency model
What are LWTs?
Why do we need them?
How do they work?
How do you use them?
Writes
Client A
C
Concurrent writes
Client A
C
Client B
C
QUORUM based consistency
Client A
C
Client B
C
QUORUM based consistency
Client A
C
Client B
C
QUORUM based consistency
Client A
C
Client B
C
Voucher example
CREATE TABLE vouchers_mutable (
name text PRIMARY KEY,
sold int
)
Read and write race condition with Quorum
● Client A read the number of ticket sales at 299
● Client B read the number of ticket sales at 299
● Client A sells ticket 300
● Client B sells ticket 300
Compare and set
Enter Light Weight Transactions
● Client A read the number of ticket sales at 299
● Client B read the number of ticket sales at 299
● Client A sells ticket 300 if total sold is 299
● Client B sells ticket 300 if total sold is 299
Examples
Uniqueness
CREATE TABLE users (
user_name text PRIMARY KEY,
email text,
password text
)
INSERT INTO users (user_name, password, email )
VALUES ( 'chbatey', 'different',
'adifferentchris@gmail.com' ) IF NOT EXISTS;
Finite resource
CREATE TABLE vouchers_mutable (
name text PRIMARY KEY,
sold int
)
UPDATE vouchers_mutable SET sold = 1
WHERE name = 'free tv' IF sold = 0;
Immutable events
CREATE TABLE vouchers (
name text,
when timeuuid,
who text,
PRIMARY KEY (name, when)
);
Batches + LWTs
CREATE TABLE vouchers (
name text,
when timeuuid,
sold int static,
who text,
PRIMARY KEY (name, when)
);
INSERT INTO vouchers (name, sold) VALUES
( 'free tv', 0);
Batches + LWTs
BEGIN BATCH
UPDATE vouchers SET sold = 1 WHERE name = 'free tv' IF sold = 0
INSERT INTO vouchers (name, when, who) VALUES ( 'free tv', now(), 'chris')
APPLY BATCH ;
[applied]
-----------
True
Batches + LWTs
BEGIN BATCH
UPDATE vouchers SET sold = 1 WHERE name = 'free tv' IF sold = 0
INSERT INTO vouchers (name, when, who) VALUES ( 'free tv', now(), 'charlie')
APPLY BATCH ;
[applied] | name | when | sold
-----------+---------+------+------
False | free tv | null | 1
How they work
LWTs be puzzling
1. Why does a LWT have two consistency levels?
2. What is this SERIAL consistency I keep hearing about?
3. What are SERIAL reads?
4. Why does my LWT fail but the value still get written?
5. Why are they so damn slow?
Consensus for a partition
Consensus for a partition
Stages of a LWT
● Prepare and promise
● Read existing value
● Propose and accept
● Commit
Consensus for a partition
Promised
I want the
value to be
5, as long as
it currently 4
prepare propose
Accepted
commit
Committed
Condition
met
read
Prepare and promise 1
Client
partition table promised accepted committed
Prepare 1
Prepare and promise 1
Client
partition table promised accepted committed
A vouchers 1
Prepare and promise 2
Client
partition table promised accepted committed
A vouchers 1
Prepare 2
Prepare and promise 2
Client
partition table promised accepted committed
A vouchers 2
Prepare and promise - rejection
Client
partition table promised accepted committed
A vouchers 2
Prepare 1
Prepare and promise - rejection
Client
partition table promised accepted committed
A vouchers 2
Rejected
- ClientRequest.CASWrite.contentions
Prepare 1
Prepare and promise - trace
Parsing insert into users (user_name, password, email ) values ( 'chbatey', 'chrisrocks', 'christopher.batey@gmail.com' ) if
not exists; [SharedPool-Worker-1] | 2016-08-22 12:38:44.132000 | 127.0.0.1 | 1125
Sending PAXOS_PREPARE message to /127.0.0.3 [MessagingService-Outgoing-/127.0.0.3] | 2016-08-22 12:38:44.141000
| 127.0.0.1 | 10414
Sending PAXOS_PREPARE message to /127.0.0.2 [MessagingService-Outgoing-/127.0.0.2] | 2016-08-22 12:38:44.142000
| 127.0.0.1 | 10908
Promising ballot fb282190-685c-11e6-71a2-e0d2d098d5d6 [SharedPool-Worker-1] | 2016-08-22 12:38:44.147000 |
127.0.0.3 | 4325
Prepare and promise - trace
Promising ballot fb282190-685c-11e6-71a2-e0d2d098d5d6 [SharedPool-Worker-1] | 2016-08-22 12:38:44.147000 |
127.0.0.3 | 4325
Promising ballot fb282190-685c-11e6-71a2-e0d2d098d5d6 [SharedPool-Worker-3] | 2016-08-22 12:38:44.166000 |
127.0.0.1 | 35282
Read
LOCAL_SERIAL => LOCAL_QUORUM
SERIAL => QUORUM
ClinetRequest.CASWrite.conditionNotMet
Propose and accept
Client
partition table promised accepted committed
A vouchers 1
Propose 1
Propose and accept
Client
partition table promised accepted committed
A vouchers 1 1
Propose 1
Propose and accept - rejection
Client
partition table promised accepted committed
A vouchers 2
Propose 1
Rejected
- ClientRequest.CASWrite.contentions
Propose and accept - trace
Sending PAXOS_PROPOSE message to /127.0.0.2 [MessagingService-Outgoing-/127.0.0.2] | 2016-08-22
12:38:44.196000 | 127.0.0.1 | 65606
Sending PAXOS_PROPOSE message to /127.0.0.1 [MessagingService-Outgoing-/127.0.0.1] | 2016-08-22
12:38:44.196000 | 127.0.0.1 | 65606
PAXOS_PROPOSE message received from /127.0.0.1 [MessagingService-Incoming-/127.0.0.1] | 2016-08-22
12:38:44.197000 | 127.0.0.1 | 65986
Sending PAXOS_PROPOSE message to /127.0.0.3 [MessagingService-Outgoing-/127.0.0.3] | 2016-08-22
12:38:44.197000 | 127.0.0.1 | 66139
Propose and accept - trace
Accepting proposal Commit(fb282190-685c-11e6-71a2-e0d2d098d5d6, [lwts.users] key=chbatey columns=[[] | [email
password]]n Row: EMPTY | email=christopher.batey@gmail.com, password=chrisrocks) [SharedPool-Worker-2] |
2016-08-22 12:38:44.199000 | 127.0.0.1 | 67804
Commit
● The normal consistency is now used for the commit
Consensus for a partition
Promised
I want the
value to be
5, as long as
it currently 4
prepare propose
Accepted
commit
Committed
Condition
met
read
SERIAL reads
o.a.c.s.StorageProxy.readWithPaxos
● For a single partition
● Runs a prepare and ensures all replicas have the latest commit
● Then runs the read at either Q or LQ
Write timestamps
Client A
C
2
1
3
C
Write timestamps
Client A
C
2
1
3
C
Write timestamps
Client A
C
2
1
3
C
T = 1
T = 2
Some numbers
Setup
4 * i2xLarge
RF = 3
10 clients trying to buy 1000 vouchers each - 10k total operations
Contention: all clients buying the same voucher (same partitoin)
No contention: all clients after different vouchers (different partition)
Mutable field
CREATE TABLE vouchers_mutable (
name text PRIMARY KEY,
sold int
) UPDATE vouchers_mutable SET sold = 1
WHERE name = 'free tv' IF sold = 0;
UPDATE vouchers_mutable SET sold = 1
WHERE name = 'free tv'1)
2)
Histogram
Histogram
Batches
CREATE TABLE vouchers (
name text,
when timeuuid,
sold int static,
who text,
PRIMARY KEY (name, when)
);
BEGIN BATCH
UPDATE vouchers SET sold = 1 WHERE name = 'free tv' IF sold = 0
INSERT INTO vouchers (name, when, who) VALUES ( 'free tv', now(), 'charlie')
APPLY BATCH ;
Histogram
Histogram
Summary
LWT Batch Contention Incorrect results 99th %ile
(milliseconds)
N N Y 87% Lost 48
Y N Y 0% Lost
1% Unknown
81% CNM
191
Y N N 0% Lost
0% Unknown
0% CNM
52
Y Y Y 0% Lost
<1% Unknown
82% CNM
192
Summary
Summary
● LWTs are expensive
● They are more complex and less mature than the regular read and write path
● Might be a lot easier than bringing in a second technology
Questions?
Christopher Batey
@chbatey
The Last Pickle

Weitere ähnliche Inhalte

Was ist angesagt?

5952 database systems administration (comp 1011.1)-cw1
5952   database systems administration (comp 1011.1)-cw15952   database systems administration (comp 1011.1)-cw1
5952 database systems administration (comp 1011.1)-cw1
saeedkhan841514
 
Store and Process Big Data with Hadoop and Cassandra
Store and Process Big Data with Hadoop and CassandraStore and Process Big Data with Hadoop and Cassandra
Store and Process Big Data with Hadoop and Cassandra
Deependra Ariyadewa
 
Sqlxml vs xquery
Sqlxml vs xquerySqlxml vs xquery
Sqlxml vs xquery
Amol Pujari
 

Was ist angesagt? (20)

Temporary Cache Assistance (Transients API): WordCamp Phoenix 2014
Temporary Cache Assistance (Transients API): WordCamp Phoenix 2014Temporary Cache Assistance (Transients API): WordCamp Phoenix 2014
Temporary Cache Assistance (Transients API): WordCamp Phoenix 2014
 
Functional streams with Kafka - A comparison between Akka-streams and FS2
Functional streams with Kafka - A comparison between Akka-streams and FS2Functional streams with Kafka - A comparison between Akka-streams and FS2
Functional streams with Kafka - A comparison between Akka-streams and FS2
 
Design Patterns in Micro-services architectures & Gilmour
Design Patterns in Micro-services architectures & GilmourDesign Patterns in Micro-services architectures & Gilmour
Design Patterns in Micro-services architectures & Gilmour
 
Temporary Cache Assistance (Transients API): WordCamp Birmingham 2014
Temporary Cache Assistance (Transients API): WordCamp Birmingham 2014Temporary Cache Assistance (Transients API): WordCamp Birmingham 2014
Temporary Cache Assistance (Transients API): WordCamp Birmingham 2014
 
Cassandra Day SV 2014: Fundamentals of Apache Cassandra Data Modeling
Cassandra Day SV 2014: Fundamentals of Apache Cassandra Data ModelingCassandra Day SV 2014: Fundamentals of Apache Cassandra Data Modeling
Cassandra Day SV 2014: Fundamentals of Apache Cassandra Data Modeling
 
Cassandra nice use cases and worst anti patterns
Cassandra nice use cases and worst anti patternsCassandra nice use cases and worst anti patterns
Cassandra nice use cases and worst anti patterns
 
QB Into the Box 2018
QB Into the Box 2018QB Into the Box 2018
QB Into the Box 2018
 
MongoDB .local Paris 2020: La puissance du Pipeline d'Agrégation de MongoDB
MongoDB .local Paris 2020: La puissance du Pipeline d'Agrégation de MongoDBMongoDB .local Paris 2020: La puissance du Pipeline d'Agrégation de MongoDB
MongoDB .local Paris 2020: La puissance du Pipeline d'Agrégation de MongoDB
 
5952 database systems administration (comp 1011.1)-cw1
5952   database systems administration (comp 1011.1)-cw15952   database systems administration (comp 1011.1)-cw1
5952 database systems administration (comp 1011.1)-cw1
 
MongoDB World 2018: Using Change Streams to Keep Up with Your Data
MongoDB World 2018: Using Change Streams to Keep Up with Your DataMongoDB World 2018: Using Change Streams to Keep Up with Your Data
MongoDB World 2018: Using Change Streams to Keep Up with Your Data
 
Cassandra Day Atlanta 2015: Building Your First Application with Apache Cassa...
Cassandra Day Atlanta 2015: Building Your First Application with Apache Cassa...Cassandra Day Atlanta 2015: Building Your First Application with Apache Cassa...
Cassandra Day Atlanta 2015: Building Your First Application with Apache Cassa...
 
Cassandra Day Chicago 2015: Advanced Data Modeling
Cassandra Day Chicago 2015: Advanced Data ModelingCassandra Day Chicago 2015: Advanced Data Modeling
Cassandra Day Chicago 2015: Advanced Data Modeling
 
Advanced data modeling with apache cassandra
Advanced data modeling with apache cassandraAdvanced data modeling with apache cassandra
Advanced data modeling with apache cassandra
 
Apache Cassandra, part 2 – data model example, machinery
Apache Cassandra, part 2 – data model example, machineryApache Cassandra, part 2 – data model example, machinery
Apache Cassandra, part 2 – data model example, machinery
 
Store and Process Big Data with Hadoop and Cassandra
Store and Process Big Data with Hadoop and CassandraStore and Process Big Data with Hadoop and Cassandra
Store and Process Big Data with Hadoop and Cassandra
 
How to Use JSON in MySQL Wrong
How to Use JSON in MySQL WrongHow to Use JSON in MySQL Wrong
How to Use JSON in MySQL Wrong
 
Lab1-DB-Cassandra
Lab1-DB-CassandraLab1-DB-Cassandra
Lab1-DB-Cassandra
 
DB2 Native XML
DB2 Native XMLDB2 Native XML
DB2 Native XML
 
Sqlxml vs xquery
Sqlxml vs xquerySqlxml vs xquery
Sqlxml vs xquery
 
Service discovery and configuration provisioning
Service discovery and configuration provisioningService discovery and configuration provisioning
Service discovery and configuration provisioning
 

Andere mochten auch

Andere mochten auch (18)

Docker and jvm. A good idea?
Docker and jvm. A good idea?Docker and jvm. A good idea?
Docker and jvm. A good idea?
 
Think your software is fault-tolerant? Prove it!
Think your software is fault-tolerant? Prove it!Think your software is fault-tolerant? Prove it!
Think your software is fault-tolerant? Prove it!
 
Cassandra Day London: Building Java Applications
Cassandra Day London: Building Java ApplicationsCassandra Day London: Building Java Applications
Cassandra Day London: Building Java Applications
 
NYC Cassandra Day - Java Intro
NYC Cassandra Day - Java IntroNYC Cassandra Day - Java Intro
NYC Cassandra Day - Java Intro
 
Manchester Hadoop Meetup: Cassandra Spark internals
Manchester Hadoop Meetup: Cassandra Spark internalsManchester Hadoop Meetup: Cassandra Spark internals
Manchester Hadoop Meetup: Cassandra Spark internals
 
LJC: Microservices in the real world
LJC: Microservices in the real worldLJC: Microservices in the real world
LJC: Microservices in the real world
 
Devoxx France: Fault tolerant microservices on the JVM with Cassandra
Devoxx France: Fault tolerant microservices on the JVM with CassandraDevoxx France: Fault tolerant microservices on the JVM with Cassandra
Devoxx France: Fault tolerant microservices on the JVM with Cassandra
 
The Best and Worst of Cassandra-stress Tool (Christopher Batey, The Last Pick...
The Best and Worst of Cassandra-stress Tool (Christopher Batey, The Last Pick...The Best and Worst of Cassandra-stress Tool (Christopher Batey, The Last Pick...
The Best and Worst of Cassandra-stress Tool (Christopher Batey, The Last Pick...
 
IoT London July 2015
IoT London July 2015IoT London July 2015
IoT London July 2015
 
1 Dundee - Cassandra 101
1 Dundee - Cassandra 1011 Dundee - Cassandra 101
1 Dundee - Cassandra 101
 
Dublin Meetup: Cassandra anti patterns
Dublin Meetup: Cassandra anti patternsDublin Meetup: Cassandra anti patterns
Dublin Meetup: Cassandra anti patterns
 
3 Dundee-Spark Overview for C* developers
3 Dundee-Spark Overview for C* developers3 Dundee-Spark Overview for C* developers
3 Dundee-Spark Overview for C* developers
 
Cassandra London - C* Spark Connector
Cassandra London - C* Spark ConnectorCassandra London - C* Spark Connector
Cassandra London - C* Spark Connector
 
2 Dundee - Cassandra-3
2 Dundee - Cassandra-32 Dundee - Cassandra-3
2 Dundee - Cassandra-3
 
Manchester Hadoop User Group: Cassandra Intro
Manchester Hadoop User Group: Cassandra IntroManchester Hadoop User Group: Cassandra Intro
Manchester Hadoop User Group: Cassandra Intro
 
Manchester Hadoop Meetup: Spark Cassandra Integration
Manchester Hadoop Meetup: Spark Cassandra IntegrationManchester Hadoop Meetup: Spark Cassandra Integration
Manchester Hadoop Meetup: Spark Cassandra Integration
 
Load Testing Cassandra Applications (Ben Slater, Instaclustr) | C* Summit 2016
Load Testing Cassandra Applications (Ben Slater, Instaclustr) | C* Summit 2016Load Testing Cassandra Applications (Ben Slater, Instaclustr) | C* Summit 2016
Load Testing Cassandra Applications (Ben Slater, Instaclustr) | C* Summit 2016
 
Paris Day Cassandra: Use case
Paris Day Cassandra: Use caseParis Day Cassandra: Use case
Paris Day Cassandra: Use case
 

Ähnlich wie Cassandra summit LWTs

Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...
Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...
Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...
StreamNative
 

Ähnlich wie Cassandra summit LWTs (20)

apidays Singapore 2023 - Iterate Faster with Dynamic Flows, Yee Hui Poh, Wise
apidays Singapore 2023 - Iterate Faster with Dynamic Flows, Yee Hui Poh, Wiseapidays Singapore 2023 - Iterate Faster with Dynamic Flows, Yee Hui Poh, Wise
apidays Singapore 2023 - Iterate Faster with Dynamic Flows, Yee Hui Poh, Wise
 
The Current State of Table API in 2022
The Current State of Table API in 2022The Current State of Table API in 2022
The Current State of Table API in 2022
 
Cli deep dive
Cli deep diveCli deep dive
Cli deep dive
 
Vienna Feb 2015: Cassandra: How it works and what it's good for!
Vienna Feb 2015: Cassandra: How it works and what it's good for!Vienna Feb 2015: Cassandra: How it works and what it's good for!
Vienna Feb 2015: Cassandra: How it works and what it's good for!
 
AWS July Webinar Series: Amazon Redshift Optimizing Performance
AWS July Webinar Series: Amazon Redshift Optimizing PerformanceAWS July Webinar Series: Amazon Redshift Optimizing Performance
AWS July Webinar Series: Amazon Redshift Optimizing Performance
 
Should it be routine to use coroutines?
Should it be routine to use coroutines?Should it be routine to use coroutines?
Should it be routine to use coroutines?
 
Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...
Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...
Exactly-Once Made Easy: Transactional Messaging in Apache Pulsar - Pulsar Sum...
 
Cassandra 2.2 & 3.0
Cassandra 2.2 & 3.0Cassandra 2.2 & 3.0
Cassandra 2.2 & 3.0
 
Advanced smart contract
Advanced smart contractAdvanced smart contract
Advanced smart contract
 
Cassandra Day London 2015: Apache Cassandra Anti-Patterns
Cassandra Day London 2015: Apache Cassandra Anti-PatternsCassandra Day London 2015: Apache Cassandra Anti-Patterns
Cassandra Day London 2015: Apache Cassandra Anti-Patterns
 
Webinar Cassandra Anti-Patterns
Webinar Cassandra Anti-PatternsWebinar Cassandra Anti-Patterns
Webinar Cassandra Anti-Patterns
 
Cassandra lesson learned - extended
Cassandra   lesson learned  - extendedCassandra   lesson learned  - extended
Cassandra lesson learned - extended
 
PowerShell Tips & Tricks for Exchange
PowerShell Tips & Tricks for ExchangePowerShell Tips & Tricks for Exchange
PowerShell Tips & Tricks for Exchange
 
Testing in the World of Functional Programming
Testing in the World of Functional ProgrammingTesting in the World of Functional Programming
Testing in the World of Functional Programming
 
Real data models of silicon valley
Real data models of silicon valleyReal data models of silicon valley
Real data models of silicon valley
 
Event-Driven Microservices: Back to the Basics
Event-Driven Microservices: Back to the BasicsEvent-Driven Microservices: Back to the Basics
Event-Driven Microservices: Back to the Basics
 
Cassandra - lesson learned
Cassandra  - lesson learnedCassandra  - lesson learned
Cassandra - lesson learned
 
Apache Cassandra, part 3 – machinery, work with Cassandra
Apache Cassandra, part 3 – machinery, work with CassandraApache Cassandra, part 3 – machinery, work with Cassandra
Apache Cassandra, part 3 – machinery, work with Cassandra
 
Dun ddd
Dun dddDun ddd
Dun ddd
 
Lec2&3 data structure
Lec2&3 data structureLec2&3 data structure
Lec2&3 data structure
 

Mehr von Christopher Batey

Mehr von Christopher Batey (10)

Data Science Lab Meetup: Cassandra and Spark
Data Science Lab Meetup: Cassandra and SparkData Science Lab Meetup: Cassandra and Spark
Data Science Lab Meetup: Cassandra and Spark
 
Munich March 2015 - Cassandra + Spark Overview
Munich March 2015 -  Cassandra + Spark OverviewMunich March 2015 -  Cassandra + Spark Overview
Munich March 2015 - Cassandra + Spark Overview
 
Reading Cassandra Meetup Feb 2015: Apache Spark
Reading Cassandra Meetup Feb 2015: Apache SparkReading Cassandra Meetup Feb 2015: Apache Spark
Reading Cassandra Meetup Feb 2015: Apache Spark
 
LA Cassandra Day 2015 - Testing Cassandra
LA Cassandra Day 2015  - Testing CassandraLA Cassandra Day 2015  - Testing Cassandra
LA Cassandra Day 2015 - Testing Cassandra
 
LA Cassandra Day 2015 - Cassandra for developers
LA Cassandra Day 2015  - Cassandra for developersLA Cassandra Day 2015  - Cassandra for developers
LA Cassandra Day 2015 - Cassandra for developers
 
Voxxed Vienna 2015 Fault tolerant microservices
Voxxed Vienna 2015 Fault tolerant microservicesVoxxed Vienna 2015 Fault tolerant microservices
Voxxed Vienna 2015 Fault tolerant microservices
 
Jan 2015 - Cassandra101 Manchester Meetup
Jan 2015 - Cassandra101 Manchester MeetupJan 2015 - Cassandra101 Manchester Meetup
Jan 2015 - Cassandra101 Manchester Meetup
 
LJC: Fault tolerance with Apache Cassandra
LJC: Fault tolerance with Apache CassandraLJC: Fault tolerance with Apache Cassandra
LJC: Fault tolerance with Apache Cassandra
 
Cassandra Summit EU 2014 Lightning talk - Paging (no animation)
Cassandra Summit EU 2014 Lightning talk - Paging (no animation)Cassandra Summit EU 2014 Lightning talk - Paging (no animation)
Cassandra Summit EU 2014 Lightning talk - Paging (no animation)
 
Cassandra Summit EU 2014 - Testing Cassandra Applications
Cassandra Summit EU 2014 - Testing Cassandra ApplicationsCassandra Summit EU 2014 - Testing Cassandra Applications
Cassandra Summit EU 2014 - Testing Cassandra Applications
 

Kürzlich hochgeladen

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 

Kürzlich hochgeladen (20)

Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 

Cassandra summit LWTs

  • 1. LWTs in practice Christopher Batey @chbatey The Last Pickle
  • 2.
  • 3.
  • 4. Overview Review of Cassandra’s consistency model What are LWTs? Why do we need them? How do they work? How do you use them?
  • 10. Voucher example CREATE TABLE vouchers_mutable ( name text PRIMARY KEY, sold int )
  • 11. Read and write race condition with Quorum ● Client A read the number of ticket sales at 299 ● Client B read the number of ticket sales at 299 ● Client A sells ticket 300 ● Client B sells ticket 300
  • 13. Enter Light Weight Transactions ● Client A read the number of ticket sales at 299 ● Client B read the number of ticket sales at 299 ● Client A sells ticket 300 if total sold is 299 ● Client B sells ticket 300 if total sold is 299
  • 15. Uniqueness CREATE TABLE users ( user_name text PRIMARY KEY, email text, password text ) INSERT INTO users (user_name, password, email ) VALUES ( 'chbatey', 'different', 'adifferentchris@gmail.com' ) IF NOT EXISTS;
  • 16. Finite resource CREATE TABLE vouchers_mutable ( name text PRIMARY KEY, sold int ) UPDATE vouchers_mutable SET sold = 1 WHERE name = 'free tv' IF sold = 0;
  • 17. Immutable events CREATE TABLE vouchers ( name text, when timeuuid, who text, PRIMARY KEY (name, when) );
  • 18. Batches + LWTs CREATE TABLE vouchers ( name text, when timeuuid, sold int static, who text, PRIMARY KEY (name, when) ); INSERT INTO vouchers (name, sold) VALUES ( 'free tv', 0);
  • 19. Batches + LWTs BEGIN BATCH UPDATE vouchers SET sold = 1 WHERE name = 'free tv' IF sold = 0 INSERT INTO vouchers (name, when, who) VALUES ( 'free tv', now(), 'chris') APPLY BATCH ; [applied] ----------- True
  • 20. Batches + LWTs BEGIN BATCH UPDATE vouchers SET sold = 1 WHERE name = 'free tv' IF sold = 0 INSERT INTO vouchers (name, when, who) VALUES ( 'free tv', now(), 'charlie') APPLY BATCH ; [applied] | name | when | sold -----------+---------+------+------ False | free tv | null | 1
  • 22. LWTs be puzzling 1. Why does a LWT have two consistency levels? 2. What is this SERIAL consistency I keep hearing about? 3. What are SERIAL reads? 4. Why does my LWT fail but the value still get written? 5. Why are they so damn slow?
  • 23. Consensus for a partition
  • 24. Consensus for a partition
  • 25. Stages of a LWT ● Prepare and promise ● Read existing value ● Propose and accept ● Commit
  • 26. Consensus for a partition Promised I want the value to be 5, as long as it currently 4 prepare propose Accepted commit Committed Condition met read
  • 27. Prepare and promise 1 Client partition table promised accepted committed Prepare 1
  • 28. Prepare and promise 1 Client partition table promised accepted committed A vouchers 1
  • 29. Prepare and promise 2 Client partition table promised accepted committed A vouchers 1 Prepare 2
  • 30. Prepare and promise 2 Client partition table promised accepted committed A vouchers 2
  • 31. Prepare and promise - rejection Client partition table promised accepted committed A vouchers 2 Prepare 1
  • 32. Prepare and promise - rejection Client partition table promised accepted committed A vouchers 2 Rejected - ClientRequest.CASWrite.contentions Prepare 1
  • 33. Prepare and promise - trace Parsing insert into users (user_name, password, email ) values ( 'chbatey', 'chrisrocks', 'christopher.batey@gmail.com' ) if not exists; [SharedPool-Worker-1] | 2016-08-22 12:38:44.132000 | 127.0.0.1 | 1125 Sending PAXOS_PREPARE message to /127.0.0.3 [MessagingService-Outgoing-/127.0.0.3] | 2016-08-22 12:38:44.141000 | 127.0.0.1 | 10414 Sending PAXOS_PREPARE message to /127.0.0.2 [MessagingService-Outgoing-/127.0.0.2] | 2016-08-22 12:38:44.142000 | 127.0.0.1 | 10908 Promising ballot fb282190-685c-11e6-71a2-e0d2d098d5d6 [SharedPool-Worker-1] | 2016-08-22 12:38:44.147000 | 127.0.0.3 | 4325
  • 34. Prepare and promise - trace Promising ballot fb282190-685c-11e6-71a2-e0d2d098d5d6 [SharedPool-Worker-1] | 2016-08-22 12:38:44.147000 | 127.0.0.3 | 4325 Promising ballot fb282190-685c-11e6-71a2-e0d2d098d5d6 [SharedPool-Worker-3] | 2016-08-22 12:38:44.166000 | 127.0.0.1 | 35282
  • 35. Read LOCAL_SERIAL => LOCAL_QUORUM SERIAL => QUORUM ClinetRequest.CASWrite.conditionNotMet
  • 36. Propose and accept Client partition table promised accepted committed A vouchers 1 Propose 1
  • 37. Propose and accept Client partition table promised accepted committed A vouchers 1 1 Propose 1
  • 38. Propose and accept - rejection Client partition table promised accepted committed A vouchers 2 Propose 1 Rejected - ClientRequest.CASWrite.contentions
  • 39. Propose and accept - trace Sending PAXOS_PROPOSE message to /127.0.0.2 [MessagingService-Outgoing-/127.0.0.2] | 2016-08-22 12:38:44.196000 | 127.0.0.1 | 65606 Sending PAXOS_PROPOSE message to /127.0.0.1 [MessagingService-Outgoing-/127.0.0.1] | 2016-08-22 12:38:44.196000 | 127.0.0.1 | 65606 PAXOS_PROPOSE message received from /127.0.0.1 [MessagingService-Incoming-/127.0.0.1] | 2016-08-22 12:38:44.197000 | 127.0.0.1 | 65986 Sending PAXOS_PROPOSE message to /127.0.0.3 [MessagingService-Outgoing-/127.0.0.3] | 2016-08-22 12:38:44.197000 | 127.0.0.1 | 66139
  • 40. Propose and accept - trace Accepting proposal Commit(fb282190-685c-11e6-71a2-e0d2d098d5d6, [lwts.users] key=chbatey columns=[[] | [email password]]n Row: EMPTY | email=christopher.batey@gmail.com, password=chrisrocks) [SharedPool-Worker-2] | 2016-08-22 12:38:44.199000 | 127.0.0.1 | 67804
  • 41. Commit ● The normal consistency is now used for the commit
  • 42. Consensus for a partition Promised I want the value to be 5, as long as it currently 4 prepare propose Accepted commit Committed Condition met read
  • 43. SERIAL reads o.a.c.s.StorageProxy.readWithPaxos ● For a single partition ● Runs a prepare and ensures all replicas have the latest commit ● Then runs the read at either Q or LQ
  • 48. Setup 4 * i2xLarge RF = 3 10 clients trying to buy 1000 vouchers each - 10k total operations Contention: all clients buying the same voucher (same partitoin) No contention: all clients after different vouchers (different partition)
  • 49. Mutable field CREATE TABLE vouchers_mutable ( name text PRIMARY KEY, sold int ) UPDATE vouchers_mutable SET sold = 1 WHERE name = 'free tv' IF sold = 0; UPDATE vouchers_mutable SET sold = 1 WHERE name = 'free tv'1) 2)
  • 52. Batches CREATE TABLE vouchers ( name text, when timeuuid, sold int static, who text, PRIMARY KEY (name, when) ); BEGIN BATCH UPDATE vouchers SET sold = 1 WHERE name = 'free tv' IF sold = 0 INSERT INTO vouchers (name, when, who) VALUES ( 'free tv', now(), 'charlie') APPLY BATCH ;
  • 55. Summary LWT Batch Contention Incorrect results 99th %ile (milliseconds) N N Y 87% Lost 48 Y N Y 0% Lost 1% Unknown 81% CNM 191 Y N N 0% Lost 0% Unknown 0% CNM 52 Y Y Y 0% Lost <1% Unknown 82% CNM 192
  • 57. Summary ● LWTs are expensive ● They are more complex and less mature than the regular read and write path ● Might be a lot easier than bringing in a second technology