Cassandra 2.0 and timeseries

©2013 DataStax Conﬁdential. Do not distribute without consent.
@PatrickMcFadin
Patrick McFadin
Chief Evangelist/Solution Architect - DataStax
Cassandra 2.0: Intro + Time Series
Friday, October 11, 13

Who I am
2
• Patrick McFadin
• Solution Architect at DataStax
• Cassandra MVP
• User for years
• Follow me for more:
I talk about Cassandra and building scalable, resilient apps ALL THE TIME!
@PatrickMcFadin
Dude.
Uptime == $$

Cassandra - An introduction

Cassandra - Intro
• Based on Amazon Dynamo and Google BigTable paper
• Shared nothing
• Data safe as possible
• Predictable scaling
4
Dynamo
BigTable

Cassandra - More than one server
• All nodes participate in a cluster
• Shared nothing
• Add or remove as needed
• More capacity? Add a server
5
• Each node owns a token
• Tokens denote a range of keys
• 4 nodes? -> Key range/4
• Each node owns 1/4 the data

Cassandra - Locally Distributed
• Client writes to any node
• Node coordinates with others
• Data replicated in parallel
• Replication factor: How many
copies of your data?
• RF = 3 here
6
Each node stores 3/4 of
clusters total data.

Cassandra - Geographically Distributed
• Client writes local
• Data syncs across WAN
• Replication Factor per DC
7
Single coordinator

Cassandra - Consistency
• Consistency Level (CL)
• Client specifies per read or write
8
• ALL = All replicas ack
• QUORUM = > 51% of replicas ack
• LOCAL_QUORUM = > 51% in local DC ack
• ONE = Only one replica acks

Cassandra - Transparent to the application
• A single node failure shouldn’t bring failure
• Replication Factor + Consistency Level = Success
• This example:
• RF = 3
• CL = QUORUM
9
>51% Ack so we are good!

Cassandra Applications - Drivers
• DataStax Drivers for Cassandra
• Java
• C#
• Python
• more on the way
10

Application Example - Layout
• Active-Active
• Service based DNS routing
11
Cassandra Replication

Application Example - Uptime
12
• Normal server maintenance
• Application is unaware
Cassandra Replication

Application Example - Failure
13
• Data center failure
• Data is safe. Route traffic.
33
Another happy user!

Cassandra 2.0 - Big new features

Five Years of Cassandra
Jul-09 May-10 Feb-11 Dec-11 Oct-12 Jul-13
0.1 0.3 0.6 0.7 1.0 1.2
...
2.0
DSE
Jul-08

SELECT * FROM users
WHERE username = ’jbellis’
[empty resultset]
Session 1
SELECT * FROM users
[empty resultset]
Session 2
Lightweight transactions: the problem
INSERT INTO users
(username,password)
VALUES (’jbellis’,‘xdg44hh’)
INSERT INTO users
(userName,password)
VALUES (’jbellis’,‘8dhh43k’)
It’s a Race!
Who wins?

Client
(locks)
Coordinatorrequest
Replica
internal
request
Why Locking Doesn’t Work
• Client locks
• Write times out
• Lock released
• Hint is replayed!!

Client
(locks)
Coordinatorrequest
Replica
internal
request
X
• Client locks
• Write times out
• Lock released

Client
(locks)
Coordinatorrequest
Replica
internal
request
hint
X
• Client locks
• Write times out
• Lock released

Client
(locks)
Coordinatorrequest
Replica
internal
request
hint
timeout
response
X
• Client locks
• Write times out
• Lock released

Paxos
• Consensus algorithm
• All operations are quorum-based
• Each replica sends information about unfinished operations to the leader
during prepare
• Paxos made Simple

LWT: details
• 4 round trips vs 1 for normal updates
• Paxos state is durable
• Immediate consistency with no leader election or failover
• ConsistencyLevel.SERIAL
• http://www.datastax.com/dev/blog/lightweight-transactions-in-
cassandra-2-0

LWT: Use with caution
• Great for 1% of your application
• Eventual consistency is your friend
• http://www.slideshare.net/planetcassandra/c-summit-2013-eventual-consistency-
hopeful-consistency-by-christos-kalantzis

UPDATE USERS
SET email = ’jonathan@datastax.com’, ...
IF email = ’jbellis@datastax.com’;
INSERT INTO USERS (username, email, ...)
VALUES (‘jbellis’, ‘jbellis@datastax.com’, ... )
IF NOT EXISTS;
Using LWT
• Don’t overwrite an existing record
• Only update record if condition is met

CQL Improvements
• Cursors
• Large result sets now have ->next() functionality
• Prevents massive result sets OOMing
• No more client side hacks with LIMIT
• Warning: Not isolated

CQL Improvements
• ALTER DROP
• Remove a field from a CQL table.
• Conditional schema changes
• Only execute if condition met
CREATE KEYSPACE IF NOT EXISTS ks
WITH replication = { 'class': 'SimpleStrategy','replication_factor' :
3 };
CREATE TABLE IF NOT EXISTS test (k int PRIMARY KEY);
DROP KEYSPACE IF EXISTS ks;
ALTER TABLE users DROP address3;

CQL Improvements
• Aliases in SELECT
• Limit and TTL in prepared statements
SELECT event_id, dateOf(created_at) AS creation_date,
blobAsText(content) AS content
FROM timeline;
event_id | creation_date | content
-------------------------+--------------------------+----------------------
550e8400-e29b-41d4-a716 | 2013-07-26 10:44:33+0200 | Something happened!?
SELECT * FROM myTable LIMIT ?;
UPDATE myTable USING TTL ? SET v = 2 WHERE k = 'foo';

Triggers
CREATE TRIGGER <name> ON <table> USING <classname>;
DROP TRIGGER <name> ON [<keyspace>.]<table>;
• Executed on the coordinator before mutation
• Takes original mutation and adds any new
• Jars deployed per server

Trigger implementation
class MyTrigger implements ITrigger
{
public Collection<RowMutation> augment(ByteBuffer key, ColumnFamily update)
{
...
}
}
• You have to implement your own ITrigger (for now)
• Compile and deploy to each server

Experimental!
• Relies on internal RowMutation, ColumnFamily classes
• Not sandboxed. Be careful!
• Expect changes in 2.1

Cassandra and Time Series

Time Series Taming the beast
• Peter Higgs and Francois Englert. Nobel prize for Physics
• Theorized the existence of the Higgs boson
• Found using ATLAS
• Data stored in P-BEAST
• Time series running on Cassandra

Use Cassandra for time series

Use Cassandra for time series
Get a nobel prize

Time Series Why
• Storage model from BigTable is perfect
• One row key and tons of (variable)columns
• Single layout on disk
Row Key Column Name Column Name
Column Value Column Value

Time Series Example
• Storing weather data
• One weather station
• Temperature measurements every minute
WeatherStation ID 2013-10-09 10:00 AM 2013-10-09 10:00 AM 2013-10-10 11:00 AM
72 Degrees 72 Degrees 65 Degrees

Time Series Example
• Query data
• Weather Station ID = Locality of single node
WeatherStation ID
100
2013-10-09 10:00 AM 2013-10-09 10:00 AM 2013-10-10 11:00 AM
72 Degrees 72 Degrees 65 Degrees
Date query weatherStationID = 100 AND
date = 2013-10-09 10:00 AM
weatherStationID = 100 AND
date > 2013-10-09 10:00 AM AND
date < 2013-10-10 11:01 AM
Date Range
OR

Time Series How
• CQL expresses this well
• Data partitioned by weather station ID and time
• Easy to insert data
• Easy to query
CREATE TABLE temperature (
weatherstation_id text,
event_time timestamp,
temperature text,
PRIMARY KEY (weatherstation_id,event_time)
);
INSERT INTO temperature(weatherstation_id,event_time,temperature)
VALUES ('1234ABCD','2013-04-03 07:01:00','72F');
SELECT temperature
FROM temperature
WHERE weatherstation_id='1234ABCD'
AND event_time > '2013-04-03 07:01:00'
AND event_time < '2013-04-03 07:04:00';

Time Series Further partitioning
• At every minute you will eventually run out of rows
• 2 billion columns per storage row
• Data partitioned by weather station ID and time
• Use the partition key to split things up
CREATE TABLE temperature_by_day (
weatherstation_id text,
date text,
event_time timestamp,
temperature text,
PRIMARY KEY ((weatherstation_id,date),event_time)
);

Time Series Further Partitioning
• Still easy to insert
• Still easy to query
INSERT INTO temperature_by_day(weatherstation_id,date,event_time,temperature)
VALUES ('1234ABCD','2013-04-03','2013-04-03 07:01:00','72F');
SELECT temperature
FROM temperature_by_day
WHERE weatherstation_id='1234ABCD'
AND date='2013-04-03'
AND event_time > '2013-04-03 07:01:00'
AND event_time < '2013-04-03 07:04:00';

Time Series Use cases
• Logging
• Thing Tracking (IoT)
• Sensor Data
• User Tracking
• Fraud Detection
• Nobel prizes!

Thank you!
Apache Cassandra 2.0 - Data model on ﬁre
Next talk in my data model series!

Cassandra 2.0 and timeseries

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Cassandra 2.0 and timeseries

Ähnlich wie Cassandra 2.0 and timeseries (20)

Mehr von Patrick McFadin

Mehr von Patrick McFadin (6)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Cassandra 2.0 and timeseries