Data Science Lab Meetup: Cassandra and Spark

1. @chbatey
Christopher Batey
Technical Evangelist for Apache Cassandra
Time series analysis with Spark and
Cassandra

2. @chbatey
Who am I?
• Technical Evangelist for Apache Cassandra
•Founder of Stubbed Cassandra
•Help out Apache Cassandra users
• DataStax
•Builds enterprise ready version of Apache
Cassandra
• Previous: Cassandra backed apps at BSkyB

3. @chbatey
Agenda
• Motivation
• Cassandra
• Replication
• Fault tolerance
• Data modelling
• Spark
• Use cases
• Stream processing
• Time series example: Weather station data

4. @chbatey
OLTP OLAP Batch

5. Weather data streaming
Incoming
weather
events
Apache Kafka
Producer
Consumer
NodeGuardian
Dashboard

6. @chbatey

7. @chbatey

8. @chbatey
Run this your self
• https://github.com/killrweather/killrweather

9. @chbatey
Cassandra

10. @chbatey
Cassandra for Applications
APACHE
CASSANDRA

11. @chbatey
Common use cases
•Ordered data such as time series
-Event stores
-Financial transactions
-IoT e.g Sensor data

12. @chbatey
Common use cases
•Ordered data such as time series
-Event stores
-Financial transactions
-IoT e.g Sensor data
•Non functional requirements:
-Linear scalability
-High throughout durable writes
-Multi datacenter including active-active
-Analytics without ETL

13. @chbatey
Cassandra
Cassandra
• Distributed masterless
database (Dynamo)
• Column family data model
(Google BigTable)

14. @chbatey
Datacenter and rack aware
Europe
• Distributed master less
database (Dynamo)
• Column family data model
(Google BigTable)
• Multi data centre replication
built in from the start
USA

15. @chbatey
Cassandra
Online
• Distributed master less
database (Dynamo)
• Column family data model
(Google BigTable)
• Multi data centre replication
built in from the start
• Analytics with Apache SparkAnalytics

16. @chbatey
Dynamo 101

17. @chbatey
Dynamo 101
• The parts Cassandra took
- Consistent hashing
- Replication
- Gossip
- Hinted handoff
- Anti-entropy repair
• And the parts it left behind
- Key/Value
- Vector clocks

18. @chbatey
Picking the right nodes
• You don’t want a full table scan on a 1000 node cluster!
• Dynamo to the rescue: Consistent Hashing

19. @chbatey
Murmer3 Example
• Data:
• Murmer3 Hash Values:
jim age: 36 car: ford gender: M
carol age: 37 car: bmw gender: F
johnny age: 12 gender: M
suzy: age: 10 gender: F
Primary Key Murmur3 hash value
jim 350
carol 998
johnny 50
suzy 600
Primary Key
Real hash range: -9223372036854775808 to 9223372036854775807

20. @chbatey
Murmer3 Example
Four node cluster:
Node Murmur3 start range Murmur3 end range
A 0 249
B 250 499
C 500 749
D 750 999

21. @chbatey
Pictures are better
A
B
C
D
999
249
499
750
749
0
250
500
B
CD
A

22. @chbatey
Murmer3 Example
Data is distributed as:
Node Start range End range Primary
key
Hash value
A 0 249 johnny 50
B 250 499 jim 350
C 500 749 suzy 600
D 750 999 carol 998

23. @chbatey
Replication

24. @chbatey
Replication strategy
• Simple
- Give it to the next node in the ring
- Don’t use this in production
• NetworkTopology
- Every Cassandra node knows its DC and Rack
- Replicas won’t be put on the same rack unless Replication Factor > # of racks
- Unfortunately Cassandra can’t create servers and racks on the fly to fix this :(

25. @chbatey
Replication
DC1 DC2
client
RF3 RF3
C
RC
WRITE
CL = 1 We have replication!

26. 26

27. @chbatey
Tunable Consistency
•Data is replicated N times
•Every query that you execute you give a consistency
-ALL
-QUORUM
-LOCAL_QUORUM
-ONE
• Christos Kalantzis Eventual Consistency != Hopeful Consistency: http://
youtu.be/A6qzx_HE3EU?list=PLqcm6qE9lgKJzVvwHprow9h7KMpb5hcUU

28. @chbatey
Scaling shouldn’t be hard
• Throw more nodes at a cluster
• Bootstrapping + joining the ring
• For large data sets this can take some time

29. @chbatey
Spark Time

30. @chbatey
Scalability & Performance
• Scalability
- No single point of failure
- No special nodes that become the bottle neck
- Work/data can be re-distributed
• Operational Performance i.e single digit ms
- Single node for query
- Single disk seek per query

31. @chbatey
But but…
• Sometimes you don’t need a answers in milliseconds
• Reports / analysis
• Data models done wrong - how do I fix it?
• New requirements for old data?
• Ad-hoc operational queries
• Managers always want counts / maxs

32. @chbatey

33. @chbatey
Apache Spark
• 10x faster on disk,100x faster in memory than Hadoop
MR
• Works out of the box on EMR
• Fault tolerant distributed datasets
• Batch, iterative and streaming analysis
• In memory storage and disk
• Integrates with most file and storage options

34. @chbatey
Part of most Big Data Platforms
Analytic
Search
• All Major Hadoop Distributions Include
Spark
• Spark Is Also Integrated With Non-
Hadoop Big Data Platforms like DSE
• Spark Applications Can Be Written Once
and Deployed Anywhere
SQL
Machine
Learning
Streaming Graph
Core
Deploy Spark Apps Anywhere

35. @chbatey
Components
Shark
or
Spark SQL
Streaming ML
Spark (General execution engine)
Graph
Cassandra
Compatible

36. @chbatey
org.apache.spark.rdd.RDD
• Resilient Distributed Dataset (RDD)
• Created through transformations on data (map,filter..) or other RDDs
• Immutable
• Partitioned
• Reusable

37. @chbatey
RDD Operations
• Transformations - Similar to Scala collections API
• Produce new RDDs
• filter, flatmap, map, distinct, groupBy, union, zip, reduceByKey, subtract
• Actions
• Require materialization of the records to generate a value
• collect: Array[T], count, fold, reduce..

38. @chbatey
Word count
val file: RDD[String] = sc.textFile("hdfs://...")
val counts: RDD[(String, Int)] = file.flatMap(line =>
line.split(" "))
.map(word => (word, 1))
.reduceByKey(_ + _)
counts.saveAsTextFile("hdfs://...")

39. zillions of bytes gigabytes per second
Spark Versus Spark Streaming

40. DStream - Micro Batches
μBatch (ordinary RDD) μBatch (ordinary RDD) μBatch (ordinary RDD)
Processing of DStream = Processing of μBatches, RDDs
DStream
• Continuous sequence of micro batches
• More complex processing models are possible with less effort
• Streaming computations as a series of deterministic batch
computations on small time intervals

41. @chbatey
Time series example time

42. @chbatey
Deployment
• Spark worker in each of the
Cassandra nodes
• Partitions made up of LOCAL
cassandra data
S C
S C
S C
S C

43. Weather Station Analysis
• Weather station collects data
• Cassandra stores in sequence
• Spark rolls up data into new
tables
Windsor California
July 1, 2014
High: 73.4F
Low : 51.4F

44. raw_weather_data
CREATE TABLE raw_weather_data (
weather_station text, // Composite of Air Force Datsav3 station number and NCDC WBAN numbe
year int, // Year collected
month int, // Month collected
day int, // Day collected
hour int, // Hour collected
temperature double, // Air temperature (degrees Celsius)
dewpoint double, // Dew point temperature (degrees Celsius)
pressure double, // Sea level pressure (hectopascals)
wind_direction int, // Wind direction in degrees. 0-359
wind_speed double, // Wind speed (meters per second)
sky_condition int, // Total cloud cover (coded, see format documentation)
sky_condition_text text, // Non-coded sky conditions
one_hour_precip double, // One-hour accumulated liquid precipitation (millimeters)
six_hour_precip double, // Six-hour accumulated liquid precipitation (millimeters)
PRIMARY KEY ((weather_station), year, month, day, hour)
) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC, hour DESC);
Reverses data in the storage engine.

45. Primary key relationship
PRIMARY KEY (weatherstation_id,year,month,day,hour)

46. Primary key relationship
PRIMARY KEY (weatherstation_id,year,month,day,hour)
Partition Key

47. Primary key relationship
PRIMARY KEY (weatherstation_id,year,month,day,hour)
Partition Key Clustering Columns

48. Primary key relationship
PRIMARY KEY (weatherstation_id,year,month,day,hour)
Partition Key Clustering Columns
10010:99999

49. 2005:12:1:7
-5.6
Primary key relationship
PRIMARY KEY (weatherstation_id,year,month,day,hour)
Partition Key Clustering Columns
10010:99999
-5.3-4.9-5.1
2005:12:1:8 2005:12:1:9 2005:12:1:10

50. Data Locality
weatherstation_id=‘10010:99999’ ?
1000 Node Cluster
You are here!

51. Query patterns
• Range queries
• “Slice” operation on disk
SELECT weatherstation,hour,temperature
FROM raw_weather_data
WHERE weatherstation_id=‘10010:99999'
AND year = 2005 AND month = 12 AND day = 1
AND hour >= 7 AND hour <= 10;
Single seek on disk
2005:12:1:12
-5.4
2005:12:1:11
-4.9-5.3-4.9-5.1
2005:12:1:7
-5.6
2005:12:1:8 2005:12:1:9
10010:99999
2005:12:1:10
Partition key for locality

52. Query patterns
• Range queries
• “Slice” operation on disk
Programmers like this
Sorted by event_time
2005:12:1:7
-5.6
2005:12:1:8
-5.1
2005:12:1:9
-4.9
10010:99999
10010:99999
10010:99999
weather_station hour temperature
2005:12:1:10
-5.3
10010:99999
SELECT weatherstation,hour,temperature
FROM raw_weather_data
WHERE weatherstation_id=‘10010:99999'
AND year = 2005 AND month = 12 AND day = 1
AND hour >= 7 AND hour <= 10;

53. weather_station
CREATE TABLE weather_station (
id text PRIMARY KEY, // Composite of Air Force Datsav3 station number and NCDC WBAN number
name text, // Name of reporting station
country_code text, // 2 letter ISO Country ID
state_code text, // 2 letter state code for US stations
call_sign text, // International station call sign
lat double, // Latitude in decimal degrees
long double, // Longitude in decimal degrees
elevation double // Elevation in meters
);
Lookup table

54. daily_aggregate_temperature
CREATE TABLE daily_aggregate_temperature (
weather_station text,
year int,
month int,
day int,
high double,
low double,
mean double,
variance double,
stdev double,
PRIMARY KEY ((weather_station), year, month, day)
) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC);
SELECT high, low FROM daily_aggregate_temperature
WHERE weather_station='010010:99999'
AND year=2005 AND month=12 AND day=3;
high | low
------+------
1.8 | -1.5

55. daily_aggregate_precip
CREATE TABLE daily_aggregate_precip (
weather_station text,
year int,
month int,
day int,
precipitation counter,
PRIMARY KEY ((weather_station), year, month, day)
) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC);
SELECT precipitation FROM daily_aggregate_precip
WHERE weather_station='010010:99999'
AND year=2005 AND month=12 AND day>=1 AND day <= 7;
0
10
20
30
40
1 2 3 4 5 6 7
17
26
2
0
33
12
0

56. Result
wsid | year | month | day | high | low
--------------+------+-------+-----+------+------
725300:94846 | 2012 | 9 | 30 | 18.9 | 10.6
725300:94846 | 2012 | 9 | 29 | 25.6 | 9.4
725300:94846 | 2012 | 9 | 28 | 19.4 | 11.7
725300:94846 | 2012 | 9 | 27 | 17.8 | 7.8
725300:94846 | 2012 | 9 | 26 | 22.2 | 13.3
725300:94846 | 2012 | 9 | 25 | 25 | 11.1
725300:94846 | 2012 | 9 | 24 | 21.1 | 4.4
725300:94846 | 2012 | 9 | 23 | 15.6 | 5
725300:94846 | 2012 | 9 | 22 | 15 | 7.2
725300:94846 | 2012 | 9 | 21 | 18.3 | 9.4
725300:94846 | 2012 | 9 | 20 | 21.7 | 11.7
725300:94846 | 2012 | 9 | 19 | 22.8 | 5.6
725300:94846 | 2012 | 9 | 18 | 17.2 | 9.4
725300:94846 | 2012 | 9 | 17 | 25 | 12.8
725300:94846 | 2012 | 9 | 16 | 25 | 10.6
725300:94846 | 2012 | 9 | 15 | 26.1 | 11.1
725300:94846 | 2012 | 9 | 14 | 23.9 | 11.1
725300:94846 | 2012 | 9 | 13 | 26.7 | 13.3
725300:94846 | 2012 | 9 | 12 | 29.4 | 17.2
725300:94846 | 2012 | 9 | 11 | 28.3 | 11.7
725300:94846 | 2012 | 9 | 10 | 23.9 | 12.2
725300:94846 | 2012 | 9 | 9 | 21.7 | 12.8
725300:94846 | 2012 | 9 | 8 | 22.2 | 12.8
725300:94846 | 2012 | 9 | 7 | 25.6 | 18.9
725300:94846 | 2012 | 9 | 6 | 30 | 20.6
725300:94846 | 2012 | 9 | 5 | 30 | 17.8
725300:94846 | 2012 | 9 | 4 | 32.2 | 21.7
725300:94846 | 2012 | 9 | 3 | 30.6 | 21.7
725300:94846 | 2012 | 9 | 2 | 27.2 | 21.7
725300:94846 | 2012 | 9 | 1 | 27.2 | 21.7
SELECT wsid, year, month, day, high, low
FROM daily_aggregate_temperature
WHERE wsid = '725300:94846'
AND year=2012 AND month=9 ;

57. Weather Station Stream Analysis
• Weather station collects data
• Data processed in stream
• Data stored in Cassandra
Windsor California
Today
Rainfall total: 1.2cm
High: 73.4F
Low : 51.4F

58. Incoming data from Kafka
725030:14732,2008,01,01,00,5.0,-3.9,1020.4,270,4.6,2,0.0,0.0

59. @chbatey
Creating a Stream

60. @chbatey
Saving the raw data

61. @chbatey
Building an aggregate
CREATE TABLE daily_aggregate_precip (
weather_station text,
year int,
month int,
day int,
precipitation counter,
PRIMARY KEY ((weather_station), year, month, day)
) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC);
CQL Counter

62. Weather data streaming
Load
Generator or
Data import
Apache Kafka
Producer
Consumer
NodeGuardian
Dashboard

63. @chbatey

64. @chbatey

65. @chbatey
Run this your self
• https://github.com/killrweather/killrweather

66. @chbatey
Summary
• Cassandra
- always-on operational database
• Spark
- Batch analytics
- Stream processing and saving back to Cassandra

67. @chbatey
Thanks for listening
• Follow me on twitter @chbatey
• Cassandra + Fault tolerance posts a plenty:
• http://christopher-batey.blogspot.co.uk/
• Cassandra resources: http://planetcassandra.org/
• Full free day of Cassandra talks/training:
• http://www.eventbrite.com/e/cassandra-day-london-2015-
april-22nd-2015-tickets-15053026006?aff=meetup1