Presented at Cassandra EU on 28 March 2012

1. A Cassandra Usage Example
Gary Dusbabek
@gdusbabek

2. Goals
CM Overview
Control Cluster
Data Cluster

3. CM Overview
Thousands of servers
Pre-existing solutions
Lessons learned from
Cloudkick
Internal versus external
Millions of checks

4. Terminology
Entity
Something with an IP address or
host name

5. Terminology
Check
Tied to a entity
Is an action
Produces metrics

6. Terminology
Alarm
Tied to an entity
Evaluates check output
Specified by DSL

7. Terminology
Notification Plan
Determines what to do for various
alarm states

8. Features
Remote checks
Five datacenters
collectors in five
processing in three
Alerting

9. 1,000 words

10. All REST all
the time
More Ops friendly
Features Agent
Metrics
https://github.com/racker/virgo

11. Future
Automation
Prediction
Support hooks
Agent expansion
Correlation
Aggregation
Entity Spanning

12. Two Clusters
Data
Control

13. Control
Cluster
Metadata
State
Three datacenters
High RF
Wide rows
Easy dump & load
https://github.com/racker/cassandra-syncer

14. Data Model
Rich but simple
Objects used together stored together
Simple parent-child relations
One row per customer (tenant)
Composite column names

15. Data Model
Good: Bad:
Single Parent/Child Complex
Acyclic Multiple Parent/Child

16. Data
Model
Object types:
Foo, Bar, Baz
Relationship:
1:*

17. Data Model
New Foo: 123
Row
Column
Value
Key Column
Name

18. Data Model
New Foo: 124

19. Data Model
New Bar attached to fo123.
ID is “fo123” ID is “fo123:ba001”

20. Data Model
New Bar attached to fo123.

21. Data Model
New Bar attached to fo124.

22. Data Model
New Bar attached to fo123.
ID is “fo123:ba003”

23. Data Model
Baz attached to ba003
ID is “fo123:ba001:bz001”

24. Data Model
Select Foo children for 123:
select „fo123‟..‟fo123:x7f‟;
Select single Bar object attached to fo123:
select „fo123:ba001‟..‟fo123:ba001:x7f‟;
Object hierarchy reduced to a single slice.
Parent keys available from context (no extra
reads or indexes)

25. Control Cluster
API server is Node.js
Javascript ORM library
• Define object model in JS
• Read/write entire objects
• Never think about CQL
node-cassandra-client
https://github.com/racker/node-cassandra-client

26. Control Cluster

27. Data
Cluster
The fun
starts
here

28. Data Cluster
Goal: Fast graphs
Time series data
Fewer data points
OK to shave resolution
Recent data is most important

30. Granularity
Full, 5m, 20m, 80m, 320m, 1280m
time-to-live (TTL)

31. Rollup Concepts
Slot
Pegged at 4096 slots
Circular buffer of metrics names that
have come into the system
Key is granularity + slot
Columns index keys in rollup table

32. Keyed by ascii
Bigint column names
Blob column values
JDBC
Rollups

33. Arrival
Full Resolution!
time, name, several metrics
metric = name, type, value
Create a locator assigned to slot
Insert metrics
Single Cassandra APPLY BATCH;

34. Process: Rolling up
• rollup(src_gran, from, to);
• from, to determine slots
• For each slot, get locator row slice
– Cols tell us which metrics are active (need roll up)
– For each column in locator row slice, slice in data
row to get metrics
• For each column (metric) in slice, add to calculations
• Store rolled up calculation in dst_gran CF
• Store locator column for slot, dst_gran
• Also rollup(metric, src_gran, from, to);

35. Rollup operations are idempotent*
Simplifies availability
Rollups are easily parallelized
Partition the locator space

36. But…
What if data arrives after rollup is
performed?
More than 24hrs late: don‟t care,
forget it
Figure out which slot the data
belongs it, flag for each
granularity in a meta row
Recalculate rollups

37. http://www.flickr.com/photos/nateone/3768979925
/
HBase
@gdusbabek

38. Image Credits
soccer http://www.flickr.com/photos/ollesvensson/4252196844/
apples http://www.flickr.com/photos/jean_koulev/2697677595/
clock http://www.flickr.com/photos/ollesvensson/4252196844/
corridor http://www.flickr.com/photos/orinrobertjohn/4354716077/
car http://www.flickr.com/photos/ellenm1/3541270451
controller http://www.flickr.com/photos/germanium/117612088/
model http://www.flickr.com/photos/mikeschinkel/2703438152/
objects 1 http://www.flickr.com/photos/radder86/389283265/
objects 2 http://www.flickr.com/photos/radder86/389283262
robot http://www.flickr.com/photos/kb35/430976324/
fun http://www.flickr.com/photos/pierrebedat/1095337445/
blobs http://www.flickr.com/photos/inl/5097547405/
dish http://www.flickr.com/photos/mrpbps/2862208028/
sand http://www.flickr.com/photos/backkratze/3480338854/
slot http://www.flickr.com/photos/andresrueda/2925383781
sushi http://www.flickr.com/photos/basykes/4348613931/
airplane http://www.flickr.com/photos/fhashemi/72489620/
scale http://www.flickr.com/photos/puuikibeach/4765115333
hourglass http://www.flickr.com/photos/aidanmorgan/2331754875/
street http://www.flickr.com/photos/nateone/3768979925/