OpenNMS User Conference Europe presentation on using Apache Cassandra and Newts for time-series data storage.

1. It’s Not You...
Ending a 15 year relationship with RRD
OpenNMS User Conference Europe
April 11, 2014
Eric Evans
eevans@opennms.org
@jericevans

2. Five Stages of Grief
1. Denial
2. Anger
3. Bargaining
4. Depression
5. Acceptance

4. RRDTool
● Round robin database
● First released 1999
● Time-series storage
● File-based
● Constant-size
● Automatic, amortized aggregation

5. Graph All The Things

6. Consider
● 2 5+ IOPs per update (read-modify-write)!
● 1 RRD per data source (storeByGroup=false)
● 100,000s of data sources, 1,000s IOPS
● 1,000,000s of data sources, 10,000s IOPS
● 15,000 RPM SAS drive, ~175-200 IOPS

7. Also
● Not everything is a graph
● Inflexible
● Incremental backups impractical
● ...

9. Bottleneck!?
● We need to be collecting even more!
● We need to be collecting more frequently!
● The Internet of Things is upon us!!

11. But can’t we … ?
● Serialize RRD writes?
● Cache?
● Distribute the RRDs?
● … ?

14. How about:
● Distributed, decoupled architecture
● High throughput
● Horizontally scalable
● Pluggable, extensible graphing
● Facilitate new forms of analytics
● More?
Starting Over Can Be Fun!

15. Observation #1
We collect and write a great deal; We read
(graph) relatively little.
We are read-optimized.

16. Observation #2
Grouping samples that are accessed together
is an easy optimization.

17. Project: Newts
Goals:
● Stand-alone time-series data store
● High-throughput
● Horizontally scalable
● Grouped metric storage/retrieval
● Late-aggregating

18. Cassandra
● Apache top-level project
● Distributed database
● Tunable consistency
● NoSQL MoSQL

19. Partitioning
A
B
C
Key: Apple
...

20. Placement
A
B
C
Key: Apple
...

21. Replication
A
B
C
Key: Apple
...

22. Cap Theorem
Consistency
Availability
Partition tolerance

23. Consistency
A
?
?
W=1

24. Consistency
A
B
C
R=3

25. Consistency
A
B
?
W=2

26. Consistency
?
B
C
R=2
R+W > N

27. Properties
● Symmetrical
● Linearly scalable
● Redundant
● Highly available

28. SSTables
Writes
Commitlog
Memtable
SSTable
Disk
Memory

29. Properties
● Optimized for write throughput
● Sorted on disk
● Perfect for time series!

30. Gist
● Samples stored as-is.
● Samples can be retrieved as-is.
● Measurements are aggregations calculated
from samples (at time of query).

31. Samples vs. Measurements
sam·ple
/ˈsampəl/
noun
1. a small part or quantity intended to show what the whole is like.
"investigations involved analyzing samples of handwriting"
synonyms: representative, illustrative, selected, specimen, test, trial, typical
meas·ure·ment
/ˈmeZHərmənt/
noun
1. the action of measuring something.
"accurate measurement is essential"
synonyms: quantification, computation, calculation, mensuration

32. Sample
{
‘timestamp’: 1395278097,
‘resource’ : ‘localhost.eth0’,
‘name’ : ‘ifInOctets’,
‘type’ : ‘COUNTER’,
‘value’ : 457283782231
}

33. Newts
CREATE TABLE newts.samples (
resource text,
collected_at timestamp,
metric_name text,
metric_type text,
value blob,
attributes map<text, text>,
PRIMARY KEY(resource, collected_at, metric_name)
);

34. Behind the scenes...
KSAT (1970-02-10 12:42:00,dewPoint,value):
0xc01a0000
(1970-02-10 12:42:00,maxTemp,value):
0x40280000
...
Ascending Order

35. Newts
resource | collected_at | metric_name | value
---------+---------------------+--------------+-----------
KSAT | 1970-02-10 12:42:00 | dewPoint | 0xc01a0000
KSAT | 1970-02-10 12:42:00 | maxTemp | 0x40280000
KSAT | 1970-02-10 12:42:00 | maxWindGust | 0x7ff80000
KSAT | 1970-02-10 12:42:00 | maxWindSpeed | 0x40180000
KSAT | 1970-02-10 12:42:00 | meanTemp | 0xbfe00000

36. POST
POST /samples HTTP/1.1
Host: example.com
Content-Type: application/json
[
{
‘timestamp’: 1395278097,
‘resource’ : ‘localhost.eth0’,
‘name’ : ‘ifInOctets’,
‘type’ : ‘COUNTER’,
‘value’ : 457283782231
},
{
...
},
]

37. GET samples
GET /samples/localhost.eth0?start=900000000 HTTP/1.1
[
[
{“name”: “ifInOctets”, “timestamp”: 900000000, “type”: “COUNTER”,
“value”: 12345678900},
{“name”: “ifOutOctets”, “timestamp”: 900000000, “type”: “COUNTER”,
“value”: 87654321000}
],
[
{“name”: “ifInOctets”, “timestamp”: 900000300, “type”: “COUNTER”,
“value”: 23456789000},
{“name”: “ifOutOctets”, “timestamp”: 900000300, “type”: “COUNTER”,
“value”: 98765432100}
]
]

38. GET measurements
GET /measurements/octets/localhost.eth0?resolution=20m HTTP/1.1
[
[
{“name”: “ifInOctets”, “timestamp”: 900000000, “value”: 102400.00},
{“name”: “ifOutOctets”, “timestamp”: 900000000, “value”: 409600.00}
],
[
{“name”: “ifInOctets”, “timestamp”: 900001200, “value”: 102400.00},
{“name”: “ifOutOctets”, “timestamp”: 900001200, “value”: 409600.00}
]
]

39. http://github.com/OpenNMS/newts