Whether it's statistics, weather forecasting, astronomy, finance, or network management, time series data plays a critical role in analytics and forecasting. Unfortunately, while many tools exist for time series storage and analysis, few are able to scale past memory limits, or provide rich query and analytics capabilities outside what is necessary to produce simple plots; For those challenged by large volumes of data, there is much room for improvement. Apache Cassandra is a fully distributed second-generation database. Cassandra stores data in key-sorted order making it ideal for time series, and its high throughput and linear scalability make it well suited to very large data sets. This talk will cover some of the requirements and challenges of large scale time series storage and analysis. Cassandra data and query modeling for this use-case will be discussed, and Newts, an open source Cassandra-based time series store under development at The OpenNMS Group will be introduced.

1. Time Series Data With
Apache Cassandra
Berlin Buzzwords
May 27, 2014
Eric Evans
eevans@opennms.org
@jericevans

2. Open

3. Open

4. Open

5. Open

6. Network
Management
System

7. OpenNMS: What It Is
● Network Management System
○ Discovery and Provisioning
○ Service monitoring
○ Data collection
○ Event management, notifications
● Java, open source, GPLv3
● Since 1999

8. Time series: RRDTool
● Round Robin Database
● First released 1999
● Time series storage
● File-based, constant-size, self-maintaining
● Automatic, incremental aggregation

9. … and oh yeah, graphing

10. Consider
● 5+ IOPs per update (read-modify-write)!
● 100,000s of metrics, 1,000s IOPS
● 1,000,000s of metrics, 10,000s IOPS
● 15,000 RPM SAS drive, ~175-200 IOPS

12. Hmmm
We collect and write a great deal; We read
(graph) relatively little.
So why are we aggregating everything?

13. Also
● Not everything is a graph
● Inflexible
● Incremental backups impractical
● Availability subject to filesystem access

14. TIL
Metrics typically appear in groups that are
accessed together.
Optimizing storage for grouped access is a
great idea!

15. What OpenNMS needs:
● High throughput
● High availability
● Late aggregation
● Grouped storage/retrieval

16. Cassandra
● Apache top-level project
● Distributed database
● Highly available
● High throughput
● Tunable consistency

17. SSTables
Writes
Commitlog
Memtable
SSTable
Disk
Memory

18. Write Properties
● Optimized for write throughput
● Sorted on disk
● Perfect for time series!

19. Partitioning
A
B
C
Key: Apple
...
AZ

20. Placement
A
B
C
Key: Apple
...

21. Replication
A
B
C
Key: Apple
...

22. CAP Theorem
Consistency
Availability
Partition tolerance

23. Consistency
A
B
?
W=2

24. Consistency
?
B
C
R=2
R+W > N

25. Distribution Properties
● Symmetrical
● Linearly scalable
● Redundant
● Highly available

26. D ata odelM

27. Data Model
resource

28. Data Model
resource
T1 T2 T3

29. Data Model
resource
T1
M1 M2
V1 V2
M3
V3
T2
M1 M2
V1 V2
M3
V3
T3
M1 M2
V1 V2
M3
V3

31. Data Model
CREATE TABLE samples (
T timestamp,
M text,
V double,
resource text,
PRIMARY KEY(resource, T, M)
);

32. Data model
V1T1 M1 V2T1 M2 T1 V3M3resource

33. Data model
SELECT * FROM samples
WHERE resource = ‘resource’
AND T = ‘T1’;
V1T1 M1 V2T1 M2 T1 V3M3resource

34. Data model
T1 M1 V1resource
V1T1 M1 V2T1 M2 T1 V3M3resource

35. Data model
T1 M1 V1
T1 M2 V2
resource
resource
V1T1 M1 V2T1 M2 T1 V3M3resource

36. Data model
T1 M1 V1
T1 M2 V2
T1 M3 V3
resource
resource
resource
V1T1 M1 V2T1 M2 T1 V3M3resource

37. Data model
SELECT * FROM samples
WHERE resource = ‘resource’
AND T >= ‘T1’ AND T <= ‘T3’;
V1T1 M1 V1T2 M1 T3 V1M1resource

38. Newts
● Standalone time series data-store
● Raw sample storage and retrieval
● Flexible aggregations (computed at read)
○ Rate (counter types)
○ Functions pluggable
○ Arbitrary calculations
● Cassandra-speed

39. Newts
● Java API
● REST interface
● Apache licensed
● Github (http://github.com/OpenNMS/newts)

40. Fin