Product comparison of services that consume CloudWatch data and create dashboards. A talk at the AWS Melbourne user group February 2014.

1. Which watcher watches
cloudwatch?
David Lutz
@dlutzy

2. about me
sysadmin

3. about me
sysadmin
devops

4. about me
sysadmin
devops
operations

5. about me
sysadmin
devops
web operations

6. about me
sysadmin
devops
web operations

7. about me
#monitoringlove
#infracoders
I “drive” website
infrastructure

8. What do I mean by “drive”?
Operate, anything not build related.
● Gather telemetry or metrics on running system
● Use metrics to make decisions about capacity and
architecture
● Know what the bottlenecks are and when we’ll hit
them
● Advise the business about cost vs performance
tradeoffs
● Fix things when the break

9. Work with my friends the
developers…
They build the things I drive.

10. What the developers think they built

11. What they actually built

12. How operations see themselves

13. How dev sees ops

14. The Perfect Dashboard

15. How fast are we
going?
(km/h)

16. How hard is the
engine working
(RPM)

17. Is anything
broken?
(alerts)

18. How fast can we
go before redline
How far can we go
before running
out of fuel
(Thresholds)

19. How fast are we going?
latency (ms)

20. How hard are the servers working?
throughput (rpm)

21. Is
anything
broken?

22. What about thresholds?
You must load test to find the
breaking points and bottlenecks
yourself.
Load testing is hard.

23. AWS CloudWatch pros and cons.
Comparison of three SAAS products
that consume CloudWatch metrics.
NewRelic
Stackdriver
Librato

24. CloudWatch limitations

25. CloudWatch limitations...
●
●
●
●
two week data retention
can’t create custom dashboards
UI is better than it used to be…
can’t time shift metrics to compare last
week to this week

26. However in CloudWatch’s defense
●
●
●
●
good for ad hoc exploration of data
most up to date
can set thresholds and alert on them
can push custom metrics (does anyone use
this?)

30. Setting up access
Create IAM group and
user
Give Access Key and
Secret Key to Librato

31. Setting up access
Create IAM group and user
Launch t1.micro AMI from
marketplace (or run it on your
own server)
Configure and run Sync process
(is a ruby gem)

32. Setting up access
Create role for
“cross account
access” for “3rd
party”, “read only”
Give stackdriver the
Role ARN.

33. Data retention
NewRelic
Stackdriver
Librato
3 months
13 months
(Elite tier)
12 months
(rolled up to
1 hour
resolution)

34. Features
NewRelic
Stackdriver
Librato
Custom dashboards
✓
✓
✓
Real time update
✓
✓
✓
✓
Metric lag time display
Time shift overlay
✓
Server agent
✓
Mobile client
✓
Custom metrics
✓
✓
✓
Events (like deployments)
✓
✓
✓
✓

35. Key benefits IMHO
NewRelic
Stackdriver
integration
with existing
system
easily define and integration
clusters and
with existing
aggregate
system
open up access
to those who
wouldn’t
otherwise have
access
imports
CloudTrail data,
who did what
when?
anomaly detection
Librato
flexibility of
data
collection and
retention

36. Costs (per month)
NewRelic
Stackdriver
Librato
Free if you
already use
NewRelic
$12 per resource
(Elite tier)
AWS Billing $0.60
DynamoDB $1.10
EBS $0.50
EC2 $0.50
ELB $0.65
ElastiCache $1.95
EMR $1.30
OpsWorks $0.75
RDS $0.70
Redshift $0.75
Route53 $0.05
SNS $0.20
SQS $0.40
...
But you need a
server to run
the collector
(per instance)

41. Say g’day to me
@dlutzy