An introduction to graphite and why it's so great.

1. Graphite:
An Introduction
Scaling real-time monitoring

2. The purpose today

3. What is graphite

4. Why it’s so great

5. How to graph
(It’s really easy!)

6. How we use graphite

7. First, a definition

8. Alerts+Metrics=Monitoring
Graphite
Cacti
Munin
Nagios
Icinga
Both
Zenoss
Hyperic
Zabbix
PNP4Nagios
Alerting
Metrics

9. What is graphite

12. About graphite
● Django web application consisting of 3 parts:
○ carbon (relays, caches, aggregates metrics)
○ whisper (graphite’s equivalent of RRD files)
○ Web UI (graph composer, simple dashboard)

13. Why graphite?

14. Why graphing?
Discover trends and patterns
What time of the day do we get the most users?
When x happened, what was the effect on y?
How many hits am I getting per hour?
How does this compare to last week? last month?
Predict future events
When will we need to add more servers? Databases?
Negative feedback
Did the release into production fix problem x?

15. Cacti SUCKS
A few reasons:
Ancient user interface (no javascript/ajax), terrible workflow, cannot push metrics, no
formulas, no graph introspection, cannot push metrics, cannot feed out of sequence
metrics, ugly graphs, no API, expose system/os metrics on host via snmp, no graph
composer, no custom graphs, predefine metrics, predefine graphs, static polling interval,
unscalable, tons of work to create one graph, no 3rd party ecosystem, etc.

16. Graphite ++

17. Simple

18. Powerful

19. Functions
(sum, derivatives, integrals, timeshift, mostDeviant, scale,
averages, etc.)

20. API
(Nagios integration, 3rd party custom dashboards)

22. Scalable

23. Easy to feed data

24. Wide ecosystem of 3rd party
tools and dashboards
http://graphite.readthedocs.org/en/latest/tools.html

25. Tools

26. StatsD

27. Logster

28. Skyline

29. Collectd

30. Dashboards

36. Graphite --

37. No poller

38. No all in one solution

39. No easy backups

40. It probably will become
business critical

41. How to graph

42. There are tons of ways to
feed graphite your data

43. Bash
#!/bin/bash
timestamp = `date +%s`
value = 10
echo "dot.delimited.metric.name $value $timestamp" | nc -w 1 graphite.
host.name 2003
Python
def send_msg(message, HOST, PORT):
sock = socket.create_connection((HOST, PORT))
sock.send(message)
sock.close()
Python using graphite-pymetrics
from metrics import timing
@timing("heavy.task")
def heavy_task( x, y, z):
# do heavy stuff here

44. Ruby
require 'socket'
Host = 'somegraphitehost'
conn = TCPSocket.new Host, 2003
conn.puts 'Metrics value timestamp'
conn.close
Java
import java.io.DataOutputStream;
import java.net.Socket;
Socket conn = new Socket("somegraphitehost" , 2003);
DataOutputStream dos = new DataOutputStream(conn .getOutputStream());
dos.writeBytes("metrics value timestamp" );
conn.close();

45. How we use graphite

46. 700K + metrics per minute

47. A Common Graphite Stack
Graphite-web
Collectd
Poller(s)
Applications
Carbon Whisper
Dashboards
Statsd
Scripts
Nagios

48. Collectd
Agent for system/hardware level metrics
Growing repository of plugins for a wide variety
of applications:
disk i/o, disk space, cpu, memory, mysql,
JMX, java, Redis, file sizes, load, etc.
https://collectd.org/wiki/index.php/Table_of_Plugins
Write your custom plugin in python

49. Nagios integration
You can write Nagios plugins that can alert off
of metrics values
Nagios can also feed graphite
performance data, events (ie: update
counter each time email is sent), etc.

50. What to collect?

51. Hardware/OS metrics

52. Load

53. Disk space

54. Disk I/O

55. Network data

56. Application metrics

57. How often function x is called

58. Average value of function x

59. Average running time of
function x

60. Database/Datastore

61. performance metrics

62. number of records with
value == ?

63. number of slow queries

64. Events

65. Deployments

66. send a 1, draw as infinite

67. Log files

68. http access logs
(2xx, 3xx, 4xx, 5xx)

69. Application logs
Exception counts, results, important events, hits

70. Final Musings

71. Treat graphite like ‘Big Data’

72. You don’t know what metrics
you need until you need it

73. Get Raid 10 SSD’s once you
decide to scale

74. More devopsy

75. You can start graphing today!