Slides from my lightning talk about our Celery Prometheus Exporter at Pycon X in Florence

1. PROMETHEUS
CELERY EXPORTER
Fabio Todaro
Head of DevOps/IT @ OvalMoney
PyconX Lightning Talk- 4/5/2019
Florence, IT

2. WHAT IS CELERY?
• Celery is an asynchronous task
queue/job queue based on
distributed message passing.
• It is focused on real-time operation,
but supports scheduling as well.
• The execution units, called tasks,
are executed concurrently on a
single or more worker servers.
• Tasks can execute asynchronously
(in the background) or
synchronously (wait until ready).

3. HOW DO WE USE CELERY

4. WHATTO MONITOR?
• Tasks Sent (Pending) rate
• Tasks Received rate
• Tasks Started rate
• Task Success/Failure rate
• Tasks Retry/Revoke
• Tasks Runtime
• Tasks Latency
• Tasks Queue

5. HOWTO MONITOR IT?
We already use Prometheus for IT monitoring and searched for
a celery exporter:
zerok/celery-prometheus-exporter
ISSUE
• Uses celery.events.state.State that leaks memory
• Doesn’t respect Prometheus naming standards
• Doesn’t export information about queues

6. OVALMONEY/CELERY-EXPORTER
https://github.com/OvalMoney/celery-exporter
https://grafana.com/dashboards/10026

10. HOW DOES IT WORK?

11. FabioTodaro
https://www.linkedin.com/in/sharpedge/
https://github.com/SharpEdgeMarshall
OvalMoney
www.ovalmoney.com
https://github.com/OvalMoney
Thank you.