Prometheus - basics slides

1. Cloud-Native Monitoring
With Prometheus
Valentina Contenti
Kubernets Solutions Engineer
valentina@sighup.io

2. What is Prometheus?
Community Driven Open-source Monitoring and
Alerting stack, ships a time series database, an
alerting entity and a number of integration tools to
expose metrics.
Made for dynamic cloud environments.

3. What is Prometheus made for?
• Instrumentation for applications and
systems
• Metrics collection and storage
• Querying, alerting, dashboarding

4. What is Prometheus not made for?
• Logging or tracing
• Automatic anomaly detection
• Scalable or durable storage

5. A brief but distinguished history
Started in 2012 as a SoundCloud
internal project
Second project to join CNCF after
Kubernetes
Prometheus v1.0.0 released in 2016
Prometheus v2.0.0 released in 2017

6. Core features
• Simplicity + efficiency
• Dimensional data model
• Powerful query language - PromQL
• Service discovery integration

7. Architecture

8. Simplicity + efficiency:
• Local storage, no clustering
• 1 million+ samples/s
• Millions of series
• 1-2 bytes per sample
• HA by running two
• Go: static binary

9. Data model
https://prometheus.io/docs/concepts/data_model/
• Prometheus stores all data as time series: streams
of timestamped values belonging to the same metric
and the same set of labeled dimensions.
• Every time series is uniquely identified by its metric
name and optional key-value pairs called labels.
• The metric name specifies the general feature of a
system that is measured

10. Time series with labels
node_cpu_seconds_total{cpu="0",instance="demo.robustperception.io:9100",job="nod
e",mode="idle"} 14838327.84
metric name labels
• Flexible
• No hierarchy
• Explicit dimensions

11. Time series with labels
• Time series: tuple {time, value}

12. PromQL - simple but powerful

13. Queries in action

14. Better, persistent graphs: Grafana
https://grafana.com/

15. Pulling metrics
How do you expose those metrics?
• Exporters
• Language specific client libraries
• Service Discovery

16. /metrics endpoints

17. Exporters/Clientlibs
The community has contributed
exporters and clientlibs for pretty
much everything
https://prometheus.io/docs/instrumenting/exporters/
https://prometheus.io/docs/instrumenting/clientlibs/

18. Dynamic Environments: new challenges!
• Dynamic VMs
• Cluster schedulers
• Microservices
• …many services, dynamic hosts, and
ports

19. Service Discovery
https://prometheus.io/docs/prometheus/latest/configuration/configuration/
Prometheus has built-in support for:
• VM providers (AWS, Azure, Google, ...)
• Cluster managers (Kubernetes,
Marathon, …)
• Generic mechanisms (DNS, Consul,
Zookeeper, custom, ...)

20. Service Discovery
https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-
kubernetes.yml
Let’s use Kubernetes as an example

21. Alerting
https://prometheus.io/docs/practices/alerting/
You can write your own alerting rules.
(of course, the community has
examples/projects)

22. Writing alerts: a simple example
- alert: <alert_name>
annotations:
message: '{{ $labels.<some_label> }} <summary>'
doc: "<description>"
expr: |
<condition>
for: 15m
labels:
severity: warning

23. Writing alerts: a real-world example
- alert: NodeCPUStuckInIOWait
annotations:
message: '{{ $labels.instance }} spent more than half its CPU time in
IOWait in the last 5 minutes'
doc: "This alert fires if CPU time in IOWait mode calculated on a 5
minutes window for a given instance was more than 50% in the last 15
minutes."
expr: |
rate(node_cpu_seconds_total{mode="iowait"}[5m]) > 0.5
for: 15m
labels:
severity: warning

24. Alert Dispatching
The alertmanager dispatches alerts to
the right channel according to their
severity

25. Alertmanger
https://prometheus.io/docs/alerting/configuration/
You can configure your dispatching service just the way you like it.
(again, of course, there are examples)

26. To recap:
• Simplicity + efficiency
• Dimensional data model
• Powerful query language
• Service discovery integration

27. Try it at home!
http://demo.robustperception.io:9090/consoles/index.html