The document discusses collecting accurate metrics from microchip manufacturing processes. It notes that networks can be unreliable and data may be missing, so metrics collection systems need to account for these issues. It presents collecting sums, counts, and histograms of duration metrics as ways to mitigate problems from missing or unreliable data. Taking this approach helps detect failures and analyze key metrics like project durations and failure rates more accurately.

1. Accurate Metrics
{devday.23}
Christina Zeller & Marcus Crestani
1

2. Software Development
Consulting
Trainings
Functional Programming
Functional Software Architecture
active-group.de
funktionale-programmierung.de
2

3. Microchip manufacturing
3

4. Monitoring
4

5. What happened?
5

6. Missing data?
Unreliable network?
Erratic monitoring system?
Flawed collection of metrics?
6

7. Metrics
Time Metric Duration
0 duration{device=A} 1
0 duration{device=B} 3
1 duration{device=A} 4
1 duration{device=B} 5
2 duration{device=A} 3
2 duration{device=B} 3
3 duration{device=B} 5
4 duration{device=A} 1
4 duration{device=B} 1
5 duration{device=A} 2
5 duration{device=B} 1
6 duration{device=A} 2
6 duration{device=B} 3
7

8. Stored in time-series database
8

9. Failure rate
9

10. Retention policy
10

11. Retention policy
11

12. Retention policy
12

13. Sampling rate
13

14. Missing data
14

15. Missing data
15

16. No failure detected
16

17. Missing data
Unavoidable truths:
Networks are unreliable
Storage is limited
Data is missing
Possible mitigation by shaping metrics differently?
17

18. Sums and counters
Time Metric (Duration) Duration sum Count
0 duration{device=A} (1) 1 1
0 duration{device=B} (3) 3 1
1 duration{device=A} (4) 5 2
1 duration{device=B} (5) 8 2
2 duration{device=A} (3) 8 3
2 duration{device=B} (3) 11 3
3 duration{device=A} 8 3
3 duration{device=B} (5) 16 4
4 duration{device=A} (1) 9 4
4 duration{device=B} (1) 17 5
5 duration{device=A} (2) 11 5
5 duration{device=B} (1) 18 6
6 duration{device=A} (2) 13 6
6 duration{device=B} (3) 21 7
18

19. Stored in time-series database
19

20. Rate of increase
20

21. Project durations
21

22. Failure rate
22

23. Sampling rate 2
23

24. Rate of increase
24

25. Project durations
25

26. Higher failure rate
26

27. Histograms
Time Metric (Duration) Duration sum Count Failure bucket
0 duration{device=A} (1) 1 1 0
0 duration{device=B} (3) 3 1 0
1 duration{device=A} (4) 5 2 1
1 duration{device=B} (5) 8 2 1
2 duration{device=A} (3) 8 3 1
2 duration{device=B} (3) 11 3 1
3 duration{device=A} 8 3 1
3 duration{device=B} (5) 16 4 2
4 duration{device=A} (1) 9 4 1
4 duration{device=B} (1) 17 5 2
5 duration{device=A} (2) 11 5 1
5 duration{device=B} (1) 18 6 2
6 duration{device=A} (2) 13 6 1
6 duration{device=B} (3) 21 7 2
27

28. Sampling rate 1
28

29. Sampling rate 2
29

30. Project durations
30

31. Failure rate with histograms
31

32. Takeaway
Always collect metrics!
Check that your metrics match reality!
Always consider missing data!
For duration metrics always use histograms!
32

33. Accurate Metrics
33

34. What's next
Code readability vs. metrics collection
https://github.com/active-group/active-logger
34

35. Accurate Metrics
{devday.23}
Christina Zeller & Marcus Crestani
35