Weitere ähnliche Inhalte
Ähnlich wie Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020 (20)
Kürzlich hochgeladen (20)
Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020
- 1. Steven Soroka and Anais Dotis-Georgiou
Machine Learning
with Telegraf Execd
Processor Plugin
- 3. © 2020 InfluxData. All rights reserved. 3
Introduction to Telegraf
● Active community
● Over 200 plugins
○ 170+ Input
○ 40+ Output
○ 15+ Processors
○ 5+ Aggregators
○ 5+ Serializers
● Single Binary
● 9.2k Stars
● Extendable in any language
- 4. © 2020 InfluxData. All rights reserved. 4
Simple example of a Telegraf Configuration
[agent]
interval = "1s"
flush_interval = "10s"
omit_hostname = true
[[inputs.cpu]]
[[outputs.influxdb_v2]]
urls = ["http://127.0.0.1:9999"]
token = "$INFLUX_TOKEN"
organization = "$INFLUX_ORG"
bucket = "brew"
- 6. © 2020 InfluxData. All rights reserved. 6
Introduction to the Execd Processor Plugin
● Runs an external program as a separate process, passes
metrics back and forth.
● Your program must read Influx Line Protocol on STDIN and
write metrics in Influx Line Protocol to STDOUT.
● Your program runs continuously in STDIN read loop, so you
only pay startup cost once.
● Can be any language, not just Go
- 9. © 2020 InfluxData. All rights reserved. 9
Simple example of Execd Processor Plugin
# A simple processor. It reads stdin and writes it to stdout.
import sys
def main():
for line in sys.stdin:
print(line.rstrip())
sys.stdout.flush()
if __name__ == '__main__':
main()
[[processors.execd]]
command = ["python", "./processors/forecasting.py"]
- 10. © 2020 InfluxData. All rights reserved. 10
Context: Monitoring temperatures of two beers (haze_v5 and bv_1). The ideal
temperature for the batch of beer is between 18-21.
Objectives:
● Forecast temperatures for the next 10 minutes
● Detect anomalous spikes in temperature that may indicate attention is needed
or equipment failure.
Monitoring a home brew
- 12. © 2020 InfluxData. All rights reserved. 13
Seven Segment OCR
ssocr -b black -f white -t 85 make_mono remove_isolated crop 30 95 460
220 shear 22 photo.png
- 13. © 2020 InfluxData. All rights reserved. 14
Execd Processor Plugin for Forecasting and
Anomaly Detection
Using the Execd Processor Plugin to achieve our goal and keep the beer at a steady
temperature.
- 16. © 2020 InfluxData. All rights reserved. 17
● Double exponential smoothing employs a level component
and a trend component at each period. Double exponential
smoothing uses two weights, (also called smoothing
parameters), to update the components at each period.
● In other words, an exponentially weighted average of the
trend and values of the data together.
What is Exponential Smoothing?
- 17. © 2020 InfluxData. All rights reserved. 18
● Statsmodels is a Python package that allows users to explore
data, estimate statistical models, and perform statistical
tests.
● Dataset considerations–small and sparse
● Easy to use
● Statsmodels enables more model configuration options than
Flux holtWitners() function
● Using the damped version of Holt helps eliminate false
positives.
● Algorithm works with small, irregular, and sparse datasets
Algorithm Selection: Holt’s Exponential
Smoothing
- 19. © 2020 InfluxData. All rights reserved. 20
FAQ about the Execd Processor Plugin
● What are the advantages to running the Execd plugin as
opposed to a cron job?
● Can I use the Execd Processor Plugin in combination with a
Neural Net?
● When shouldn’t I use the Execd Plugin?