ELKMétricas e monitoramento "real-time"

Sobre
Waldemar Neto @waldemarnt
http://walde.co/
2

Elastic.co
https://www.elastic.
co/
3

Elastic.co
Produtos
● Elasticsearch
● Logstash
● Kibana
● Beats
● Shield
● Marvel
● Graph
● Watcher
4
Serviços
● Suporte
● Treinamento
● Cloud
● Consultoria

ELK
● Elasticsearch
● Logstash
● Kibana
5

Elasticsearch
https://www.elastic.co/products/elasticsearch
6

Logstash
https://www.elastic.co/products/logstash
7

Kibana
https://www.elastic.co/products/kibana
8

PARSEARINDEXARVISUALIZAR
9 ELK Workflow

Etapa de log parsing com Logstash
Entrada
● Beats
● Couchdb_changes
● Heroku
● File
● Kafka
● TCP
● sqs
● Redis
● Rabbitmq
● Elasticsearch
Filtro
● Aggregate
● Mutate
● Alter
● CSV
● environment
● TCP
● JSON
● range
● grok
● geoip
Saída
● CSV
● Datadog
● Email
● Elasticsearch
● Exec
● File
● Graphite
● InfluxDB
● MongoDB
● s3
10

Etapa de indexar no Elasticsearch
● Normalização de texto
○ Acentos
○ Sinônimos
○ Linguística...
11

Etapa de visualizar com Kibana
● Dashboard em tempo real
○ Com varias visualizations
● Vários tipos de gráficos
○ Pie chart
○ Mapas
○ Barras
○ Gráfico de linhas
● Criação de vizualizations dinâmica
○ Suporte a queries do Elasticsearch
○ Suporte a agregações
12

Criando Visualization13

Monitorando servidor14

Monitoramento e métricas de APIs15

Métricas de negócio, BigData e anomalias16

Shield
Watcher
Elastalert
Alertas e proteção17

Na produção
● ELK as a service
○ Logz.io
○ Qbox.io
● Cloud
○ AWS
○ DigitalOcean
○ Elastic
18

Valeu!
Perguntas?

Referências20
http://walde.co/tag/elasticsearch/
https://github.com/guardian/elk-stack
http://techblog.netflix.com/2016/02/evolution-of-netflix-data-pipeline.html
http://www.front2backdev.com/2016/05/03/geo-temporal-anomaly/
https://dzone.com/articles/which-elk-stack-solution-is-right-for-you?
utm_content=buffer29ebc&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
http://blog.takipi.com/hosted-elasticsearch-the-future-of-your-elk-stack/
http://everythingshouldbevirtual.com/highly-available-elk-elasticsearch-logstash-kibana-setup