Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Logs aggregation and analysis

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Nächste SlideShare
Log aggregation and analysis
Log aggregation and analysis
Wird geladen in …3
×

Hier ansehen

1 von 37 Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (20)

Anzeige

Ähnlich wie Logs aggregation and analysis (20)

Weitere von Divante (20)

Anzeige

Aktuellste (20)

Logs aggregation and analysis

  1. 1. Logs aggregation and analysis
  2. 2. Agenda ● Problems with logs ● How do we do it at Divante - ELK Stack o ElasticSearch o Logstash o Kibana o Architecture o Additional tools ● Summary ● Questions
  3. 3. What problems with logs may we encounter?
  4. 4. Problems with logs No consistent log format http://blog.tersmitten.nl/how-to-colorize-your-log-files-with-ccze.html
  5. 5. Problems with logs • cat • grep • awk • sed • tail • regular expressions Hampered log analysis, increased response time. Log search, analysis – old school
  6. 6. Problems with logs Heavy server load, low application performance • Synchronous, blocking writing • IO operations burdening the server • Limited amount of inodes in the file system • Relatively slow write speed • Logs cleanup http://wiki.processmaker.com/index.php/Advanced_Performance_Monitor_Dashboards
  7. 7. Problems with logs Complex architecture - read and write problem • Read/write on servers after the load balancer - NFS application? https://www.digitalocean.com/community/tutorials/5-common-server-setups-for-your-web-application
  8. 8. Problems with logs Continuous monitoring • Continuous monitoring of all application parameters is not easy • Technical knowledge is required
  9. 9. How do we do it at Divante? ELK Stack
  10. 10. ELK Stack - what is it? ELK Stack is a set of tools: It provides a centralized log management in distributed, high-availability systems.
  11. 11. E as in ElasticSearch ElasticSearch ● + NoSQL database ● + Full-text search ● + REST Api (json) ● + Based on Apache Lucene ● + Replication, snapshots ● + Official PHP and JavaScript libraries available ● - No transactions ● Requirements: Java
  12. 12. E as in ElasticSearch Installation and setup ● Java Oracle installation o sudo apt-get install python-software-properties o sudo add-apt-repository -y ppa:webupd8team/java o sudo apt-get update o sudo apt-get -y install oracle-java8-installer ● ElasticSearch installation o wget and unzip of the latest distribution available at https://www.elastic.co/downloads o Setup in the elasticsearch.yml file ● Running ElasticSearch: o ./bin/elasticsearch -d
  13. 13. L as in Logstash ● Aggregation of logs from multiple sources ● Normalization (parsing, filtering) of the collected data ● Sending the normalized data to various sources ● Requirements: Java INPUT FILTER PARSE OUTPUT
  14. 14. L as in Logstash Input: (41) ● elasticsearch, imap ● rabbitmq, redis ● sqlite, syslog ● tcp, twitter ● varnishlog, websocket Output: (55) ● elasticsearch, mongodb, ● redis, email, file, csv ● hipchat, http ● jira, redmine, rabbitmq ● tcp, websocket, zabbix More at: http://logstash.net/docs/1.4.2/ INPUT FILTER PARSE OUTPUT
  15. 15. L as in Logstash Filters: (50) ● checksum ● csv, date ● elasticsearch ● fingerprint ● geoip, grep, grok ● json, json_encode ● ruby, split ● translate ● urldecode ● useragent, xml More at: http://logstash.net/docs/1.4.2/ INPUT FILTER PARSE OUTPUT
  16. 16. L as in Logstash GROK filter ● Parsing and analysis of any text ● Grok is the best way to process even the most unstructured data ● Over 120 samples and possibility to create new ones [ERROR] - 2015/13/03-11:01:31 - 192.168.0.1 - Some error message [%{GREEDYDATA:message}] - %{DATESTAMP} - %{IP:client} - %{GREEDYDATA:message}
  17. 17. L as in Logstash Installation and setup ● Java Oracle and Logstash installation o wget and unzip of the latest distribution available at https://www.elastic.co/downloads ● Setup: input { file { type => "syslog" path => ["/var/log/auth.log", "/var/log/syslog"] }} ● Running Logstash: o ./bin/logstash
  18. 18. K as in Kibana ● Data visualization in the form of a web app ● Data search, filters and analysis ● Intuitive interface, not only for the programmers ● Instant sharing and embedding of multiple dashboards ● Easy dashboard adjustment - JSON ● Export of the results
  19. 19. K as in Kibana https://www.elastic.co/blog/kibana-4-literally
  20. 20. K as in Kibana Bar chart: https://www.elastic.co/blog/kibana-4-beta-2-get-now
  21. 21. K as in Kibana Pie chart: https://www.elastic.co/blog/kibana-4-for-investigating-pacs-super-pacs-and-your-neighbors
  22. 22. K as in Kibana Histogram: http://blog.qbox.io/kibana-4-and-elasticsearch-v-1-4-4-and-1-3-9
  23. 23. K as in Kibana Data table: https://www.elastic.co/blog/kibana-4-for-investigating-pacs-super-pacs-and-your-neighbors
  24. 24. K as in Kibana Geolocation: https://www.elastic.co/blog/kibana-4-literally
  25. 25. K as in Kibana Relation chart: http://demo.packetbeat.com/#/dashboard/elasticsearch/Packetbeat%2520Statistics
  26. 26. K as in Kibana Installation and setup ● Kibana installation: o wget and unzip of the latest distribution available at https://www.elastic.co/downloads ● Setup: o Kibana is set up on the default ElasticSearch port - 9300 ● Running Kibana: o ./bin/kibana ● Start in a browser: o http://YOURDOMAIN.com:5601
  27. 27. ELK Stack ARCHITECTURE
  28. 28. ELK - Architecture Basic: What is the potential problem? ElasticSearch REST API
  29. 29. ELK - Architecture Performance + scalability Setup with Redis Providers Indexer
  30. 30. ELK - Architecture Performance + scalability Setup with the use of a queuing mechanism Indexer Providers
  31. 31. Additional TOOLS
  32. 32. Tools ElasticSearch Plugin: Head ElasticSearch monitoring and management panel Installation: ● elasticsearch/bin/plugin -install mobz/elasticsearch-head ● http://localhost:9200/_plugin/head/
  33. 33. Tools ElasticSearch Plugin: Morfologik Plugin enabling the use of Polish characters in ElasticSearch queries Installation: ● cd elasticsearch ● bin/plugin -install com.github.chytreg/elasticsearch-analysis- morfologik/2.3.1
  34. 34. Tools Marvel Application for the continuous monitoring of ElasticSearch http://blog.qbox.io/elasticsearch-marvel-released
  35. 35. ELK Stack Summary ● Consistent log structure ● Centralized log system ● Dashboard supporting the search and analysis processes ● Continuous monitoring ● Reduced response time ● Happy customer receives new charts :)
  36. 36. VS http://blog.rootshell.be/tag/unix/ http://imgbuddy.com/kibana-demo.asp
  37. 37. Thank you! Bartosz Picho sales@divante.co divante.co/blog

×