Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
ELK Stack - An end to end solution for
analytics, logging, search & visualization.
By Vineeth Mohan
About Author
 Certified Elasticsearch trainer
 Author of Elasticsearch blueprints
 Author of Lucene 4 cookbook
 Over 5...
Overview
1. Business needs
2. Challenges in understand logs
3. How ELK helps us
Imagine the following system
1. We are operating a site having heavy traffic
2. To catch up with the traffic , we have a l...
Challenges
Challenge 01 - Mixed Log Structures
a. There is no universal log data structure format existing.
b. The formats of the log...
Mixed Log Structures
Mixed Log Structures
Mixed Log Structures
Mixed Log Structures
Challenge 02 - Different formats for time
a. The most important data in a log file is its time field.
b. But what happens ...
Different formats for time
Different formats for time
Challenge 03 - Log location and access
Logs of interest maybe
a. Spread across different machines
b. Depending on the mach...
Challenge 04 - Need for expertise
In order to get useful insights from the data
a. The data must be accessible. In most ca...
Understanding the logs visually
1. It is difficult for people to understand and make inferences from the textual data of t...
Understanding the logs visually
2. Suppose if we are able to visualize the data from the logs visually.
From the previous ...
How ELK can help us?
How ELK solves the problem for us?
1. Would collect all the data, centralize it
2. Parse the logs to a common format, incl...
ELK Stack architecture
ELK Stack - Logstash
1. Transform the log data to the structure of our preference.
2. Numerous tools and plugins to suppor...
ELK Stack - Elasticsearch
Provides the facility for
1. Near real time search
2. Extensive analytic capabilities.
ELK Stack - Kibana
1. Tool for visualizing the data from elasticsearch
2. Several methods of visualization for easy unders...
Get certified and #BeTheExpert
FOLLOW US ON SOCIAL MEDIATO STAY UPDATED ONTHE UPCOMING WEBINARS
 We have INSTRUCTOR LED -...
Nächste SlideShare
Wird geladen in …5
×

Elastic - ELK, Logstash & Kibana

1.620 Aufrufe

Veröffentlicht am

ELK Stack workshop covers real-world use cases and works with the participants to - implement them. This includes Elastic overview, Logstash configuration, creation of dashboards in Kibana, guidelines and tips on processing custom log formats, designing a system to scale, choosing hardware, and managing the lifecycle of your logs.

Veröffentlicht in: Technologie
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Gehören Sie zu den Ersten, denen das gefällt!

Elastic - ELK, Logstash & Kibana

  1. 1. ELK Stack - An end to end solution for analytics, logging, search & visualization. By Vineeth Mohan
  2. 2. About Author  Certified Elasticsearch trainer  Author of Elasticsearch blueprints  Author of Lucene 4 cookbook  Over 5 years of experience in Elasticsearch stack and Lucene  Runs Elasticsearch based consulting - Factweavers
  3. 3. Overview 1. Business needs 2. Challenges in understand logs 3. How ELK helps us
  4. 4. Imagine the following system 1. We are operating a site having heavy traffic 2. To catch up with the traffic , we have a load balancer and 1000 apache web servers behind it. 3. There is also a storage like mysql DB behind these servers which are used to query and insert data. 4. Every apache web servers logs their activities to their own server.
  5. 5. Challenges
  6. 6. Challenge 01 - Mixed Log Structures a. There is no universal log data structure format existing. b. The formats of the logs can depend on various factors like the device type, vendor, application etc. c. This inconsistency in log structures would make the searching on logs a difficult process
  7. 7. Mixed Log Structures
  8. 8. Mixed Log Structures
  9. 9. Mixed Log Structures
  10. 10. Mixed Log Structures
  11. 11. Challenge 02 - Different formats for time a. The most important data in a log file is its time field. b. But what happens when the time formats are different across different logs?. c. It becomes very difficult for us to do operations based on time.
  12. 12. Different formats for time
  13. 13. Different formats for time
  14. 14. Challenge 03 - Log location and access Logs of interest maybe a. Spread across different machines b. Depending on the machine logs differ in formats c. On different locations in the same machine
  15. 15. Challenge 04 - Need for expertise In order to get useful insights from the data a. The data must be accessible. In most cases the data is accessible only to the admins who are working on the servers. b. Need for experienced workforce who are able to understand the log data
  16. 16. Understanding the logs visually 1. It is difficult for people to understand and make inferences from the textual data of the logs. Imagine the log below of apache logs, where we have the data of the login information from cities : From the above logs it is very difficult to deduct the city wise statistics.
  17. 17. Understanding the logs visually 2. Suppose if we are able to visualize the data from the logs visually. From the previous logs, if we are able to extract the city names information and represent it as a pie chart like below. Now the data looks more eye candy and understandable.
  18. 18. How ELK can help us?
  19. 19. How ELK solves the problem for us? 1. Would collect all the data, centralize it 2. Parse the logs to a common format, including time details 3. Makes the logs quickly searchable and analyzable 4. Visualize the data in numerous ways with a wide range of analytics 5. Allows the end user to draw infrences from data with minimal technical overhead
  20. 20. ELK Stack architecture
  21. 21. ELK Stack - Logstash 1. Transform the log data to the structure of our preference. 2. Numerous tools and plugins to support the transformation.
  22. 22. ELK Stack - Elasticsearch Provides the facility for 1. Near real time search 2. Extensive analytic capabilities.
  23. 23. ELK Stack - Kibana 1. Tool for visualizing the data from elasticsearch 2. Several methods of visualization for easy understanding
  24. 24. Get certified and #BeTheExpert FOLLOW US ON SOCIAL MEDIATO STAY UPDATED ONTHE UPCOMING WEBINARS  We have INSTRUCTOR LED - both Online LIVE & Classroom Session  Classroom sessions in Bangalore & Delhi (NCR)  We have delivered more than 5000 trainings and have over 400 courses and a vast pool of over 200 experts to makeYOU the EXPERT! Certified Partners

×