7. Step 1 – Beats – PacketbeatsStep 1 – Beats – Packetbeats
● Packetbeat caveat – performance impact
● Traffic capturing options
– pcap / af_packet / pf_ring: use af_packet on AWS!
– memory mapped sniffing
– 200k packets per second before dropping packets
8. Step 1 – Beats – TopbeatsStep 1 – Beats – Topbeats
● Topbeat use cases
– System wide stats: hooked onto the Linux top
command for system load, used/idle times,
free/used memory
– Per process stats: Process name, PID, CPU time,
memory size
– File system stats: Device name, mount point,
available disk space, used disk space
10. Step 1 – Beats – FilebeatsStep 1 – Beats – Filebeats
● Filebeat properties
● Send at least once by confirmation
● Handles log rotation
● Last reading state in case you restart your
system of LogStash is not reachable => upon
revive it will send all missing logs
● By default send new log lines every 10
seconds
14. Step 3 – ElasticSearchStep 3 – ElasticSearch
● ElasticSearch
– Distributed, open source search and analytics engine
– Uses JSON Documents, is schema-less and RESTful
– Based on Lucene (Java): reverse indexing
– Performance profile:
● Slow in write (re-indexing)
● Fast in read => analysis
15. Step 4 – KibanaStep 4 – Kibana
● Kibana
– Open source data visualization platform
– Interact with your data through powerful graphics
– Ongoing battle against Apache Solr
● Kibana dashboards per client => a 4x win
– DevOps (ssh/grep/alerting)
– Developers (performance analysis, API optimization)
– PM (pro-active vs. fire extinguishing)
– Customers => new revenue streams!
● Technical SEO
● Business Intelligence