The ability to read and understand log files is very important for a system administrator. But what if, he is so busy in interpreting and analyzing them that he has no time for something else? How can the lazy system administrator hand this task over to somebody else, without providing access to the server?
This talk will show, by means of examples, the evolution of self-written CGI scripts, which worked directly on the log files, to the first data based systems until today, where techniques from Web 2.0 are used.
2. 2
Capgemini Outsourcing Services
• Infrastructur Engineer
wer-kennt-wen.de GmbH
• System Operator Linux
1blu AG
• System Engineer (Shared
Hosting)
• Teamlead System Operations
HostEurope GmbH
• Teamlead Shared Hosting
Operations
• Postmaster
Jan Doberstein
#OSS #bash #vi
3. 3
Eine Logdatei … enthält das
automatisch geführte Protokoll aller
oder bestimmter Aktionen von
Prozessen auf einem Computersystem
…
Wichtige Anwendungen finden sich
vor allem bei der Prozesskontrolle und
Automatisierung. Prinzipiell werden
alle Aktionen mitgeschrieben, die für
eine spätere Untersuchung
erforderlich sind oder sein könnten.
https://de.wikipedia.org/wiki/Logdatei
4. 4
10.41.148.32 - - [18/Mar/2013:12:43:43 +0100] "GET
/health/currentthroughput HTTP/1.1" 200 21
"http://10.43.248.109/users/new" "Mozilla/5.0
(Windows NT 6.1) AppleWebKit/537.22 (KHTML, like
Gecko) Chrome/25.0.1364.172 Safari/537.22"
10.41.148.32 - - [18/Mar/2013:13:06:17 +0100] "GET
/visuals/fetch/streamgraph?hours=2&stream_id=514
6bccd772ae72c95000006 HTTP/1.1" 200 2162
"http://10.43.248.109/streams/5146bccd772ae72c95
000006-user/messages" "Mozilla/5.0 (Windows NT
6.1) AppleWebKit/537.22 (KHTML, like Gecko)
Chrome/25.0.1364.172 Safari/537.22"
10.41.148.32 - - [18/Mar/2013:13:06:22 +0100] "GET
/messages HTTP/1.1" 200 57786
"http://10.43.248.109/streams/5146bccd772ae72c95
000006-user/messages" "Mozilla/5.0 (Windows NT
6.1) AppleWebKit/537.22 (KHTML, like Gecko)
Chrome/25.0.1364.172 Safari/537.22"
Mar 19 09:30:01 sv004972 /usr/sbin/cron[7657]: (root)
CMD (/root/bin/http_connection >>
/tmp/http_connection.log)
Mar 19 09:30:02 sv004972 sshd[7761]: Accepted
publickey for root from 192.168.97.41 port 37209
ssh2
Mar 19 09:31:01 sv004972 /usr/sbin/cron[8098]:
(/root/bin/http_connection >>
/tmp/http_connection.log)
Mar 19 09:30:02 sv004972 sshd[7761]: Accepted
publickey for root from 192.168.97.41 port 37209
ssh2
Mar 19 09:30:03 sv004972 sshd[7795]: Accepted
publickey for root from 192.168.97.41 port 37212
ssh2
Mar 19 09:31:01 sv004972 /usr/sbin/cron[8098]: (root)
CMD (/root/bin/http_connection >>
/tmp/http_connection.log)
10. 10
syslog ist ein De-facto-Standard
zur Übermittlung von Log-
Meldungen [...] Der Begriff
„syslog“ wird oft sowohl für das
eigentliche syslog-
Netzwerkprotokoll als auch für
die Anwendung oder Bibliothek
benutzt, die syslog-Meldungen
sendet oder empfängt.
http://de.wikipedia.org/wiki/Syslog
15. 15
Graylog2 enables you to unleash the power that
lays inside your logs. Use it to run analytics,
alerting, monitoring and powerful searches over
your whole log base. […] Create streams for
every consumer and have them always only one
click away.
http://graylog2.org
22. 22
logstash is a tool for managing events and
logs. You can use it to collect logs, parse them,
and store them for later use (like, for
searching).
http://logstash.net