Logstash - CeBIT 2014 - Open Source Forum

www.netways.de // blog.netways.de // @netways
We love Open Source#CeBIT 2014 - Halle 6 / E16 / 310
CEBIT 2014 – 12.03.2014
LOG- UND EVENTMANAGEMENT
MIT LOGSTASH
BERND ERK | NETWAYS GMBH

AGENDA
￭ Kurzvorstellung
￭ Einführung
￭ Architektur
￭ Installation
￭ Routing und Filterung von Events
￭ Interfaces & API
￭ Integration in Nagios und Icinga
￭ Fragen & Antworten

KURZVORSTELLUNG

KURZVORSTELLUNG NETWAYS
• Firmengründung 1995
• Open Source seit 1997
• 40 Mitarbeiter
• Spezialisierung in den Bereichen
Open Source Systems
Management und Open Source
Datacenter Infrastructure
http://jobs.netways.de

NETWAYS KOMPETENZEN
• Monitoring & Reporting
• Configuration Management
• Service Management
• Knowledge Management
• Backup & Recovery
• High Availability & Clustering
• Cloud Computing
• Load Balancing
• Virtualization
• Database Management
OPEN SOURCE
SYSTEMS MANAGEMENT
OPEN SOURCE
DATA CENTER
MANAGED SERVICES MONITORING HARDWARE KONFERENZEN

Open Source Datacenter Conference
• 08. – 10. April 2014
• Datacenter | Automation | DevOps
PuppetCamp 2014
• 11. April Berlin
Open Source Monitoring Conference
• 11. April Berlin
NETWAYS KONFERENZEN

EINFÜHRUNG

LOGS
Logs -> Fluss an unstrukturierten Daten
Oct 4 16:57:24 web sshd[25828]: Received disconnect from 10.10.0.31: 11: disconnected by user
bestehend aus Timestamp und Message

EVENTS
Event -> Fluss an strukturierten Daten
Event {
Time: Oct 4 16:57:24
Process: sshd
State: Received disconnect from 10.10.0.31
Client: 10.10.0.31
}
bestehend aus konkreten Attributen

LOG & EVENTMANAGEMENT
Logs > Event > Analyse (Korrelation) > Aktion

TOOLS
￭ Nagios & Icinga Addons
• check_logfiles
• NagTrap
• EventDB
• EDBC
￭ Logmanagement-Tools
• Graylog
• Fluentd
• Logstash

LOGSTASH
Logstash

ARCHITEKTUR & INSTALLATION

LOGSTASH
￭ Logmanagement auf Basis von JRuby
￭ Konfigurierbare “Pipe”
￭ Flexible Plugin-Architektur für
• Input
• Filter
• Output
￭ Standardplugins für alle gängige Protokolle
￭ Webinterface
￭ Single File Deployment

Outputs
• amqp
• boundary
• circonus
• cloudwatch
• datadog
• datadog_metrics
• elasticsearch
• elasticsearch_http
• elasticsearch_river
• email
• exec
• file
• ganglia
• gelf
• gemfire
• google_cloud_storage
• graphite
• graphtastic
• hipchat
LOGSTASH - IO
Inputs
• amqp
• drupal_dblog
• elasticsearch
• eventlog
• exec
• file
• ganglia
• gelf
• gemfire
• generator
• graphite
• heroku
• imap
• irc
• log4j
• lumberjack
• pipe
• rabbitmq
• redis
• relp
• s3
• snmptrap
• sqlite
• sqs
• stdin
• stomp
• syslog
• tcp
• twitter
• udp
• unix
• varnishlog
• websocket
• wmi
• xmpp
• zenoss
• zeromq
• http
• irc
• jira
• juggernaut
• librato
• loggly
• lumberjack
• metriccatcher
• mongodb
• nagios
• nagios_nsca
• null
• opentsdb
• pagerduty
• pipe
• rabbitmq
• redis
• riak
• riemann
• s3
• sns
• sqs
• statsd
• stdout
• stomp
• syslog
• tcp
• udp
• websocket
• xmpp
• zabbix
• zeromq

INSTALLATION - LOGSTASH
￭ Download - http://logstash.net
￭ java -jar logstash-x.x.x-flatjar.jar agent -f <config-file>

ARCHITEKTUR
Shipper
Shipper
Shipper
Broker Search & Storage WebinterfaceIndexer

REDIS
￭ NoSQL in memory auf Basis von C
￭ Unterstützung verschiedener “Datentypen”
• strings
• hashes
• lists
• sets and sorted sets
￭ Support für verschiedene Replikationsszenarien
￭ SAUSCHNELL
$ ./redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -q
SET: 122556.53 requests per second
GET: 123601.76 requests per second
LPUSH: 136752.14 requests per second
LPOP: 132424.03 requests per second

INSTALLATION - REDIS
￭ Download - http://redis.io/download
￭ make
￭ make test
￭ make install
￭ /usr/local/bin/redis-server

ELASTICSEARCH
￭ Schemafreier RESTful Suchserver auf Basis von Java
￭ Basierend auf Lucene Core
￭ “Vergleichbar” mit Apache Solr
￭ Verteilte Architektur durch
• Shards
• Replicas
• Gateways
￭ Realtime-Suche als Basis für Kibana

INSTALLATION - ELASTICSEARCH
￭ Download – http://elasticsearch.org/download/
￭ Entpacken des Archives
￭ Ausführung von bin/elasticsearch

ROUTING UND FILTERUNG VON EVENTS

ÜBERSICHT
Shipper
Shipper
Shipper

KONFIGURATION - LOGSTASH - SHIPPER
￭ Übermittlung von Logs an Logstash
• Logstash
• Lumberjack
• Syslog
• Log4J
• Gelf
• File-Read
• u.v.a.m.

KONFIGURATION - LOGSTASH - SHIPPER
￭ Konfiguration
input {
file {
path => "/root/osmc/demodata/access.log.1”
type => "apache-access"
}
}
output {
stdout {
debug => true
}
redis {
host => "127.0.0.1"
data_type => "list"
key => "logstash.apache"
}
}
￭ java -jar logstash-current.jar agent -f
logstash_shipper.conf
Shipper
Shipper
Shipper

KONFIGURATION - LOGSTASH - INDEXER
￭ Konfiguration
input {
redis {
host => "127.0.0.1"
type => "redis-input"
# these settings should match the output of the agent
data_type => "list"
key => "logstash.apache”
}
}
output {
stdout {
debug => true
}
elasticsearch_http {
host => "127.0.0.1"
}
}
Shipper
Shipper
Shipper

KONFIGURATION - LOGSTASH – INDEXER - APACHE
￭ Konfiguration für Apache-Logs
input {
redis {
host => "127.0.0.1"
type => "apache-access”
data_type => "list"
format => "json_event"
}
}
filter {
if [type] == "apache-access" {
grok {
match => [ "message", "%{COMBINEDAPACHELOG}" ]
}
}
}
output {
elasticsearch_http {
host => "127.0.0.1”
}
}
Shipper
Shipper
Shipper

KONFIGURATION - LOGSTASH – INDEXER - GEOIP
￭ Konfiguration für Geo-Daten
input {
redis {
host => "127.0.0.1"
data_type => "list"
}
}
filter {
grok {
type => "apache-access"
pattern => "%{COMBINEDAPACHELOG}"
}
geoip {
source => "clientip"
add_tag => ["geotag"]
}
}
output {
elasticsearch_http {host => "127.0.0.1”}
}
Shipper
Shipper
Shipper

INTERFACES & API

KIBANA
Kibana

KIBANA

ELASTICHQ

KIBANA - DEMO
DEMO

INTEGRATION NAGIOS UND ICINGA

REALTIME LOGANALYSE
￭ Analyse verschiedener Quellen in Realtime
￭ Prüfung auf Patterns und States
• Facilitites
• Regex
• Programs
￭ Übermittlung als Passiver Event

ÜBERSICHT LOGSTASH UND ICINGA
Search & Storage WebinterfaceIndexer
Icinga –WebIcinga - Commandpipe

KONFIGURATION - LOGSTASH – INDEXER - ICINGA
￭ Konfiguration für Icinga-Alert
input {
…
}
filter {
if [type] == "syslog" {
grok {match => [ "message", "%{SYSLOGBASE}" ] }
grep {
match => [ "message", "Error" ]
drop => false
add_tag => "nagios-update"
add_field => [
# "nagios_host", "%{@source_host}",
"nagios_host", "localhost",
"nagios_service", "Logstash",
"nagios_level", "2”]
}}}
output {
elasticsearch {host => "127.0.0.1”}
nagios {
commandfile => "/var/lib/icinga/rw/icinga.cmd"
}}
Shipper
Shipper
Shipper

LOGSTASH – ICINGA - DEMO
DEMO

ZUGABE

REALTIME GRAPHING

STATSD & GRAPHITE
￭ StatsD
• Netzwerkdaemon auf Basis von UDP
• Bucket -> Value -> Flush
• Entkoppelte Zwischenaggretion für Statisik
￭ Graphite
• Graphing-Framework bestehend aus
• Whisper (Datenbank)
• Carbon (Engine)
• Graphite-Web (Interface)

INSTALLATION – STATSD - NODEJS
￭ apt-get install make python g++ checkinstall
￭ mkdir nodejs && cd nodejs
￭ wget -N http://nodejs.org/dist/node-latest.tar.gz
￭ tar xzvf node-latest.tar.gz && cd `ls -rd node-v*`
￭ checkinstall

INSTALLATION – STATSD
￭ wget https://github.com/etsy/statsd/archive/master.zip
￭ unzip master.zip
￭ node stats.js config.js

MONITORING - STATSD
￭ Status Informationen
• echo stats | nc 127.0.0.1 8126
• echo health | nc 127.0.0.1 8126
￭ Timer- und Counterinfo
• echo counters | nc 127.0.0.1 8126
• echo timers| nc 127.0.0.1 8126

INSTALLATION – GRAPHITE
￭ Download der Sources
• git clone https://github.com/graphite-
project/graphite-web.git
project/carbon.git
project/whisper.git

INSTALLATION – GRAPHITE
￭ Installation Whisper
pushd whisper
sudo python setup.py install
popd
￭ Installation Carbon
pushd carbon
sudo python setup.py install
popd
￭ Konfiguration Carbon
pushd /opt/graphite/conf
cp carbon.conf.example carbon.conf
cp storage-schemas.conf.example storage-schemas.conf

INSTALLATION – GRAPHITE - WEBAPP
￭ Check Dependencies Graphite webapp
pushd graphite-web
python check-dependencies.py
popd
￭ Installation Graphite webapp
pushd graphite-web
python setup.py install
popd
￭ Konfiguration Apache
example-graphite-vhost.conf

ÜBERSICHT STATSD UND GRAPHITE
Search & Storage WebinterfaceIndexer
GraphiteStatsd

KONFIGURATION - LOGSTASH – INDEXER - STATSD
￭ Konfiguration für Statsd
input {
redis {
host => "127.0.0.1"
data_type => "list"
key => "logstash.apache” format => "json_event”
add_field=> ["sitename","www.icinga.org"]
}
}
filter { if [type] == "apache-access" { grok {match => [ "message", "%{COMBINEDAPACHELOG}" ] } }}
output {
stdout { debug => true }
if [type] == "apache-access" {
statsd {
host => "localhost"
port => 8125
namespace => "logstash"
debug => false
increment => "apache.%{sitename}.response.%{response}”
count => ["apache.%{sitename}.bytes", "%{bytes}"]
}
}
elasticsearch_http {host => "127.0.0.1”}}
Shipper
Shipper
Shipper
StatsD

GRAPHITE - DEMO
DEMO

FRAGEN & ANTWORTEN

NETWAYS GmbH
Deutschherrnstrasse 15-19
90429 Nürnberg
Tel: +49 911 92885-0
Fax: +49 911 92885-77
Email: info@netways.de
Website: www.netways.de
Twitter: twitter.com/netways
Facebook: facebook.com/netways
Blog: blog.netways.de
FRAGEN & ANTWORTEN
DANKE

Logstash - CeBIT 2014 - Open Source Forum

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Logstash - CeBIT 2014 - Open Source Forum

Ähnlich wie Logstash - CeBIT 2014 - Open Source Forum (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Logstash - CeBIT 2014 - Open Source Forum