The ELK stack (Elasticsearch-Logstash-Kibana) provides a cost effective alternative to commercial SIEMs for ingesting and managing OSSEC alert logs. This presentation will show you how to construct a low cost SIEM based on ELK that rivals the capabilties of commercials SIEMs.

1. Vic Hargrave | vichargrave@gmail.com | @vichargrave
1

2. • Software Architect for Trend Micro Data Analytics Group
• Blogger for Trend Micro Security Intelligence and Simply
Security
• Email: vichargrave@gmail.com
• Twitter: @vichargrave
• LinkedIn: www.linkedin.com/in/vichargrave
2

3. • Open Source SECurity
• Open Source Host-based Intrusion Detection System
• Founded by Daniel Cid
• Log analysis and file integrity monitoring for Windows,
Linux, Mac OS, Solaris and many *nix systems
• Agent – Server architecture
• http://www.ossec.net
3

4. 4
commercial or
open source
SIEM
Syslog
Syslog
Syslog
syslog

5. 5
commercial
SIEM

6. Logstash Kibana
6

7. 7

8. • Open source, distributed, full text search engine
• Based on Apache Lucene
• Stores data as structured JSON documents
• Supports single system or multi-node clusters
• Easy to set up and scale – just add more nodes
• Provides a RESTful API
• Installs with RPM or DEB packages and is controlled
with a service script.
8

9. • Index – contains documents, ≅ table
• Document – contains fields, ≅ row
• Field – contains string, integer, JSON object, etc.
• Shard – smaller divisions of data that can be stored
across nodes
• Replica – copy of the primary shard
9

10. # default configuration file - /etc/elasticsearch/elasticsearch.yml
######################### Cluster #########################
# Cluster name identifies your cluster for auto-discovery
#
cluster.name: ossec-mgmt-cluster
########################## Node ###########################
# Node names are generated dynamically on startup, so you're relieved
# from configuring them manually. You can tie this node to a specific name:
#
node.name: "es-node-1" # e.g. Elasticsearch nodes numbered 1 – N
########################## Paths ##########################
# Path to directory where to store index data allocated for this node.
#
path.data: /data/0, /data/1
10

11. • Log aggregator and parser
• Supports transferring parsed data directly to
Elasticsearch
• Controlled by a configuration file that specifies input,
filtering (parsing) and output
• Key to adapting Elasticsearch to other log formats
• Run logstash in logstash home directory as follows:
bin/logstash ––conf <logstash config file>
11

12. 12
input {
# stdin{}
udp {
port => 9000
type => "syslog"
}
}
filter {
if [type] == "syslog" {
grok {
# SEE NEXT SLIDE
}
mutate {
remove_field => [ "syslog_hostname", "syslog_message", "syslog_pid", "message",
"@version", "type", "host" ]
}
}
}
output {
# stdout {
# codec => rubydebug
# }
elasticsearch_http {
host => "10.0.0.1"
}
}

13. • OSSEC syslog alert
Jan 7 11:44:30 ossec ossec: Alert Level: 3; Rule: 5402 - Successful sudo to ROOT executed; Location:
localhost->/var/log/secure; user: user; Jan 7 11:44:29 localhost sudo: user : TTY=pts/0 ;
PWD=/home/user ; USER=root ; COMMAND=/bin/su
• grok { }
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_host}
%{DATA:syslog_program}: Alert Level: %{NONNEGINT:Alert_Level}; Rule: %{NONNEGINT:Rule} -
%{DATA:Description}; Location: %{DATA:Location}; (user: %{USER:User};%{SPACE})?(srcip:
%{IP:Src_IP};%{SPACE})?(user: %{USER:User};%{SPACE})?(dstip:
%{IP:Dst_IP};%{SPACE})?(src_port: %{NONNEGINT:Src_Port};%{SPACE})?(dst_port:
%{NONNEGINT:Dst_Port};%{SPACE})?%{GREEDYDATA:Details}"
13
}
add_field => [ "ossec_server", "%{host}" ]

14. • General purpose query UI
• Javascript implementation
• Query Elasticsearch without coding
• Includes many widgets
• Run Kibana in browser as follows:
http://<web server ip>:<port>/<kibana path>
14

15. /** @scratch /configuration/config.js/5
* ==== elasticsearch
*
* The URL to your elasticsearch server. You almost certainly don't
* want +http://localhost:9200+ here. Even if Kibana and Elasticsearch
* are on the same host. By default this will attempt to reach ES at the
* same host you have kibana installed on. You probably want to set it to
* the FQDN of your elasticsearch host
*/
elasticsearch: http://+"<elasticsearch node IP>"+":9200",
15

16. 16

17. 17

18. • ElasticHQ
• Elasticsearch plug-in
• Install from Elasticsearch home directory:
bin/plugin -install royrusso/elasticsearch-HQ
• Provides cluster and node management metrics and
controls
18

19. 19

20. 20

21. 21
And now for something completely
different.
The OSSEC virtual appliance

22. Free
22

23. • Designed to work in a trusted environment
• No built in security
• Easy to erase all the data
curl –XDELETE http://<server>:9200/_all
• Use with a proxy that provides authentication and
request filtering such as Nginx
– http://wiki.nginx.org/Main
23

24. • Elasticsearch
– http://www.elasticsearch.org
• Logstash
– http://logstash.net
• Kibana
– http://www.elasticsearch.org/overview/kibana/
• ElasticHQ
– http://elastichq.org
• Elasticsearch for Logging
– http://vichargrave.com/ossec-log-management-with-elasticsearch/
– http://edgeofsanity.net/article/2012/12/26/elasticsearch-for-logging.html
24

25. 25