This document discusses using Splunk as a security information and event management (SIEM) tool. It describes how Cisco's Computer Security Incident Response Team (CSIRT) uses Splunk to monitor over 1 terabyte of log data per day across Cisco's global operations. The document contrasts old approaches that relied on vendor-provided reports with new approaches like hunting for threats by building custom queries. It emphasizes an iterative process of filtering, refining queries to find bad traffic and saving reusable searches to automate threat detection.

1. Copyright © 2013 Splunk Inc.
Splunk the SIEM
Jeff Bollinger 0x506682C5
Technical Leader and Infosec Investigator:
CSIRT
Cisco Systems, Inc.
https://blogs.cisco.com/author/jeffbollinger/
https://twitter.com/jeffbollinger

2. About Me...
• Cisco Computer Security Incident Response Team (CSIRT)
• CSIRT = Security Monitoring and Incident Response
• Architecture, Engineering, Research, and Investigations
• Enterprise global threat and 24x7 incident response

3. The Numb3rs
• Cisco Systems Inc.:
– 100 countries
– 130,000 employees (with laptops and phones)
– 150,000 servers of all types
– 40,000 routers
– 1,500 labs
– 1 CSIRT analyst for every 7,000 employees

4. The Numb3rs
Cisco indexes almost 1Tb of log data per day

5. Incident Response Basics
• What am I trying to protect?
• What are the threats?
• > How do I detect them?
• How do we respond?

6. How Do I Detect?

7. Out With The Old
• You don’t know what you don’t
know
• Buy and trust a SIEM to run
canned reports
• Wait for updates from the vendor
• Try to edit/create custom reports
• Build your own collection infrastructure
• Data-centric approach
• Build your own reports
• Research your own intelligence
• Operationalize and optimize!
The Old Way The New Way

8. playbook |ˈplāˈbŏk|
(noun)
A prescriptive collection of repeatable
queries (reports) against security event data
sources that lead to incident detection and
response.
Analyze: SIEM

9. A Note on Strategy
Hunting vs. Gathering

10. Hunting: Build a Query – Find Bad Stuff
• Start with the obvious and simple:
index=wsa earliest=-24h x_wbrs_score=ns
English translation: Splunk, look at our web proxy
logs over the past 24 hours, and give me all the
web sites (objects) that had no known reputation
score.

11. Hunting: Build a Query – Find Bad Stuff
index=wsa earliest=-24h x_wbrs_score=ns
Let me stop you right there…

12. Hunting: Build a Query – Find Bad Stuff
• Filter based on unique attributes:
index=wsa earliest=-24h x_wbrs_score=ns |where isnull(cs_referer)
English translation: Splunk, look at our web proxy
logs over the past 24 hours, and give me all the
web sites (objects) that had no known reputation
score, and there was no HTTP referrer.

13. Hunting: Build a Query – Find Bad Stuff
index=wsa earliest=-24h x_wbrs_score=ns | where isnull(cs_referer)
Ok getting better, sort of…

14. Hunting: Build a Query – Find Bad Stuff
• Filter, refine, filter, refine:
index=wsa earliest=-24h application/x-dosexec ns GET 200 x_wbrs_score=ns
cs_method=GET sc_http_status=200 cs_mime_type=application/x-dosexec (java OR
MSIE) NOT (mirror OR cdn) | where isnull(cs_referer)
English translation: Splunk, query our web proxy logs over the past
24 hours, and give me all the web sites (objects) that had no known
reputation score, and there was no HTTP referrer, where either Java
or Internet Explorer successfully downloaded an executable file
from a site that didn’t have ‘mirror’ or ‘CDN’ in the URL.

15. Hunting: Build a Query – Find Bad Stuff
Here we go!
index=wsa earliest=-24h application/x-dosexec ns GET 200 x_wbrs_score=ns
cs_method=GET sc_http_status=200 cs_mime_type=application/x-dosexec (java OR
MSIE) NOT (mirror OR cdn) | where isnull(cs_referer)

16. Gathering: Build a Query – Find Bad
Stuff
If you can find or create a re-usable pattern, you
can save a search, make a report, and automate!
16

17. Gathering: Build a Query – Find Bad
Stuff
For example: this query will detect the Tracur clickfraud trojan:
index=wsa earliest=-6h@h m cs_url="*/m/*” MSIE (NOT (cs_referer="*")) | regex
cs_url="^http://(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?).(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-
9]?).(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?).(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/m/[A-Za-z0-
9/+]{50,1000}$"
http://www.microsoft.com/security/portal/threat/encyclopedia/entry.aspx?Name=Trojan%3aWin32%2fTracur

18. Do It Yourself
• Once you have:
• Solid, repeatable, saved searches
• Research and intelligence gathering
• Consistent handling procedures
• Documentation and tuning
• You have your own SIEM, running in Splunk, and
completely custom to your organization

19. Thank you