1. Quantitative Analysis of
Intrusion Detection Systems:
Snort and Suricata
Joshua S. White
Thomas T. Fitzsimmons
Jeanna N. Matthews, PhD

2. Outline
•
•
•
•
•
•
IDS Testing Background
Snort / Suricata
Our Method
Analysis
Results
Conclusion / Future Work

3. Background
• Given competing claims, an objective head-to-head
comparison of the performance of both the Snort
and Suricata Intrusion Detection Systems is
needed.

4. Snort
• Open source IDS
• Open-source community and corporate support
from SourceFire
• Single-threaded, uses a rule-based language
combining signature, protocol and anomaly
inspection methods
• http://www.snort.org/
* Snort, www.snort.org, Snort is a Registered Trademark of SourceFire Inc.
** Snort Logo Trademark of SourceFire Inc.

5. Suricata




Open source IDS
Open Information Security Foundation (OISF)
Multi-threaded, native IPv6, Snort syntax,
Unified2 output, Statistical anomaly detection,
File extraction, High-speed Regex, IP reputation,
Hardware and GPU Acceleration
http://www.openinfosecfoundation.org/

6. Method
• Be different than existing testing systems
– PytBull, 300 Tests, Aimed at rule validation
• Pytbull.sourceforge.net
• Focus on testing performance
– CPU, Memory, Scaling, PPS Processing
• Initial system consisted of 2800 LOC written in Bash
– 36 Hrs to process
• Current framework 650 LOC written in Python
– 6.5 Hrs to process

7. Method Details

8. Test Details
• Snort and Suricata
• 10 x Workloads
• 4 x Ruleset Configurations
– Snort VRT Free, ET-Free, ET-Pro, No-Rules
• 2 x IDS Configurations
– Default and Optimized
• 10 x Core Configurations
– 1,2,3,4,5,6,8,12,18,24
• Each Test Run 5 Times
• Total of 8000 tests
• Additional 600 Live Replay Tests

9. Initial Results
• Baseline tests
– PPS graph
– Suricata 1.2 performance drop at 4 Cores
• Even when using optimized configuration
– Snort consistent single threaded performance

10. Initial Results Continued
• Suggested changes asked for advice
– The OISF “Victor Julien”
• Max-Pending-Packets hard coded to 1000
– Changes now include variable configuration up to
USHRT_MAX (65535)
• Developed on dual and quad-core systems, threading didn't
consider keeping like flows together in clusters of cores
– Changes now include CPU-Affinity settings in configuration
files
» This includes sticking like flows to single core cluster,
keeps inter-CPU communication bottlenecks down

11. Initial Results Continued
• Snort.org
– Single threaded performance seemed to be major limitation
• Companies like Bivio ran custom parallelized version
– “Anonymous” at Sourcefire gave us tips for implementing a
standard parallelized version on regular hardware
» Not an easy solution to implement even for us
» Somewhat buggy startup at times
» Solved the single threading issue
– Blogs suggested replacing standard regex (Aho-Corasick
Binary NFA) with (Aho-Corasick)

12. Initial Results Continued
• Snort.org
– Single threaded performance seemed to be major limitation
• Companies like Bivio ran custom parallelized version
– “Anonymous” at Sourcefire gave us tips for implementing a
standard parallelized version on regular hardware
» Not an easy solution to implement even for us
» Somewhat buggy startup at times
» Solved the single threading issue
– Blogs suggested replacing standard regex (Aho-Corasick
Binary NFA) with (Aho-Corasick)

13. Results

14. Results

15. Results

16. Next Gen Results

17. Thanks!
• Contact:
– Joshua S. White
PhD Candidate
whitejs@clarkson.edu
Clarkson University