SlideShare ist ein Scribd-Unternehmen logo
1 von 26
MOLOCH:
SEARCHFOR
FULLPACKET
CAPTURE
It is a Great Horned Owl
Project Logo
WHYTHE OWL?
Owls are silent hunters that go after RATs. We think that’s pretty
cool.
3
HISTORY
LESSON
WHYAOL BUILT MOLOCH
WHAT IS MOLOCH?
5
Moloch is an open source, scalable IPv4 packet capture indexing and
database system, built using open source technologies.
• A simple web GUI is provided for browsing,
searching, viewing and exporting PCAP data.
• Web APIs are accessible if you wish to design your
own GUI or directly grab PCAP with various
command line tools for further analysis or
processing.
• Find it on AOL’s GitHub page:
https://github.com/aol/moloch
It’s like AOL Search for PCAP repositories!
WHAT IS MOLOCH NOT?
6
NOT IDS: NO ALERTS
NOT IPV6 (Today)
NOT SLOW
NOT CLOSED
NOT EXPENSIVE
WHYUSE MOLOCH?
7
Real-time capture of network traffic for forensic and
investigative purposes
• Combine the power of Moloch with other indicators (intelligence
feeds, alerting from IDS/anti-virus) to empower your analysts to
quickly and effectively review actions on the network to
determine the validity/threat.
• Review past network traffic for post compromise investigations.
Static PCAP repository
• Import large collections of PCAP that were created by malware.
• Import collections of PCAP from Capture The Flag events.
• Custom tagging of data at time of import.
THE PIECES OF MOLOCH
8
CAPTURE
• A C application that sniffs the network interface, parses the
traffic, and creates the Session Profile Information (SPI data)
and writes it to disk.
DATABASE
• Elasticsearch is used for storing and searching through the SPI
data generated by the capture component.
VIEWER
• A web interface that allows for GUI and API access from remote
hosts to browse or query SPI data and retrieve stored PCAP.
THE PIECES OF MOLOCH:
CAPTURE
9
Libnids based daemon written in C
Can be used to sniff network interface for live capture
Can be called from CLI to do manual imports
Parses layers 3-7 to create SPI data
• Spits them out to the Elasticsearch cluster. A lot like making owl
pellets!
THE PIECES OF MOLOCH:
DATABASE
11
Elasticsearch (http://www.elasticsearch.org)
• Powered by Apache Lucene (http://lucene.apache.org)
• Requests over HTTP(s)
• Results returned in JSON
Nosql
• Network traffic doesn’t fit the mold for relational DBs.
Documented oriented
• Great for lots and lots of network sessions.
Automatic sharding across multiple hosts
• At the time, we skipped SOLR because it couldn’t run distributed.
Fast, scalable, all that goodness
THE PIECES OF MOLOCH:
VIEWER
12
Node.js based application
• Event driven server side JavaScript platform.
• Based on Chrome’s JavaScript runtime.
• Comes with its own HTTP server and easy JSON for
communication.
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP.
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs,
consoles, and command line tools is easy.
• Easy automation to retrieve PCAP or sessions of interest.
THE PIECES OF MOLOCH:
VIEWER
13
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for
communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs,
consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
THE PIECES OF MOLOCH:
VIEWER
14
Nodejs based application
• Event driven server side JavaScript platform
• Based on Chrome’s JavaScript runtime
• Comes with its own HTTP server and easy JSON for
communication
Web based GUI
• Browsing / searching / viewing / exporting SPI data and PCAP
GUI and API use URIs
• All calls are done using URIs so integration with SEIMs,
consoles, command line tools is easy.
• Easy automation to retrieve pcap or sessions of interest.
ARCHITECTUREOF MOLOCH:
DATAFLOW
15
ARCHITECTUREOF MOLOCH:
MULTINODE WITH CLUSTER
16
ARCHITECTUREOF MOLOCH:
SCALE
17
Packets Captured Kilobytes Saved Sessions Saved
Example: Moloch Capture
Documents Disk Storage (MB)
Example: Elasticsearch
MOLOCH: SPI-DATATYPES
SESSION PROFILE
INFORMATION
18
IP
• Source
• Destination
• Ports
• Protocol
HTTP
• Method
• Status Codes
• Headers
• Content Type
DNS
• IP Address
• Hostnames
MOLOCH: SPI-DATATYPES
SESSION PROFILE
INFORMATION
19
SSL/TLS
• Cert Elements:
• Common Name
• Serial Number
• Alt Names
SSH
• Client Name
• Public Key
• Port
IRC
• Channel Name
• Hostname
MOLOCH: CAPTURE
CREATING SPI DATA
20
MOLOCH: CAPTURE
CREATING SPI DATA
21
MOLOCH: CAPTURE
CREATING SPI DATA
22
MOLOCH: CAPTURE
CREATING SPI DATA
23
MOLOCH: CAPTURE
CREATING SPI DATA
24
MOLOCH: DEMO
25
MOLOCH: QUESTIONS?
26

Weitere ähnliche Inhalte

Was ist angesagt?

High-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uringHigh-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uring
ScyllaDB
 
Accelerating Ceph with RDMA and NVMe-oF
Accelerating Ceph with RDMA and NVMe-oFAccelerating Ceph with RDMA and NVMe-oF
Accelerating Ceph with RDMA and NVMe-oF
inside-BigData.com
 

Was ist angesagt? (20)

Hadoop Strata Talk - Uber, your hadoop has arrived
Hadoop Strata Talk - Uber, your hadoop has arrived Hadoop Strata Talk - Uber, your hadoop has arrived
Hadoop Strata Talk - Uber, your hadoop has arrived
 
MongoDB - Sharded Cluster Tutorial
MongoDB - Sharded Cluster TutorialMongoDB - Sharded Cluster Tutorial
MongoDB - Sharded Cluster Tutorial
 
Using Asterisk and Kamailio for Reliable, Scalable and Secure Communication S...
Using Asterisk and Kamailio for Reliable, Scalable and Secure Communication S...Using Asterisk and Kamailio for Reliable, Scalable and Secure Communication S...
Using Asterisk and Kamailio for Reliable, Scalable and Secure Communication S...
 
SIP Attack Handling (Kamailio World 2021)
SIP Attack Handling (Kamailio World 2021)SIP Attack Handling (Kamailio World 2021)
SIP Attack Handling (Kamailio World 2021)
 
Cilium - BPF & XDP for containers
 Cilium - BPF & XDP for containers Cilium - BPF & XDP for containers
Cilium - BPF & XDP for containers
 
Astricon 10 (October 2013) - SIP over WebSocket on Kamailio
Astricon 10 (October 2013) - SIP over WebSocket on KamailioAstricon 10 (October 2013) - SIP over WebSocket on Kamailio
Astricon 10 (October 2013) - SIP over WebSocket on Kamailio
 
Geode - Day 1
Geode - Day 1Geode - Day 1
Geode - Day 1
 
Linux Network Stack
Linux Network StackLinux Network Stack
Linux Network Stack
 
Using S3 Select to Deliver 100X Performance Improvements Versus the Public Cloud
Using S3 Select to Deliver 100X Performance Improvements Versus the Public CloudUsing S3 Select to Deliver 100X Performance Improvements Versus the Public Cloud
Using S3 Select to Deliver 100X Performance Improvements Versus the Public Cloud
 
Dataplane programming with eBPF: architecture and tools
Dataplane programming with eBPF: architecture and toolsDataplane programming with eBPF: architecture and tools
Dataplane programming with eBPF: architecture and tools
 
Real-Time Status Commands
Real-Time Status CommandsReal-Time Status Commands
Real-Time Status Commands
 
High-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uringHigh-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uring
 
Kernel Recipes 2019 - XDP closer integration with network stack
Kernel Recipes 2019 -  XDP closer integration with network stackKernel Recipes 2019 -  XDP closer integration with network stack
Kernel Recipes 2019 - XDP closer integration with network stack
 
Scaling Asterisk with Kamailio
Scaling Asterisk with KamailioScaling Asterisk with Kamailio
Scaling Asterisk with Kamailio
 
Centralised logging with ELK stack
Centralised logging with ELK stackCentralised logging with ELK stack
Centralised logging with ELK stack
 
RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compaction
 
JBoss Fuse vs Tibco Matrix
JBoss Fuse vs Tibco MatrixJBoss Fuse vs Tibco Matrix
JBoss Fuse vs Tibco Matrix
 
Performance Wins with BPF: Getting Started
Performance Wins with BPF: Getting StartedPerformance Wins with BPF: Getting Started
Performance Wins with BPF: Getting Started
 
Splunk Distributed Management Console
Splunk Distributed Management Console                                         Splunk Distributed Management Console
Splunk Distributed Management Console
 
Accelerating Ceph with RDMA and NVMe-oF
Accelerating Ceph with RDMA and NVMe-oFAccelerating Ceph with RDMA and NVMe-oF
Accelerating Ceph with RDMA and NVMe-oF
 

Andere mochten auch

Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Alex Pinto
 

Andere mochten auch (11)

Cyber after Snowden (OA Cyber Summit)
Cyber after Snowden (OA Cyber Summit)Cyber after Snowden (OA Cyber Summit)
Cyber after Snowden (OA Cyber Summit)
 
Workshop - Linux Memory Analysis with Volatility
Workshop - Linux Memory Analysis with VolatilityWorkshop - Linux Memory Analysis with Volatility
Workshop - Linux Memory Analysis with Volatility
 
Creating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & VisualizationCreating Your Own Threat Intel Through Hunting & Visualization
Creating Your Own Threat Intel Through Hunting & Visualization
 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualization
 
Introduction to Python for Security Professionals
Introduction to Python for Security ProfessionalsIntroduction to Python for Security Professionals
Introduction to Python for Security Professionals
 
Workshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityWorkshop: Big Data Visualization for Security
Workshop: Big Data Visualization for Security
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
 
The Cyber Threat Intelligence Matrix
The Cyber Threat Intelligence MatrixThe Cyber Threat Intelligence Matrix
The Cyber Threat Intelligence Matrix
 
No Easy Breach DerbyCon 2016
No Easy Breach DerbyCon 2016No Easy Breach DerbyCon 2016
No Easy Breach DerbyCon 2016
 
Taking the Attacker Eviction Red Pill (v2.0)
Taking the Attacker Eviction Red Pill (v2.0)Taking the Attacker Eviction Red Pill (v2.0)
Taking the Attacker Eviction Red Pill (v2.0)
 
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
 

Ähnlich wie MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
My own sweet home!
 
AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101
Timothy Spann
 
Building high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache ThriftBuilding high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache Thrift
RX-M Enterprises LLC
 
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Timothy Spann
 

Ähnlich wie MOLOCH: Search for Full Packet Capture (OA Cyber Summit) (20)

Deploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM Bluemix
Deploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM BluemixDeploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM Bluemix
Deploy and Access WebSphere Liberty and StrongLoop REST Endpoints on IBM Bluemix
 
Introduction to Kong API Gateway
Introduction to Kong API GatewayIntroduction to Kong API Gateway
Introduction to Kong API Gateway
 
Building Content-Rich Java Apps in the Cloud with the Alfresco API
Building Content-Rich Java Apps in the Cloud with the Alfresco APIBuilding Content-Rich Java Apps in the Cloud with the Alfresco API
Building Content-Rich Java Apps in the Cloud with the Alfresco API
 
Monitor OpenStack Environments from the bottom up and front to back
Monitor OpenStack Environments from the bottom up and front to backMonitor OpenStack Environments from the bottom up and front to back
Monitor OpenStack Environments from the bottom up and front to back
 
Apache Beam @ GCPUG.TW Flink.TW 20161006
Apache Beam @ GCPUG.TW Flink.TW 20161006Apache Beam @ GCPUG.TW Flink.TW 20161006
Apache Beam @ GCPUG.TW Flink.TW 20161006
 
Rest API with Swagger and NodeJS
Rest API with Swagger and NodeJSRest API with Swagger and NodeJS
Rest API with Swagger and NodeJS
 
RESTful web APIs (build, document, manage)
RESTful web APIs (build, document, manage)RESTful web APIs (build, document, manage)
RESTful web APIs (build, document, manage)
 
Go mico
Go micoGo mico
Go mico
 
Cisco CSIRT Case Study: Forensic Investigations with NetFlow
Cisco CSIRT Case Study: Forensic Investigations with NetFlowCisco CSIRT Case Study: Forensic Investigations with NetFlow
Cisco CSIRT Case Study: Forensic Investigations with NetFlow
 
開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)開放原始碼 Ch1.2   intro - oss - apahce foundry (ver 2.0)
開放原始碼 Ch1.2 intro - oss - apahce foundry (ver 2.0)
 
Overview of Apache Flink: Next-Gen Big Data Analytics Framework
Overview of Apache Flink: Next-Gen Big Data Analytics FrameworkOverview of Apache Flink: Next-Gen Big Data Analytics Framework
Overview of Apache Flink: Next-Gen Big Data Analytics Framework
 
Integrating Alfresco with Portals
Integrating Alfresco with PortalsIntegrating Alfresco with Portals
Integrating Alfresco with Portals
 
DBCC 2021 - FLiP Stack for Cloud Data Lakes
DBCC 2021 - FLiP Stack for Cloud Data LakesDBCC 2021 - FLiP Stack for Cloud Data Lakes
DBCC 2021 - FLiP Stack for Cloud Data Lakes
 
Apache Cordova 4.x
Apache Cordova 4.xApache Cordova 4.x
Apache Cordova 4.x
 
Using the Splunk Java SDK
Using the Splunk Java SDKUsing the Splunk Java SDK
Using the Splunk Java SDK
 
Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...
Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...
Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...
 
AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101AIDevWorldApacheNiFi101
AIDevWorldApacheNiFi101
 
Building high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache ThriftBuilding high performance microservices in finance with Apache Thrift
Building high performance microservices in finance with Apache Thrift
 
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
Budapest Data/ML - Building Modern Data Streaming Apps with NiFi, Flink and K...
 
Deploying Confluent Platform for Production
Deploying Confluent Platform for ProductionDeploying Confluent Platform for Production
Deploying Confluent Platform for Production
 

Mehr von Open Analytics

Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Open Analytics
 
Using Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & PersonalizationUsing Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & Personalization
Open Analytics
 
Competing in the Digital Economy
Competing in the Digital EconomyCompeting in the Digital Economy
Competing in the Digital Economy
Open Analytics
 
Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)
Open Analytics
 
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Open Analytics
 
Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)
Open Analytics
 
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
Open Analytics
 
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Open Analytics
 
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Open Analytics
 
From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)
Open Analytics
 
Easybib Open Analytics NYC
Easybib Open Analytics NYCEasybib Open Analytics NYC
Easybib Open Analytics NYC
Open Analytics
 
MarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics MeetupMarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics Meetup
Open Analytics
 
The caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetupThe caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetup
Open Analytics
 
Verifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_finalVerifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_final
Open Analytics
 
HDScores OA DC Pitch
HDScores OA DC PitchHDScores OA DC Pitch
HDScores OA DC Pitch
Open Analytics
 

Mehr von Open Analytics (20)

Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)
Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)
Utilizing cyber intelligence to combat cyber adversaries (OA Cyber Summit)
 
CDM….Where do you start? (OA Cyber Summit)
CDM….Where do you start? (OA Cyber Summit)CDM….Where do you start? (OA Cyber Summit)
CDM….Where do you start? (OA Cyber Summit)
 
An Immigrant’s view of Cyberspace (OA Cyber Summit)
An Immigrant’s view of Cyberspace (OA Cyber Summit)An Immigrant’s view of Cyberspace (OA Cyber Summit)
An Immigrant’s view of Cyberspace (OA Cyber Summit)
 
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
Observations on CFR.org Website Traffic Surge Due to Chechnya Terrorism Scare...
 
Using Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & PersonalizationUsing Real-Time Data to Drive Optimization & Personalization
Using Real-Time Data to Drive Optimization & Personalization
 
M&A Trends in Telco Analytics
M&A Trends in Telco AnalyticsM&A Trends in Telco Analytics
M&A Trends in Telco Analytics
 
Competing in the Digital Economy
Competing in the Digital EconomyCompeting in the Digital Economy
Competing in the Digital Economy
 
Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)Piwik: An Analytics Alternative (Chicago Summit)
Piwik: An Analytics Alternative (Chicago Summit)
 
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...
 
Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)Crossing the Chasm (Ikanow - Chicago Summit)
Crossing the Chasm (Ikanow - Chicago Summit)
 
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
On the “Moneyball” – Building the Team, Product, and Service to Rival (Pegged...
 
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
Data evolutions in media, marketing, and retail (Business Adv Group - Chicago...
 
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
Characterizing Risk in your Supply Chain (nContext - Chicago Summit)
 
From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)From Insight to Impact (Chicago Summit - Keynote)
From Insight to Impact (Chicago Summit - Keynote)
 
Easybib Open Analytics NYC
Easybib Open Analytics NYCEasybib Open Analytics NYC
Easybib Open Analytics NYC
 
MarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics MeetupMarkLogic - Open Analytics Meetup
MarkLogic - Open Analytics Meetup
 
The caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetupThe caprate presentation_july2013_open analytics dc meetup
The caprate presentation_july2013_open analytics dc meetup
 
Verifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_finalVerifeed open analytics_3min deck_071713_final
Verifeed open analytics_3min deck_071713_final
 
HDScores OA DC Pitch
HDScores OA DC PitchHDScores OA DC Pitch
HDScores OA DC Pitch
 
Oas schwartz 16
Oas schwartz 16Oas schwartz 16
Oas schwartz 16
 

KĂźrzlich hochgeladen

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

KĂźrzlich hochgeladen (20)

Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 

MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

  • 2. It is a Great Horned Owl Project Logo
  • 3. WHYTHE OWL? Owls are silent hunters that go after RATs. We think that’s pretty cool. 3
  • 5. WHAT IS MOLOCH? 5 Moloch is an open source, scalable IPv4 packet capture indexing and database system, built using open source technologies. • A simple web GUI is provided for browsing, searching, viewing and exporting PCAP data. • Web APIs are accessible if you wish to design your own GUI or directly grab PCAP with various command line tools for further analysis or processing. • Find it on AOL’s GitHub page: https://github.com/aol/moloch It’s like AOL Search for PCAP repositories!
  • 6. WHAT IS MOLOCH NOT? 6 NOT IDS: NO ALERTS NOT IPV6 (Today) NOT SLOW NOT CLOSED NOT EXPENSIVE
  • 7. WHYUSE MOLOCH? 7 Real-time capture of network traffic for forensic and investigative purposes • Combine the power of Moloch with other indicators (intelligence feeds, alerting from IDS/anti-virus) to empower your analysts to quickly and effectively review actions on the network to determine the validity/threat. • Review past network traffic for post compromise investigations. Static PCAP repository • Import large collections of PCAP that were created by malware. • Import collections of PCAP from Capture The Flag events. • Custom tagging of data at time of import.
  • 8. THE PIECES OF MOLOCH 8 CAPTURE • A C application that sniffs the network interface, parses the traffic, and creates the Session Profile Information (SPI data) and writes it to disk. DATABASE • Elasticsearch is used for storing and searching through the SPI data generated by the capture component. VIEWER • A web interface that allows for GUI and API access from remote hosts to browse or query SPI data and retrieve stored PCAP.
  • 9. THE PIECES OF MOLOCH: CAPTURE 9 Libnids based daemon written in C Can be used to sniff network interface for live capture Can be called from CLI to do manual imports Parses layers 3-7 to create SPI data • Spits them out to the Elasticsearch cluster. A lot like making owl pellets!
  • 10.
  • 11. THE PIECES OF MOLOCH: DATABASE 11 Elasticsearch (http://www.elasticsearch.org) • Powered by Apache Lucene (http://lucene.apache.org) • Requests over HTTP(s) • Results returned in JSON Nosql • Network traffic doesn’t fit the mold for relational DBs. Documented oriented • Great for lots and lots of network sessions. Automatic sharding across multiple hosts • At the time, we skipped SOLR because it couldn’t run distributed. Fast, scalable, all that goodness
  • 12. THE PIECES OF MOLOCH: VIEWER 12 Node.js based application • Event driven server side JavaScript platform. • Based on Chrome’s JavaScript runtime. • Comes with its own HTTP server and easy JSON for communication. Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP. GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, and command line tools is easy. • Easy automation to retrieve PCAP or sessions of interest.
  • 13. THE PIECES OF MOLOCH: VIEWER 13 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
  • 14. THE PIECES OF MOLOCH: VIEWER 14 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
  • 17. ARCHITECTUREOF MOLOCH: SCALE 17 Packets Captured Kilobytes Saved Sessions Saved Example: Moloch Capture Documents Disk Storage (MB) Example: Elasticsearch
  • 18. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 18 IP • Source • Destination • Ports • Protocol HTTP • Method • Status Codes • Headers • Content Type DNS • IP Address • Hostnames
  • 19. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 19 SSL/TLS • Cert Elements: • Common Name • Serial Number • Alt Names SSH • Client Name • Public Key • Port IRC • Channel Name • Hostname

Hinweis der Redaktion

  1. Example of typical cover slide.