SlideShare ist ein Scribd-Unternehmen logo

Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)

•
•
Alex Pinto
Alex Pinto

Follow along with the R Markdown file at http://rpubs.com/alexcpsec/tiq-test-Summer2014-2 Source code available at: https://github.com/mlsecproject/tiq-test https://github.com/mlsecproject/tiq-test-Summer2014 https://github.com/mlsecproject/combine --------- Full Abstract: Threat Intelligence feeds are now being touted as the saving grace for SIEM and log management deployments, and as a way to supercharge incident detection and even response practices. We have heard similar promises before as an industry, so it is only fair to try to investigate. Since the actual number of breaches and attacks worldwide is unknown, it is impossible to measure how good threat intelligence feeds really are, right? Enter a new scientific breakthrough developed over the last 300 years: statistics! This presentation will consist of a data-driven analysis of a cross-section of threat intelligence feeds (both open-source and commercial) to measure their statistical bias, overlap, and representability of the unknown population of breaches worldwide. Are they a statistical good measure of the population of "bad stuff" happening out there? Is there even such a thing? How tuned to your specific threat surface are those feeds anyway? Regardless, can we actually make good use of them even if the threats they describe have no overlap with the actual incidents you have been seeing in your environment? We will provide an open-source tool for attendees to extract, normalize and export data from threat intelligence feeds to use in their internal projects and systems. It will be pre-configured with current OSINT network feed and easily extensible for private or commercial feeds. All the statistical code written and research data used (from the open-source feeds) will be made available in the spirit of reproducible research. The tool itself will be able to be used by attendees to perform the same type of tests on their own data. Join Alex and Kyle on a journey through the actual real-world usability of threat intelligence to find out which mix of open source and private feeds are right for your organization.

Daten & Analysen
1 von 42
Downloaden Sie, um offline zu lesen
Measuring  the  IQ  of  your  Threat   Intelligence  Feeds  (#TIQtest)   Alex  Pinto   MLSec  Project     @alexcpsec   @MLSecProject! Kyle  Maxwell   Researcher   @kylemaxwell!
Alex  Pinto   •  Science  guy  at  MLSec  Project   •  ML  trainer   •  Network  security  aficionado     •  Tortured  by  SIEMs  as  a  child   •  Hacker  Spirit  Animal™:   CAFFEINATED  CAPYBARA! whoami(s)   Kyle  Maxwell   •  Researcher  at  [REDACTED]   •  Math  Smuggler   •  Recovering  Incident  Responder   •  GPL  zealot   •  Hacker  Spirit  Animal™:   AXIOMATIC  ARMADILLO! (hUps://secure.flickr.com/photos/kobashi_san/)   (hUp://www.langorigami.com/art/gallery/ gallery.php?tag=mammals&name=armadillo)  
•  Threat  Intel  102   •  Measuring  Intelligence   •  Data  Preparaaon   •  Tesang  the  Data   •  Tools:   •  COMBINE   •  TIQ-­‐TEST   •  Some  parang  ideas   Agenda   (hUp://www.savagechickens.com/2008/12/iq-­‐test.html)  
Threat  Intel  102:  Capability  and   Intent   •  What  are  they  able  to  do?   •  What  are  they  intending  to  do?!
Threat  Intel  102:  Cage  Matches   •  Signatures  vs  Indicators   •  Data  vs  Intelligence   •  Tacacal  vs  Strategic   •  Atomic  vs  Composite  
Threat  Intel  102:  Pyramid  of  Pain   “Simple”  and  “easy”  aren’t  always   (David  Bianco  –  Pyramid  of  Pain)  
What  about  IP  addresses?   •  Approximately  same  value  as  hostnames  (APT  vs  DGA)   •  Finite  resource  (unal  IPv6,  that  is)   •  Managed  /  controlled  by  orgs   •  Difficulty  /  economic  incenaves  /  implied  “cost”   •  Also,  recyclable     (hUps://xkcd.com/865/)  
Given  IP  addresses  harvested  from   TI  feeds,  can  we  measure  how   much  they  “help”  our  defense   metrics?!
Introducing  TIQ-­‐TEST   •  All  these  tests  are  available  as  R  funcaons  at     •  hUps://github.com/mlsecproject/aq-­‐test   •  Have  fun,  prove  me  wrong,  suggest  stuff   •  Tools  that  implement  those  tests   •  Sample  data  +  R  Markdown  file   •  The  excuse  to  learn  a  staasacal  language  you  were  waiang   for!  
Data  Sources  –  Types  of  data   •  Extract  the  “raw”  informaaon  from  indicator  feeds   •  Both  IP  addresses  and  hostnames  were  extracted  
Data  Sources  –  Feeds  Selected   •  Data  was  separated  into  “inbound”  and  “outbound”  
Data  PreparaLon  and  Cleansing   •  Convert  the  hostname  data  to  IP  addresses:   •  Acave  IP  addresses  for  the  respecave  date  (“A”  query)   •  Passive  DNS  from  Farsight  Security  (DNSDB)   •  We  removed  non-­‐public  IPs  from  the  dataset  (RFC1918)     •  Yeah,  we  know  it  is  a  “parking  technique”   (hUps://xkcd.com/742/)  
Data  PreparaLon  and  Cleansing   •  For  each  IP  record  (including  the  ones  from  hostnames):   •  Add  asnumber  and  asname  (from  MaxMind  ASN  DB)   •  Add  country  (from  MaxMind  GeoLite  DB)   •  Add  rhost  (again  from  DNSDB)  –  most  popular  “PTR”   •  The  experiments  will  be  around  ASNs  and  Geolocaaon  
Data  PreparaLon  and  Cleansing   •  However,  we  will  NOT  be  using  maps.  Just  let  it  go.  
Data  PreparaLon  and  Cleansing   •  Small  enriched  sample:!
TesLng  the  Data   •  Let’s  generate  some  interesang  metrics:   •  NOVELTY  –  How  oxen  do  they  update  themselves?   •  OVERLAP  –  How  do  they  compare  to  what  you  got?   •  POPULATION  –  what  is  in  them  anyway?   •  Populaaon  is  tricky:   •  Could  mean  the  enare  world  (all  IPv4  space)   •  Should  ideally  mean  YOUR  world  
But  WHAT  IS  THE  IQ?!??1?   •  We  will  withhold  judgment   •  The  best  data  composiaon  is  the  best  one  for  you   •  We  will  do  our  best  to  explain  results  so  you  can  decide.   •  Maybe  on  further  (or  more  private)  research…      
Novelty  Test  –  measuring  added   and  dropped  indicators!
Novelty  Test  (1)  
Novelty  Test  (2)  
Overlap  Test  –  More  data  is  beUer,   but  make  sure  it  is  not  the  same   data!
Overlap  Test  
Overlap  Test  
Overlap  Test  
PopulaLon  Test   •  Let  us  use  the  ASN  and   GeoIP  databases  that  we   used  to  enrich  our  data  as  a   reference  of  the  “true”   populaaon.     •  But,  but,  human  beings  are   unpredictable!  We  will   never  be  able  to  forecast   this!      
Can  we  get  a  beSer  look?   •  Don’t  like  squinang  either   •  Staasacal  inference-­‐based  comparison  models   (hypothesis  tesang)   •  Exact  binomial  tests  (when  we  have  the  “true”  pop)   •  Chi-­‐squared  proporaon  tests  (similar  to   independence  tests)    
Can  we  get  a  beSer  look?   •  We  can  beUer  esamate,  with  confidence  intervals,  our   measures  of  error.   •  Also,  p-­‐values!  (with  apologies  to  Alex  HuUon)   •  We  promise  to  be  very  conservaave  in  using  them.    
Hacker  Spirit  Animal™  Guide   •  US  –  Eagle   •  CA  –  Moose   •  FR  –  Frog   •  GB  –  Bulldog   •  AU  –  Koala   •  BR  –  Capybara  /  Toucan   •  Texas  –  Armadillo   •  Disclaimer:  we  do  not  endorse  Geolocaaon-­‐based  aUribuaon  
Trend  Comparison  
Introducing  COMBINE   •  Harvesang  feeds   takes  some  work.   •  Most  of  us  let   somebody  else  do  it   without  thinking   about  what  it   actually  takes.  
Introducing  COMBINE   hUps://github.com/mlsecproject/combine  
Introducing  COMBINE   •  Components:   1.  Reaper  gathers  the  threat  data  directly  from  feeds.   2.  Thresher  normalizes  it  into  a  simplisac  data  model.   3.  Winnower  opaonally  performs  basic  validaaon  or   enrichment.   4.  Baler  transforms  the  data  into  CybOX,  CSV,  JSON,  and   CIM.  (Only  CSV  and  JSON  work  right  now).  Could  also   write  others  fairly  easily.  (nudge  nudge,  wink  wink)  
Introducing  COMBINE   •  Always  trying  to  feed   it  more.  Lots  of   possibiliaes,   including  your  own   data  sources.   •  We  clearly  do  NOT   endorse  any  included   feeds.    
Introducing  COMBINE   •  Enrichments  -­‐  think   metadata.     •  AS,  geolocaaon   •  DNS  resoluaons   courtesy  of  Farsight   DNSDB     •  Ask  them  for  an   API  key  to  test  it,  tell   them  Alex  Pinto   sent  you  ;)    
MLSec  Project   •  Both  projects  have  been  released  as  GPLv3  by  MLSec  Project   •  Will  replace  the  internal  versions  we  have  on  the  main  code     •  Looking  for  paracipants  and  data  sharing  agreements   •  Liked  TIQ-­‐TEST?  We  can  benchmark  your  private  feeds  using   these  and  other  techniques     •  Visit  hSps://www.mlsecproject.org  ,  message  @MLSecProject   or  just  e-­‐mail  me.!
Take  Aways   •  Analyze  your  data.     •  Extract  value  from  it!   •  Try  before  you  buy!  Different  test   results  mean  different  things  to   different  orgs.   •  Use  the  tools!  Suggest  new  tests!   •  Share  data  with  us!  We  take  good   care  of  it,  make  sure  it  gets   proper  exercise.    
Thanks!   •  Q&A?   •  Feedback!   Alex  Pinto     @alexcpsec   @MLSecProject   ”The  measure  of  intelligence  is  the  ability  to  change."                  -­‐  Albert  Einstein     Kyle  Maxwell   @kylemaxwell  

Empfohlen

Cyber threat intelligence: maturity and metrics
Cyber threat intelligence: maturity and metricsCyber threat intelligence: maturity and metrics
Cyber threat intelligence: maturity and metricsMark Arena
 
Knowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKKnowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKMITRE ATT&CK
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonBen Boyd
 
Leveraging MITRE ATT&CK - Speaking the Common Language
Leveraging MITRE ATT&CK - Speaking the Common LanguageLeveraging MITRE ATT&CK - Speaking the Common Language
Leveraging MITRE ATT&CK - Speaking the Common LanguageErik Van Buggenhout
 
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceMITRE - ATT&CKcon
 
Projects to Impact- Operationalizing Work from the Center
Projects to Impact- Operationalizing Work from the CenterProjects to Impact- Operationalizing Work from the Center
Projects to Impact- Operationalizing Work from the CenterMITRE ATT&CK
 
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...MITRE ATT&CK
 
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE - ATT&CKcon
 

Empfohlen

Cyber threat intelligence: maturity and metrics
Cyber threat intelligence: maturity and metricsCyber threat intelligence: maturity and metrics
Cyber threat intelligence: maturity and metricsMark Arena
 
Knowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKKnowledge for the masses: Storytelling with ATT&CK
Knowledge for the masses: Storytelling with ATT&CKMITRE ATT&CK
 
Threat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonThreat hunting - Every day is hunting season
Threat hunting - Every day is hunting seasonBen Boyd
 
Leveraging MITRE ATT&CK - Speaking the Common Language
Leveraging MITRE ATT&CK - Speaking the Common LanguageLeveraging MITRE ATT&CK - Speaking the Common Language
Leveraging MITRE ATT&CK - Speaking the Common LanguageErik Van Buggenhout
 
ATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceATTACKers Think in Graphs: Building Graphs for Threat Intelligence
ATTACKers Think in Graphs: Building Graphs for Threat IntelligenceMITRE - ATT&CKcon
 
Projects to Impact- Operationalizing Work from the Center
Projects to Impact- Operationalizing Work from the CenterProjects to Impact- Operationalizing Work from the Center
Projects to Impact- Operationalizing Work from the CenterMITRE ATT&CK
 
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...
Mapping to MITRE ATT&CK: Enhancing Operations Through the Tracking of Interac...MITRE ATT&CK
 
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...
MITRE ATT&CKcon 2018: Hunters ATT&CKing with the Data, Roberto Rodriguez, Spe...MITRE - ATT&CKcon
 
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...MITRE - ATT&CKcon
 
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™Katie Nickels
 
What is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityWhat is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityPanda Security
 
When Insiders ATT&CK!
When Insiders ATT&CK!When Insiders ATT&CK!
When Insiders ATT&CK!MITRE ATT&CK
 
The ATT&CK Philharmonic
The ATT&CK PhilharmonicThe ATT&CK Philharmonic
The ATT&CK PhilharmonicMITRE ATT&CK
 
Mapping ATT&CK Techniques to ENGAGE Activities
Mapping ATT&CK Techniques to ENGAGE ActivitiesMapping ATT&CK Techniques to ENGAGE Activities
Mapping ATT&CK Techniques to ENGAGE ActivitiesMITRE ATT&CK
 
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CKATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CKMITRE ATT&CK
 
Global Cyber Threat Intelligence
Global Cyber Threat IntelligenceGlobal Cyber Threat Intelligence
Global Cyber Threat IntelligenceNTT Innovation Institute Inc.
 
Threat Intelligence 101 - Steve Lodin - Submitted
Threat Intelligence 101 - Steve Lodin - SubmittedThreat Intelligence 101 - Steve Lodin - Submitted
Threat Intelligence 101 - Steve Lodin - SubmittedSteve Lodin
 
Threat Intelligence Workshop
Threat Intelligence WorkshopThreat Intelligence Workshop
Threat Intelligence WorkshopPriyanka Aash
 
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE - ATT&CKcon
 
MITRE ATT&CK framework
MITRE ATT&CK frameworkMITRE ATT&CK framework
MITRE ATT&CK frameworkBhushan Gurav
 
Cyber Threat hunting workshop
Cyber Threat hunting workshopCyber Threat hunting workshop
Cyber Threat hunting workshopArpan Raval
 
Cyber Threat Intelligence | Information to Insight
Cyber Threat Intelligence | Information to InsightCyber Threat Intelligence | Information to Insight
Cyber Threat Intelligence | Information to InsightDeep Shankar Yadav
 
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghCyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghOWASP Delhi
 
Welcome to the world of Cyber Threat Intelligence
Welcome to the world of Cyber Threat IntelligenceWelcome to the world of Cyber Threat Intelligence
Welcome to the world of Cyber Threat IntelligenceAndreas Sfakianakis
 
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...MITRE ATT&CK
 
Purple Teaming with ATT&CK - x33fcon 2018
Purple Teaming with ATT&CK - x33fcon 2018Purple Teaming with ATT&CK - x33fcon 2018
Purple Teaming with ATT&CK - x33fcon 2018Christopher Korban
 
Threat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceThreat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceVishal Kumar
 
The ATT&CK Latin American APT Playbook
The ATT&CK Latin American APT PlaybookThe ATT&CK Latin American APT Playbook
The ATT&CK Latin American APT PlaybookMITRE ATT&CK
 
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...Alex Pinto
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Alex Pinto
 

Weitere ähnliche Inhalte

Was ist angesagt?

MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...MITRE - ATT&CKcon
 
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™Katie Nickels
 
What is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityWhat is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityPanda Security
 
When Insiders ATT&CK!
When Insiders ATT&CK!When Insiders ATT&CK!
When Insiders ATT&CK!MITRE ATT&CK
 
The ATT&CK Philharmonic
The ATT&CK PhilharmonicThe ATT&CK Philharmonic
The ATT&CK PhilharmonicMITRE ATT&CK
 
Mapping ATT&CK Techniques to ENGAGE Activities
Mapping ATT&CK Techniques to ENGAGE ActivitiesMapping ATT&CK Techniques to ENGAGE Activities
Mapping ATT&CK Techniques to ENGAGE ActivitiesMITRE ATT&CK
 
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CKATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CKMITRE ATT&CK
 
Threat Intelligence 101 - Steve Lodin - Submitted
Threat Intelligence 101 - Steve Lodin - SubmittedThreat Intelligence 101 - Steve Lodin - Submitted
Threat Intelligence 101 - Steve Lodin - SubmittedSteve Lodin
 
Threat Intelligence Workshop
Threat Intelligence WorkshopThreat Intelligence Workshop
Threat Intelligence WorkshopPriyanka Aash
 
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE - ATT&CKcon
 
MITRE ATT&CK framework
MITRE ATT&CK frameworkMITRE ATT&CK framework
MITRE ATT&CK frameworkBhushan Gurav
 
Cyber Threat hunting workshop
Cyber Threat hunting workshopCyber Threat hunting workshop
Cyber Threat hunting workshopArpan Raval
 
Cyber Threat Intelligence | Information to Insight
Cyber Threat Intelligence | Information to InsightCyber Threat Intelligence | Information to Insight
Cyber Threat Intelligence | Information to InsightDeep Shankar Yadav
 
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghCyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghOWASP Delhi
 
Welcome to the world of Cyber Threat Intelligence
Welcome to the world of Cyber Threat IntelligenceWelcome to the world of Cyber Threat Intelligence
Welcome to the world of Cyber Threat IntelligenceAndreas Sfakianakis
 
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...MITRE ATT&CK
 
Purple Teaming with ATT&CK - x33fcon 2018
Purple Teaming with ATT&CK - x33fcon 2018Purple Teaming with ATT&CK - x33fcon 2018
Purple Teaming with ATT&CK - x33fcon 2018Christopher Korban
 
Threat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceThreat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceVishal Kumar
 
The ATT&CK Latin American APT Playbook
The ATT&CK Latin American APT PlaybookThe ATT&CK Latin American APT Playbook
The ATT&CK Latin American APT PlaybookMITRE ATT&CK
 

Was ist angesagt? (20)

MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
MITRE ATT&CKcon 2.0: ATT&CK Updates - Cyber Analytics Repository (CAR); Ivan ...
 
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
FIRST CTI Symposium: Turning intelligence into action with MITRE ATT&CK™
 
What is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda SecurityWhat is Threat Hunting? - Panda Security
What is Threat Hunting? - Panda Security
 
When Insiders ATT&CK!
When Insiders ATT&CK!When Insiders ATT&CK!
When Insiders ATT&CK!
 
The ATT&CK Philharmonic
The ATT&CK PhilharmonicThe ATT&CK Philharmonic
The ATT&CK Philharmonic
 
Mapping ATT&CK Techniques to ENGAGE Activities
Mapping ATT&CK Techniques to ENGAGE ActivitiesMapping ATT&CK Techniques to ENGAGE Activities
Mapping ATT&CK Techniques to ENGAGE Activities
 
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CKATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
ATT&CK Metaverse - Exploring the Limitations of Applying ATT&CK
 
Global Cyber Threat Intelligence
Global Cyber Threat IntelligenceGlobal Cyber Threat Intelligence
Global Cyber Threat Intelligence
 
Threat Intelligence 101 - Steve Lodin - Submitted
Threat Intelligence 101 - Steve Lodin - SubmittedThreat Intelligence 101 - Steve Lodin - Submitted
Threat Intelligence 101 - Steve Lodin - Submitted
 
Threat Intelligence Workshop
Threat Intelligence WorkshopThreat Intelligence Workshop
Threat Intelligence Workshop
 
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITREMITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
MITRE ATT&CKcon 2.0: State of the ATT&CK; Blake Strom, MITRE
 
MITRE ATT&CK framework
MITRE ATT&CK frameworkMITRE ATT&CK framework
MITRE ATT&CK framework
 
Cyber Threat hunting workshop
Cyber Threat hunting workshopCyber Threat hunting workshop
Cyber Threat hunting workshop
 
Cyber Threat Intelligence | Information to Insight
Cyber Threat Intelligence | Information to InsightCyber Threat Intelligence | Information to Insight
Cyber Threat Intelligence | Information to Insight
 
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep SinghCyber threat Intelligence and Incident Response by:-Sandeep Singh
Cyber threat Intelligence and Incident Response by:-Sandeep Singh
 
Welcome to the world of Cyber Threat Intelligence
Welcome to the world of Cyber Threat IntelligenceWelcome to the world of Cyber Threat Intelligence
Welcome to the world of Cyber Threat Intelligence
 
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
Would you Rather Have Telemetry into 2 Attacks or 20? An Insight Into Highly ...
 
Purple Teaming with ATT&CK - x33fcon 2018
Purple Teaming with ATT&CK - x33fcon 2018Purple Teaming with ATT&CK - x33fcon 2018
Purple Teaming with ATT&CK - x33fcon 2018
 
Threat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement MatriceThreat Hunting Procedures and Measurement Matrice
Threat Hunting Procedures and Measurement Matrice
 
The ATT&CK Latin American APT Playbook
The ATT&CK Latin American APT PlaybookThe ATT&CK Latin American APT Playbook
The ATT&CK Latin American APT Playbook
 

Andere mochten auch

From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...Alex Pinto
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Alex Pinto
 
BSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityBSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityAlex Pinto
 
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Alex Pinto
 
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and SharingData-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and SharingAlex Pinto
 
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionBeyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionAlex Pinto
 
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitianPoc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitianLiang Chen
 
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...North Texas Chapter of the ISSA
 
11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)Nigel Simmons
 
11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)Nigel Simmons
 
Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)Brent Muir
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Alex Pinto
 
Tarea seminario 3
Tarea seminario 3Tarea seminario 3
Tarea seminario 3carsuro25
 
11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)Nigel Simmons
 
The beauty of mathematics
The beauty of mathematicsThe beauty of mathematics
The beauty of mathematicsTarun Gehlot
 
Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information? Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information? Control Risks
 
Finite elements : basis functions
Finite elements : basis functionsFinite elements : basis functions
Finite elements : basis functionsTarun Gehlot
 
Graphing inverse functions
Graphing inverse functionsGraphing inverse functions
Graphing inverse functionsTarun Gehlot
 
Finite elements for 2‐d problems
Finite elements for 2‐d problemsFinite elements for 2‐d problems
Finite elements for 2‐d problemsTarun Gehlot
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligencemohamed nasri
 

Andere mochten auch (20)

From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
 
BSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityBSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information Security
 
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
 
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and SharingData-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
Data-Driven Threat Intelligence: Metrics on Indicator Dissemination and Sharing
 
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionBeyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
 
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitianPoc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
Poc2015 os x_kernel_is_as_strong_as_its_weakest_part_liang_shuaitian
 
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
NTXISSACSC2 - The Role of Threat Intelligence and Layered Security for Intrus...
 
11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)11 x1 t16 05 volumes (2013)
11 x1 t16 05 volumes (2013)
 
11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)11 x1 t01 03 factorising (2014)
11 x1 t01 03 factorising (2014)
 
Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)Ducky USB - Indicators of Compromise (IOCs)
Ducky USB - Indicators of Compromise (IOCs)
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
 
Tarea seminario 3
Tarea seminario 3Tarea seminario 3
Tarea seminario 3
 
11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)11 x1 t16 02 definite integral (2013)
11 x1 t16 02 definite integral (2013)
 
The beauty of mathematics
The beauty of mathematicsThe beauty of mathematics
The beauty of mathematics
 
Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information? Cyber Threat Intelligence: Who is Targeting your Information?
Cyber Threat Intelligence: Who is Targeting your Information?
 
Finite elements : basis functions
Finite elements : basis functionsFinite elements : basis functions
Finite elements : basis functions
 
Graphing inverse functions
Graphing inverse functionsGraphing inverse functions
Graphing inverse functions
 
Finite elements for 2‐d problems
Finite elements for 2‐d problemsFinite elements for 2‐d problems
Finite elements for 2‐d problems
 
Cyber Threat Intelligence
Cyber Threat IntelligenceCyber Threat Intelligence
Cyber Threat Intelligence
 

Ähnlich wie Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)

Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...OpenSource Connections
 
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014The Hive
 
Agile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics ApplicationsAgile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics ApplicationsDataWorks Summit
 
Agile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics ApplicationsAgile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics ApplicationsRussell Jurney
 
BinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in HadoopBinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in HadoopJason Trost
 
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...Big Data Spain
 
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with DaskAUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with DaskVíctor Zabalza
 
Agile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics ApplicationsAgile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics ApplicationsRussell Jurney
 
The Background Noise of the Internet
The Background Noise of the InternetThe Background Noise of the Internet
The Background Noise of the InternetAndrew Morris
 
Protect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesProtect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesLeo Loobeek
 
10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everything10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everythingRyan Kovar
 
Data Science meets Software Development
Data Science meets Software DevelopmentData Science meets Software Development
Data Science meets Software DevelopmentAlexis Seigneurin
 
Searching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data WorldSearching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data WorldOpenSource Connections
 
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwonThe basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwonKenneth Kwon
 
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsSDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsKorea Sdec
 
Ncku csie talk about Spark
Ncku csie talk about SparkNcku csie talk about Spark
Ncku csie talk about SparkGiivee The
 
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...Lucidworks (Archived)
 
Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...DataWorks Summit/Hadoop Summit
 
NotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for PentestersNotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for PentestersRob Fuller
 

Ähnlich wie Measuring the IQ of your Threat Intelligence Feeds (#tiqtest) (20)

Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
Building a Lightweight Discovery Interface for China's Patents@NYC Solr/Lucen...
 
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014
 
Agile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics ApplicationsAgile Data: Building Hadoop Analytics Applications
Agile Data: Building Hadoop Analytics Applications
 
Agile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics ApplicationsAgile Data Science: Building Hadoop Analytics Applications
Agile Data Science: Building Hadoop Analytics Applications
 
BinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in HadoopBinaryPig - Scalable Malware Analytics in Hadoop
BinaryPig - Scalable Malware Analytics in Hadoop
 
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...
 
Data Science
Data ScienceData Science
Data Science
 
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with DaskAUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
AUTOMATED DATA EXPLORATION - Building efficient analysis pipelines with Dask
 
Agile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics ApplicationsAgile Data Science: Hadoop Analytics Applications
Agile Data Science: Hadoop Analytics Applications
 
The Background Noise of the Internet
The Background Noise of the InternetThe Background Noise of the Internet
The Background Noise of the Internet
 
Protect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesProtect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying Techniques
 
10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everything10(?) holiday gifts for the SOC who has everything
10(?) holiday gifts for the SOC who has everything
 
Data Science meets Software Development
Data Science meets Software DevelopmentData Science meets Software Development
Data Science meets Software Development
 
Searching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data WorldSearching Chinese Patents Presentation at Enterprise Data World
Searching Chinese Patents Presentation at Enterprise Data World
 
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwonThe basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
The basics of hacking and penetration testing 이제 시작이야 해킹과 침투 테스트 kenneth.s.kwon
 
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and modelsSDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and models
 
Ncku csie talk about Spark
Ncku csie talk about SparkNcku csie talk about Spark
Ncku csie talk about Spark
 
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
Building a Lightweight Discovery Interface for Chinese Patents, Presented by ...
 
Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...Data infrastructure architecture for medium size organization: tips for colle...
Data infrastructure architecture for medium size organization: tips for colle...
 
NotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for PentestersNotaCon 2011 - Networking for Pentesters
NotaCon 2011 - Networking for Pentesters
 

Kürzlich hochgeladen

1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样vhwb25kk
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024thyngster
 
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPTPredictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPTBoston Institute of Analytics
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...limedy534
 
Defining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data StoryDefining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data StoryJeremy Anderson
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Colleen Farrelly
 
Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxEmmanuel Dauda
 
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdfHuman37
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFAAndrei Kaleshka
 
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...dajasot375
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfBoston Institute of Analytics
 
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptdokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptSonatrach
 
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdfKantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdfSocial Samosa
 
Multiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdfMultiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdfchwongval
 
Heart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectHeart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectBoston Institute of Analytics
 
RadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdfRadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdfgstagge
 
MK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docxMK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docxUnduhUnggah1
 
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSINGmarianagonzalez07
 

Kürzlich hochgeladen (20)

1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
 
E-Commerce Order PredictionShraddha Kamble.pptx
E-Commerce Order PredictionShraddha Kamble.pptxE-Commerce Order PredictionShraddha Kamble.pptx
E-Commerce Order PredictionShraddha Kamble.pptx
 
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPTPredictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
 
Defining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data StoryDefining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data Story
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024
 
Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptx
 
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFA
 
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
 
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptdokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
 
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdfKantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
 
Multiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdfMultiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdf
 
Heart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectHeart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis Project
 
RadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdfRadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdf
 
MK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docxMK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docx
 
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
 

Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)

  • 1. Measuring  the  IQ  of  your  Threat   Intelligence  Feeds  (#TIQtest)   Alex  Pinto   MLSec  Project     @alexcpsec   @MLSecProject! Kyle  Maxwell   Researcher   @kylemaxwell!
  • 2. Alex  Pinto   •  Science  guy  at  MLSec  Project   •  ML  trainer   •  Network  security  aficionado     •  Tortured  by  SIEMs  as  a  child   •  Hacker  Spirit  Animal™:   CAFFEINATED  CAPYBARA! whoami(s)   Kyle  Maxwell   •  Researcher  at  [REDACTED]   •  Math  Smuggler   •  Recovering  Incident  Responder   •  GPL  zealot   •  Hacker  Spirit  Animal™:   AXIOMATIC  ARMADILLO! (hUps://secure.flickr.com/photos/kobashi_san/)   (hUp://www.langorigami.com/art/gallery/ gallery.php?tag=mammals&name=armadillo)  
  • 3. •  Threat  Intel  102   •  Measuring  Intelligence   •  Data  Preparaaon   •  Tesang  the  Data   •  Tools:   •  COMBINE   •  TIQ-­‐TEST   •  Some  parang  ideas   Agenda   (hUp://www.savagechickens.com/2008/12/iq-­‐test.html)  
  • 4. Threat  Intel  102:  Capability  and   Intent   •  What  are  they  able  to  do?   •  What  are  they  intending  to  do?!
  • 5. Threat  Intel  102:  Cage  Matches   •  Signatures  vs  Indicators   •  Data  vs  Intelligence   •  Tacacal  vs  Strategic   •  Atomic  vs  Composite  
  • 6. Threat  Intel  102:  Pyramid  of  Pain   “Simple”  and  “easy”  aren’t  always   (David  Bianco  –  Pyramid  of  Pain)  
  • 7. What  about  IP  addresses?   •  Approximately  same  value  as  hostnames  (APT  vs  DGA)   •  Finite  resource  (unal  IPv6,  that  is)   •  Managed  /  controlled  by  orgs   •  Difficulty  /  economic  incenaves  /  implied  “cost”   •  Also,  recyclable     (hUps://xkcd.com/865/)  
  • 8. Given  IP  addresses  harvested  from   TI  feeds,  can  we  measure  how   much  they  “help”  our  defense   metrics?!
  • 9. Introducing  TIQ-­‐TEST   •  All  these  tests  are  available  as  R  funcaons  at     •  hUps://github.com/mlsecproject/aq-­‐test   •  Have  fun,  prove  me  wrong,  suggest  stuff   •  Tools  that  implement  those  tests   •  Sample  data  +  R  Markdown  file   •  The  excuse  to  learn  a  staasacal  language  you  were  waiang   for!  
  • 10. Data  Sources  –  Types  of  data   •  Extract  the  “raw”  informaaon  from  indicator  feeds   •  Both  IP  addresses  and  hostnames  were  extracted  
  • 11. Data  Sources  –  Feeds  Selected   •  Data  was  separated  into  “inbound”  and  “outbound”  
  • 12. Data  PreparaLon  and  Cleansing   •  Convert  the  hostname  data  to  IP  addresses:   •  Acave  IP  addresses  for  the  respecave  date  (“A”  query)   •  Passive  DNS  from  Farsight  Security  (DNSDB)   •  We  removed  non-­‐public  IPs  from  the  dataset  (RFC1918)     •  Yeah,  we  know  it  is  a  “parking  technique”   (hUps://xkcd.com/742/)  
  • 13. Data  PreparaLon  and  Cleansing   •  For  each  IP  record  (including  the  ones  from  hostnames):   •  Add  asnumber  and  asname  (from  MaxMind  ASN  DB)   •  Add  country  (from  MaxMind  GeoLite  DB)   •  Add  rhost  (again  from  DNSDB)  –  most  popular  “PTR”   •  The  experiments  will  be  around  ASNs  and  Geolocaaon  
  • 14. Data  PreparaLon  and  Cleansing   •  However,  we  will  NOT  be  using  maps.  Just  let  it  go.  
  • 15. Data  PreparaLon  and  Cleansing   •  Small  enriched  sample:!
  • 16. TesLng  the  Data   •  Let’s  generate  some  interesang  metrics:   •  NOVELTY  –  How  oxen  do  they  update  themselves?   •  OVERLAP  –  How  do  they  compare  to  what  you  got?   •  POPULATION  –  what  is  in  them  anyway?   •  Populaaon  is  tricky:   •  Could  mean  the  enare  world  (all  IPv4  space)   •  Should  ideally  mean  YOUR  world  
  • 17. But  WHAT  IS  THE  IQ?!??1?   •  We  will  withhold  judgment   •  The  best  data  composiaon  is  the  best  one  for  you   •  We  will  do  our  best  to  explain  results  so  you  can  decide.   •  Maybe  on  further  (or  more  private)  research…      
  • 18. Novelty  Test  –  measuring  added   and  dropped  indicators!
  • 21. Overlap  Test  –  More  data  is  beUer,   but  make  sure  it  is  not  the  same   data!
  • 25. PopulaLon  Test   •  Let  us  use  the  ASN  and   GeoIP  databases  that  we   used  to  enrich  our  data  as  a   reference  of  the  “true”   populaaon.     •  But,  but,  human  beings  are   unpredictable!  We  will   never  be  able  to  forecast   this!      
  • 26.
  • 27.
  • 28. Can  we  get  a  beSer  look?   •  Don’t  like  squinang  either   •  Staasacal  inference-­‐based  comparison  models   (hypothesis  tesang)   •  Exact  binomial  tests  (when  we  have  the  “true”  pop)   •  Chi-­‐squared  proporaon  tests  (similar  to   independence  tests)    
  • 29. Can  we  get  a  beSer  look?   •  We  can  beUer  esamate,  with  confidence  intervals,  our   measures  of  error.   •  Also,  p-­‐values!  (with  apologies  to  Alex  HuUon)   •  We  promise  to  be  very  conservaave  in  using  them.    
  • 30.
  • 31. Hacker  Spirit  Animal™  Guide   •  US  –  Eagle   •  CA  –  Moose   •  FR  –  Frog   •  GB  –  Bulldog   •  AU  –  Koala   •  BR  –  Capybara  /  Toucan   •  Texas  –  Armadillo   •  Disclaimer:  we  do  not  endorse  Geolocaaon-­‐based  aUribuaon  
  • 33.
  • 34.
  • 35. Introducing  COMBINE   •  Harvesang  feeds   takes  some  work.   •  Most  of  us  let   somebody  else  do  it   without  thinking   about  what  it   actually  takes.  
  • 37. Introducing  COMBINE   •  Components:   1.  Reaper  gathers  the  threat  data  directly  from  feeds.   2.  Thresher  normalizes  it  into  a  simplisac  data  model.   3.  Winnower  opaonally  performs  basic  validaaon  or   enrichment.   4.  Baler  transforms  the  data  into  CybOX,  CSV,  JSON,  and   CIM.  (Only  CSV  and  JSON  work  right  now).  Could  also   write  others  fairly  easily.  (nudge  nudge,  wink  wink)  
  • 38. Introducing  COMBINE   •  Always  trying  to  feed   it  more.  Lots  of   possibiliaes,   including  your  own   data  sources.   •  We  clearly  do  NOT   endorse  any  included   feeds.    
  • 39. Introducing  COMBINE   •  Enrichments  -­‐  think   metadata.     •  AS,  geolocaaon   •  DNS  resoluaons   courtesy  of  Farsight   DNSDB     •  Ask  them  for  an   API  key  to  test  it,  tell   them  Alex  Pinto   sent  you  ;)    
  • 40. MLSec  Project   •  Both  projects  have  been  released  as  GPLv3  by  MLSec  Project   •  Will  replace  the  internal  versions  we  have  on  the  main  code     •  Looking  for  paracipants  and  data  sharing  agreements   •  Liked  TIQ-­‐TEST?  We  can  benchmark  your  private  feeds  using   these  and  other  techniques     •  Visit  hSps://www.mlsecproject.org  ,  message  @MLSecProject   or  just  e-­‐mail  me.!
  • 41. Take  Aways   •  Analyze  your  data.     •  Extract  value  from  it!   •  Try  before  you  buy!  Different  test   results  mean  different  things  to   different  orgs.   •  Use  the  tools!  Suggest  new  tests!   •  Share  data  with  us!  We  take  good   care  of  it,  make  sure  it  gets   proper  exercise.    
  • 42. Thanks!   •  Q&A?   •  Feedback!   Alex  Pinto     @alexcpsec   @MLSecProject   ”The  measure  of  intelligence  is  the  ability  to  change."                  -­‐  Albert  Einstein     Kyle  Maxwell   @kylemaxwell