SlideShare a Scribd company logo
1 of 8
Anomaly Detection
- S A L IL NAVG IR E
Introduction
• problem of finding patterns in data that do not
conform to expected behavior
• covers diverse disciplines from statistics, machine
learning, data mining, information theory, spectral
theory
Applications
• Intrusion detection- detection of malicious activity
• Host based – OS call traces
• Network based – packet level traces

• Fraud detection - detection of criminal activities in
commercial organizations
• Credit card fraud detection

• Insurance Claim Fraud Detection
• Insider trading detection

• Industrial damage detection
• Anomaly detection in data

• Anomaly detection in sensor networks
Challenges
• Defining normal region

• Sometimes malicious agent adapt themselves to
appear as normal observation
• Different techniques for different application
domain
• Availability of labeled data for training
• Sometimes noise is similar to anomaly and difficult
to distinguish
Different aspects of detection
techniques
• Nature of input data

• Types of Anomaly
• Point Anomalies
• Contextual Anomalies
• Collective Anomalies

• Data Labels
• Supervised anomaly detection
• Semi-Supervised anomaly detection
• Unsupervised anomaly detection

• Output
• Scores
• Labels
Anomaly Detection Techniques

Anomaly
detection
techniques

Classification

Nearest
Neighbor

Clustering

Spectral

Information
theoretic

Statistical

Time Series
• Classification
• Neural network based
• Bayesian Network based
• Support Vector Machine based

• Rule based

• Nearest Neighbor
• KNN

• Relative density

• Clustering
• K means
• SOM
• Statistical
• Parametric
• Gaussian model based
• Regression model based
• Mixture of parametric distributions based

• Non-parametric
• Histogram based
• Kernel function based

• Spectral
• Dimensionality reduction

More Related Content

Similar to Anomaly Detection

Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Techsylvania
 
Forensics intro
Forensics introForensics intro
Forensics intro
test tt
 
OTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection MethodsOTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection Methods
TECHdotMN
 
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
EC-Council
 
Network Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptxNetwork Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptx
talkaton
 
Network Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdfNetwork Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdf
talkaton
 

Similar to Anomaly Detection (20)

Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
 
ids.ppt
ids.pptids.ppt
ids.ppt
 
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
 
Cybersecurity Basics - Aravindr.com
Cybersecurity Basics - Aravindr.comCybersecurity Basics - Aravindr.com
Cybersecurity Basics - Aravindr.com
 
intrusion detection system (IDS)
intrusion detection system (IDS)intrusion detection system (IDS)
intrusion detection system (IDS)
 
Anomalies and events keep us on our toes
Anomalies and events keep us on our toesAnomalies and events keep us on our toes
Anomalies and events keep us on our toes
 
Forensics intro
Forensics introForensics intro
Forensics intro
 
Trends in electronic crimes and its impact on businesses like yours
Trends in electronic crimes and its impact on businesses like yoursTrends in electronic crimes and its impact on businesses like yours
Trends in electronic crimes and its impact on businesses like yours
 
Penetration testing & Ethical Hacking
Penetration testing & Ethical HackingPenetration testing & Ethical Hacking
Penetration testing & Ethical Hacking
 
Anomaly detection (Unsupervised Learning) in Machine Learning
Anomaly detection (Unsupervised Learning) in Machine LearningAnomaly detection (Unsupervised Learning) in Machine Learning
Anomaly detection (Unsupervised Learning) in Machine Learning
 
An Introduction to Anomaly Detection
An Introduction to Anomaly DetectionAn Introduction to Anomaly Detection
An Introduction to Anomaly Detection
 
OTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection MethodsOTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection Methods
 
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
 
Network Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptxNetwork Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptx
 
Network Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdfNetwork Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdf
 
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
Finding the needle in the haystack: how Nestle is leveraging big data to defe...Finding the needle in the haystack: how Nestle is leveraging big data to defe...
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
 
Network traffic analysis with cyber security
Network traffic analysis with cyber securityNetwork traffic analysis with cyber security
Network traffic analysis with cyber security
 
SECURITY OPERATION CENTER CONTENT.pptx
SECURITY OPERATION CENTER CONTENT.pptxSECURITY OPERATION CENTER CONTENT.pptx
SECURITY OPERATION CENTER CONTENT.pptx
 
Security audit
Security auditSecurity audit
Security audit
 
Security Audit
Security AuditSecurity Audit
Security Audit
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Recently uploaded (20)

DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 

Anomaly Detection

  • 1. Anomaly Detection - S A L IL NAVG IR E
  • 2. Introduction • problem of finding patterns in data that do not conform to expected behavior • covers diverse disciplines from statistics, machine learning, data mining, information theory, spectral theory
  • 3. Applications • Intrusion detection- detection of malicious activity • Host based – OS call traces • Network based – packet level traces • Fraud detection - detection of criminal activities in commercial organizations • Credit card fraud detection • Insurance Claim Fraud Detection • Insider trading detection • Industrial damage detection • Anomaly detection in data • Anomaly detection in sensor networks
  • 4. Challenges • Defining normal region • Sometimes malicious agent adapt themselves to appear as normal observation • Different techniques for different application domain • Availability of labeled data for training • Sometimes noise is similar to anomaly and difficult to distinguish
  • 5. Different aspects of detection techniques • Nature of input data • Types of Anomaly • Point Anomalies • Contextual Anomalies • Collective Anomalies • Data Labels • Supervised anomaly detection • Semi-Supervised anomaly detection • Unsupervised anomaly detection • Output • Scores • Labels
  • 7. • Classification • Neural network based • Bayesian Network based • Support Vector Machine based • Rule based • Nearest Neighbor • KNN • Relative density • Clustering • K means • SOM
  • 8. • Statistical • Parametric • Gaussian model based • Regression model based • Mixture of parametric distributions based • Non-parametric • Histogram based • Kernel function based • Spectral • Dimensionality reduction