SlideShare a Scribd company logo
1 of 8
Anomaly Detection
- S A L IL NAVG IR E
Introduction
• problem of finding patterns in data that do not
conform to expected behavior
• covers diverse disciplines from statistics, machine
learning, data mining, information theory, spectral
theory
Applications
• Intrusion detection- detection of malicious activity
• Host based – OS call traces
• Network based – packet level traces

• Fraud detection - detection of criminal activities in
commercial organizations
• Credit card fraud detection

• Insurance Claim Fraud Detection
• Insider trading detection

• Industrial damage detection
• Anomaly detection in data

• Anomaly detection in sensor networks
Challenges
• Defining normal region

• Sometimes malicious agent adapt themselves to
appear as normal observation
• Different techniques for different application
domain
• Availability of labeled data for training
• Sometimes noise is similar to anomaly and difficult
to distinguish
Different aspects of detection
techniques
• Nature of input data

• Types of Anomaly
• Point Anomalies
• Contextual Anomalies
• Collective Anomalies

• Data Labels
• Supervised anomaly detection
• Semi-Supervised anomaly detection
• Unsupervised anomaly detection

• Output
• Scores
• Labels
Anomaly Detection Techniques

Anomaly
detection
techniques

Classification

Nearest
Neighbor

Clustering

Spectral

Information
theoretic

Statistical

Time Series
• Classification
• Neural network based
• Bayesian Network based
• Support Vector Machine based

• Rule based

• Nearest Neighbor
• KNN

• Relative density

• Clustering
• K means
• SOM
• Statistical
• Parametric
• Gaussian model based
• Regression model based
• Mixture of parametric distributions based

• Non-parametric
• Histogram based
• Kernel function based

• Spectral
• Dimensionality reduction

More Related Content

Similar to Anomaly Detection

Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationImpetus Technologies
 
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...Techsylvania
 
Cybersecurity Basics - Aravindr.com
Cybersecurity Basics - Aravindr.comCybersecurity Basics - Aravindr.com
Cybersecurity Basics - Aravindr.comAravind R
 
intrusion detection system (IDS)
intrusion detection system (IDS)intrusion detection system (IDS)
intrusion detection system (IDS)Aj Maurya
 
Anomalies and events keep us on our toes
Anomalies and events keep us on our toesAnomalies and events keep us on our toes
Anomalies and events keep us on our toesCSIRO
 
Forensics intro
Forensics introForensics intro
Forensics introtest tt
 
Trends in electronic crimes and its impact on businesses like yours
Trends in electronic crimes and its impact on businesses like yoursTrends in electronic crimes and its impact on businesses like yours
Trends in electronic crimes and its impact on businesses like yoursMotherGuardians
 
Penetration testing & Ethical Hacking
Penetration testing & Ethical HackingPenetration testing & Ethical Hacking
Penetration testing & Ethical HackingS.E. CTS CERT-GOV-MD
 
Anomaly detection (Unsupervised Learning) in Machine Learning
Anomaly detection (Unsupervised Learning) in Machine LearningAnomaly detection (Unsupervised Learning) in Machine Learning
Anomaly detection (Unsupervised Learning) in Machine LearningKuppusamy P
 
An Introduction to Anomaly Detection
An Introduction to Anomaly DetectionAn Introduction to Anomaly Detection
An Introduction to Anomaly DetectionKenneth Graham
 
OTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection MethodsOTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection MethodsTECHdotMN
 
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...EC-Council
 
Network Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptxNetwork Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptxtalkaton
 
Network Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdfNetwork Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdftalkaton
 
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
Finding the needle in the haystack: how Nestle is leveraging big data to defe...Finding the needle in the haystack: how Nestle is leveraging big data to defe...
Finding the needle in the haystack: how Nestle is leveraging big data to defe...Big Data Spain
 
Network traffic analysis with cyber security
Network traffic analysis with cyber securityNetwork traffic analysis with cyber security
Network traffic analysis with cyber securityKAMALI PRIYA P
 
SECURITY OPERATION CENTER CONTENT.pptx
SECURITY OPERATION CENTER CONTENT.pptxSECURITY OPERATION CENTER CONTENT.pptx
SECURITY OPERATION CENTER CONTENT.pptxFarzanMansoor1
 

Similar to Anomaly Detection (20)

Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
 
ids.ppt
ids.pptids.ppt
ids.ppt
 
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
Karel Obluk (Evolution Equity Partners) - Cybersecurity: Challenges and Oppor...
 
Cybersecurity Basics - Aravindr.com
Cybersecurity Basics - Aravindr.comCybersecurity Basics - Aravindr.com
Cybersecurity Basics - Aravindr.com
 
intrusion detection system (IDS)
intrusion detection system (IDS)intrusion detection system (IDS)
intrusion detection system (IDS)
 
Anomalies and events keep us on our toes
Anomalies and events keep us on our toesAnomalies and events keep us on our toes
Anomalies and events keep us on our toes
 
Forensics intro
Forensics introForensics intro
Forensics intro
 
Trends in electronic crimes and its impact on businesses like yours
Trends in electronic crimes and its impact on businesses like yoursTrends in electronic crimes and its impact on businesses like yours
Trends in electronic crimes and its impact on businesses like yours
 
Penetration testing & Ethical Hacking
Penetration testing & Ethical HackingPenetration testing & Ethical Hacking
Penetration testing & Ethical Hacking
 
Anomaly detection (Unsupervised Learning) in Machine Learning
Anomaly detection (Unsupervised Learning) in Machine LearningAnomaly detection (Unsupervised Learning) in Machine Learning
Anomaly detection (Unsupervised Learning) in Machine Learning
 
An Introduction to Anomaly Detection
An Introduction to Anomaly DetectionAn Introduction to Anomaly Detection
An Introduction to Anomaly Detection
 
OTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection MethodsOTC Data Mining Anomaly Detection Methods
OTC Data Mining Anomaly Detection Methods
 
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
TakeDownCon Rocket City: Research Advancements Towards Protecting Critical As...
 
Network Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptxNetwork Analysis Mini Project 2.pptx
Network Analysis Mini Project 2.pptx
 
Network Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdfNetwork Analysis Mini Project 2.pdf
Network Analysis Mini Project 2.pdf
 
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
Finding the needle in the haystack: how Nestle is leveraging big data to defe...Finding the needle in the haystack: how Nestle is leveraging big data to defe...
Finding the needle in the haystack: how Nestle is leveraging big data to defe...
 
Network traffic analysis with cyber security
Network traffic analysis with cyber securityNetwork traffic analysis with cyber security
Network traffic analysis with cyber security
 
SECURITY OPERATION CENTER CONTENT.pptx
SECURITY OPERATION CENTER CONTENT.pptxSECURITY OPERATION CENTER CONTENT.pptx
SECURITY OPERATION CENTER CONTENT.pptx
 
Security audit
Security auditSecurity audit
Security audit
 
Security Audit
Security AuditSecurity Audit
Security Audit
 

Recently uploaded

2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsRoshan Dwivedi
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesBoston Institute of Analytics
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024SynarionITSolutions
 

Recently uploaded (20)

2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 

Anomaly Detection

  • 1. Anomaly Detection - S A L IL NAVG IR E
  • 2. Introduction • problem of finding patterns in data that do not conform to expected behavior • covers diverse disciplines from statistics, machine learning, data mining, information theory, spectral theory
  • 3. Applications • Intrusion detection- detection of malicious activity • Host based – OS call traces • Network based – packet level traces • Fraud detection - detection of criminal activities in commercial organizations • Credit card fraud detection • Insurance Claim Fraud Detection • Insider trading detection • Industrial damage detection • Anomaly detection in data • Anomaly detection in sensor networks
  • 4. Challenges • Defining normal region • Sometimes malicious agent adapt themselves to appear as normal observation • Different techniques for different application domain • Availability of labeled data for training • Sometimes noise is similar to anomaly and difficult to distinguish
  • 5. Different aspects of detection techniques • Nature of input data • Types of Anomaly • Point Anomalies • Contextual Anomalies • Collective Anomalies • Data Labels • Supervised anomaly detection • Semi-Supervised anomaly detection • Unsupervised anomaly detection • Output • Scores • Labels
  • 7. • Classification • Neural network based • Bayesian Network based • Support Vector Machine based • Rule based • Nearest Neighbor • KNN • Relative density • Clustering • K means • SOM
  • 8. • Statistical • Parametric • Gaussian model based • Regression model based • Mixture of parametric distributions based • Non-parametric • Histogram based • Kernel function based • Spectral • Dimensionality reduction