Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Anomaly Detection using
Deep Auto-Encoders
GIANMARIO SPACAGNA
DATA SCIENCE MILAN - 18/05/2017
What you will (briefly) learn
▶ What is an anomaly (and an outlier)
▶ Popular techniques used in shallow machine
learning
...
1. Machine Learning – An
Introduction
2. Neural Networks
3. Deep Learning
Fundamentals
4. Unsupervised Feature
Learning
5....
Why this use case?
▶ Anomaly detection is crucial to many business
applications
▶ Smart feature representation => better a...
Outlier vs Anomaly
“An outlier is a legitimate data point that’s far
away from the mean or median in a distribution. It
ma...
Data modeling
▶ Point anomaly
(e.g. black sheep)
■ Contextual
anomaly
(e.g. selling ice-
creams in
January)
■ Collective
a...
Detection modeling (and its
limitations)
▶ Supervised (classification)
▶ Data skewness, lack of
counter examples
▶ Unsuper...
Real world applications
▶ Manufacturing => hardware faults
▶ Law-enforcement => reveal criminal activities
▶ Network syste...
What’s the challenge?
“Coming up with features is difficult, time-
consuming, requires expert knowledge.
When working appl...
Hierarchical Feature Learning
NVIDIA Deep Learning Course: Class #1 – Introduction to Deep Learning
https://www.youtube.co...
Structural representation
Advanced Topics, http://slideplayer.com/slide/3471890/
Signal propagation
Schematic diagram of back-propagation neural networks with two hidden layers.
Factor selection for dela...
Auto-encoders
• Signal propagation output: approximate an identity function
• Error back propagation: Mean Squared Error M...
Novelty detection using auto-encoders
1. Identify a training dataset of what is considered “normal”
2. Learn what “normal”...
Features compression
■ Use just the encoder to compress data
into a reduced dimensional space then
use traditional unsuper...
PoC examples
▶ ECG Anomaly Pulse Detection
▶ MNIST Anomaly Digit Recognition
(Optional)
▶ Jupyter notebooks available on
h...
Summary
▶ We listed a few real-world applications of anomaly
detection
▶ We covered some of the most popular techniques in...
Going deeper
▶ Advanced modeling:
▶ Denoising auto-encoders
▶ Contractive auto-encoders
▶ Sparse auto-encoders
▶ Variation...
E-book discount
▶ Use the code KVGRSF30
and get 30% discount on e-
book
▶ Only valid for 500 uses
until 31st October, 2017...
"Data scientists realize that their best days
coincide with discovery of truly odd features in
the data."
Haystacks and Ne...
Deep Neural networks
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Nächste SlideShare
Wird geladen in …5
×

Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna

3.873 Aufrufe

Veröffentlicht am

One of the determinants for a good anomaly detector is finding smart data representations that can easily evince deviations from the normal distribution. Traditional supervised approaches would require a strong assumption about what is normal and what not plus a non negligible effort in labeling the training dataset. Deep auto-encoders work very well in learning high-level abstractions and non-linear relationships of the data without requiring data labels. In this talk we will review a few popular techniques used in shallow machine learning and propose two semi-supervised approaches for novelty detection: one based on reconstruction error and another based on lower-dimensional feature compression.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna

  1. 1. Anomaly Detection using Deep Auto-Encoders GIANMARIO SPACAGNA DATA SCIENCE MILAN - 18/05/2017
  2. 2. What you will (briefly) learn ▶ What is an anomaly (and an outlier) ▶ Popular techniques used in shallow machine learning ▶ Why deep learning can make the difference ▶ Anomaly detection using deep auto— encoders ▶ H2O overview ▶ ECG pulse detection PoC example
  3. 3. 1. Machine Learning – An Introduction 2. Neural Networks 3. Deep Learning Fundamentals 4. Unsupervised Feature Learning 5. Image Recognition 6. Recurrent Neural Networks and Languages Models 7. Deep Learning for Board Games 8. Deep Learning for Computer Games 9. Anomaly Detection 10.Building a Production-ready Intrusion Detection System
  4. 4. Why this use case? ▶ Anomaly detection is crucial to many business applications ▶ Smart feature representation => better anomaly detection ▶ Deep Learning works very well on learning relationships in the underlying raw data (will see how…)
  5. 5. Outlier vs Anomaly “An outlier is a legitimate data point that’s far away from the mean or median in a distribution. It may be unusual, like a 9.6-second 100-meter dash, but still within the realm of reality. An anomaly is an illegitimate data point that’s generated by a different process than whatever generated the rest of the data.” Ravi Parikh http://data.heapanalytics.com/garbage-in-garbage-out-how-anomalies- can-wreck-your-data
  6. 6. Data modeling ▶ Point anomaly (e.g. black sheep) ■ Contextual anomaly (e.g. selling ice- creams in January) ■ Collective anomaly (e.g. sequence of suspected credit card activities)
  7. 7. Detection modeling (and its limitations) ▶ Supervised (classification) ▶ Data skewness, lack of counter examples ▶ Unsupervised (clustering) ▶ Curse of dimensionality ▶ Semi-supervised (novelty detection) ▶ Require a “normal” training dataset
  8. 8. Real world applications ▶ Manufacturing => hardware faults ▶ Law-enforcement => reveal criminal activities ▶ Network system => detect intrusions or anomalous behaviors ▶ Internet Security => malware detection ▶ Financial services => frauds ▶ Marketing / business strategy => spotting profitable customers ▶ Healthcare => Medical diagnosis
  9. 9. What’s the challenge? “Coming up with features is difficult, time- consuming, requires expert knowledge. When working applications of learning, we spend a lot of time tuning features.“ Andrew Ng, Machine Learning and AI via Brain simulations, Stanford University
  10. 10. Hierarchical Feature Learning NVIDIA Deep Learning Course: Class #1 – Introduction to Deep Learning https://www.youtube.com/watch?v=6eBpjEdgSm0
  11. 11. Structural representation Advanced Topics, http://slideplayer.com/slide/3471890/
  12. 12. Signal propagation Schematic diagram of back-propagation neural networks with two hidden layers. Factor selection for delay analysis using Knowledge Discovery in Databases
  13. 13. Auto-encoders • Signal propagation output: approximate an identity function • Error back propagation: Mean Squared Error MSE (*) between the original datum and the reconstructed one (*) in case of numerical data
  14. 14. Novelty detection using auto-encoders 1. Identify a training dataset of what is considered “normal” 2. Learn what “normal” means, aka. learn the structures of normal behavior 3. Try to reconstruct never-seen points re-using the same structure, if the error is high means the point deviates from the normal distribution TRAIN Auto- Encoder RECONSTRUCT Low error RECONSTRUCT High error
  15. 15. Features compression ■ Use just the encoder to compress data into a reduced dimensional space then use traditional unsupervised learning Tom Mitchell’s example of an auto-encoder: You can represent any combination of the 8 binary inputs using only 3 decimal values
  16. 16. PoC examples ▶ ECG Anomaly Pulse Detection ▶ MNIST Anomaly Digit Recognition (Optional) ▶ Jupyter notebooks available on https://github.com/packtmayur/Python- Deep-Learning/tree/master/chapter_9
  17. 17. Summary ▶ We listed a few real-world applications of anomaly detection ▶ We covered some of the most popular techniques in the literature with their limitations ▶ We proposed an overview of how deep neural networks work and why they are great for learning smart feature representations ▶ We proposed 2 semi-supervised approaches using deep auto-encoders: ▶ Novel detection ▶ Feature compression
  18. 18. Going deeper ▶ Advanced modeling: ▶ Denoising auto-encoders ▶ Contractive auto-encoders ▶ Sparse auto-encoders ▶ Variational auto-encoders (for better novelty detection) ▶ Stacked auto-encoders (for better feature compression) ▶ Building a production-ready intrusion detection system: ▶ Validating and testing with labels and in absence of ground truth ▶ Evaluation KPIs for anomaly detection ▶ A/B(C/D) testing
  19. 19. E-book discount ▶ Use the code KVGRSF30 and get 30% discount on e- book ▶ Only valid for 500 uses until 31st October, 2017 ▶ https://www.packtpub.com/b ig-data-and-business- intelligence/python-deep- learning
  20. 20. "Data scientists realize that their best days coincide with discovery of truly odd features in the data." Haystacks and Needles: Anomaly Detection By: Gerhard Pilcher & Kenny Darrell, Data Mining Analyst, Elder Research, Inc.
  21. 21. Deep Neural networks

×