Short overview of selected talks from Berlin Buzzwords 2014

1. BERLIN BUZZWORDS 2014
SELECTEDTALKS OVERVIEW
tech talk @ ferret
Andrii Gakhov
19/06/2014

2. DEEP LEARNING
FOR HIGH PERFORMANCETIME-SERIES DATABASES
byTed Dunning

3. ABOUT OF THE AUTHOR
• Chief Application Architect at MapR
Technologies
• Ph.D. in computing science from the University
of Sheffield
• Committer on Mahout, Drill, Zookeeper …
• http://tdunning.blogspot.de/
• @ted_dunning

4. ANOMALY DETECTION

5. ANOMALY DETECTION
99.9%-ile
Online summarizer
(t-digest)
99.9%-ile
t
x > t?
x
!
• The t-digest algorithm was developed byTed Dunning and available in Apache Machout
• With t-digest algorithm one can accurately estimate quantiles for very large data sets
with limited memory use
input signal

6. ISSUES WITH SIMPLETHRESHOLDS

7. LOOKS LIKE ANOMALY?

8. NOT SURE

9. WHAT IS NORMAL?
• We need to have a model of what is normal
• Everything that doesn’t fit model is the anomaly
• For simple signals we can assume just normal distribution

10. WINDOWS

11. WINDOWS

12. WINDOWS
• Set of windowed signals - model of the original signal
• Clustering can find the prototypes
• The result is a dictionary of shapes
• New signals can be encoded by shifting, scaling and adding
shapes from the dictionary

13. COMMON SHAPES (EKG)

14. RECONSTRUCTION

15. ANOMALY

16. ANOMALY

17. MODEL ANOMALY DETECTION
Online summarizer
99.9%-ile
t
∆ > t?
x
!
t
∆ reconstruction error
Model
x’
x-x’
input signal

18. COMPRESSION
Minimal Error
Maximum likelihood
Maximum compression
• Good anomaly detectors give good compression
• So, we are constructing an auto-encoder!

19. MODEL ANOMALY DETECTION
Online summarizer
99.9%-ile
t
∆ > t?
x
!
t
∆ reconstruction error
Encoder
x’
x-x’
input signal
shape
dict

20. CLUSTERING
• Use windowing
• Find nearest cluster for each window
• Scale cluster to the right size
• Subtract from the original signal

21. CLUSTERING AS NEURAL NETWORK

22. OVERLAPPING NETWORKS
Time series input
Reconstructed time series

23. READ MORE?
• A New Look At Anomaly
Detection
• This is the second book in the
series Practical Machine Learning by
Ted Dunning & Ellen Friedman
• FREE download from
www.mapr.com