SlideShare ist ein Scribd-Unternehmen logo
1 von 27
© 2019 SPLUNK INC.© 2019 SPLUNK INC.
Machine Learning in Action
Anomaly Detection Methods and
Applications
© 2019 SPLUNK INC.
During the course of this presentation, we may make forward-looking statements regarding future events or
the expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results could
differ materially. For important factors that may cause actual results to differ from those contained in our
forward-looking statements, please review our filings with the SEC.
The forward-looking statements made in this presentation are being made as of the time and date of its live
presentation. If reviewed after its live presentation, this presentation may not contain current or accurate
information. We do not assume any obligation to update any forward-looking statements we may make. In
addition, any information about our roadmap outlines our general product direction and is subject to change
at any time without notice. It is for informational purposes only and shall not be incorporated into any contract
or other commitment. Splunk undertakes no obligation either to develop the features or functionality
described or to include any such feature or functionality in a future release.
Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in
the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2019 Splunk Inc. All rights reserved.
Forward-Looking Statements
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
© 2019 SPLUNK INC.
Agenda
© 2019 SPLUNK INC.
1. A bit theory first: perspectives on Anomalies
2. Why Anomalies Matter for Business
3. How to spot anomalies?
4. Demotime!
5. I want to learn more
6. Q&A
Agenda
© 2019 SPLUNK INC.
A Bit Theory First
© 2019 SPLUNK INC.
Splunk Customers Want Answers from their Data
► Deviation from past behavior
► Deviation from peers
► (aka Multivariate AD or Cohesive AD)
► Unusual change in features
► ITSI MAD Anomaly Detection
► Predict Service Health Score
Predicting Churn
► Predicting Events
► Trend Forecasting
► Detecting influencing entities
► Early warning of failure –
predictive maintenance
► Identify peer groups
► Event Correlation
► Reduce alert noise
► Behavioral Analytics
► ITSI Event Analytics
Anomaly detection Predictive Analytics Clustering
© 2019 SPLUNK INC.
▶ From Latin anomalia, from Ancient Greek ἀνωμαλία (anōmalía, “irregularity,
anomaly”), from ἀνώμαλος (anṓmalos, “irregular, uneven”), negating the
meaning of ὁμαλός (homalós, “even”), from ὁμός (homós, “same”).
▶ A deviation from a rule or from what is regarded as normal; an outlier.
Synonyms: abnormality, deviance, deviation, exception, inconsistency,
irregularity, phenomenon
▶ In the natural sciences, especially in atmospheric and Earth sciences involving
applied statistics, an anomaly is the deviation in a quantity from its expected
value, e.g., the difference between a measurement and a mean or a model
prediction. […]
Perspectives on Anomalies
https://en.wiktionary.org/wiki/anomaly and https://en.wikipedia.org/wiki/Anomaly and https://en.wikipedia.org/wiki/Anomaly_(natural_sciences)
© 2019 SPLUNK INC.
▶ Only 72 pages
▶ A comprehensive
report of most
common classic
methodologies
and algorithmic
approaches
http://cucis.ece.northwestern.edu/projects/DMS/publications/AnomalyDetection.pdf
© 2019 SPLUNK INC.
Why Anomalies
Matter
© 2019 SPLUNK INC.
• Network traffic
• Access pattern
• …
• Service outages
• Infrastructure
problems
• …
• Equipment
degradation
• Preventative
Maintenance
• …
• Fraud Detection
• Insider Threats
• …
Interesting Anomalies Across Your Business
Security – IT Operations – IoT/OT – Business Analytics
© 2019 SPLUNK INC.
How to Spot
Anomalies
© 2019 SPLUNK INC.
▶ “Can Splunk detect anomalies in
my data?”
▶ “Can Splunk help me identify
unknown things?”
▶ “Can Splunk find answers for
questions that I don’t know?”
▶ Ask yourself what questions you
are asking!
Questions… there are
so many questions…
© 2019 SPLUNK INC.
Search Processing Language (SPL) Machine Learning Toolkit (MLTK)
Cheat Sheet for Anomaly Detection in Splunk
Command Description
analyzefields, af Analyze numerical fields for their ability to
predict another discrete field.
anomalies Computes an "unexpectedness" score for an
event.
anomalousvalue Finds and summarizes irregular, or
uncommon, search results.
anomalydetection Identifies anomalous events by computing a
probability for each event and then detecting
unusually small probabilities.
cluster Clusters similar events together.
kmeans Performs k-means clustering on selected
fields.
outlier Removes outlying numerical values.
rare Displays the least common values of a field.
Method / Algorithm Description
DensityFunction The DensityFunction algorithm provides a
consistent and streamlined workflow to create
and store density functions and utilize them for
anomaly detection…
LocalOutlierFactor The LocalOutlierFactor algorithm measures
the local deviation of density of a given
sample with respect to its neighbors…
OneClassSVM The OneClassSVM algorithm fits a model from
a set of features or fields for detecting
anomalies and outliers…
Clustering
Algorithms
Spot point anomalies or anomaleous clusters.
Inspect e.g. cluster_distance with KMeans,
cluster=-1 with DBSCAN…
Classifiers and
Regressors
Inspect strong residuals when applying your
well fitted model to new incoming data points.
ML SPL API Wrap your own algorithms of choice
https://docs.splunk.com/Documentation/Splunk/7.2.5/SearchReference/Commandsbycategory#Find_anomalies https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms
© 2019 SPLUNK INC.
Demo time!
© 2019 SPLUNK INC.
consider your ML dataset’s dimensional and computational complexity
computational complexity
dimensional complexity
Machine Learning Toolkit
In general: for most common ML tasks: use MLTK + MLSPL API
extensibility
Case #1: need for specific
algo / framework
Case #2: need for
distributed / gpu compute
extensibility
Recommendation Matrix
© 2019 SPLUNK INC.
I want to learn more!
© 2019 SPLUNK INC.
Where Can I Learn More About Anomaly Detection?
4 must read blog posts – don’t miss them!
© 2019 SPLUNK INC.
• DGA App for Splunk
• Sec. Essentials
• UBA
• MLTK
• ITSI
• Splunk Essentials for
Predictive Maintenance
• Splunk Security
Essentials for Fraud
Detection
Where to Find Ready Made Apps…
… for my business area of interest?
https://splunkbase.splunk.com
© 2019 SPLUNK INC.
4 Days of Innovation 350 Education Sessions 20 Hours of Networking
“Hands down the most beneficial and attendee focused conference
I have attended!”
– Michael Mills, Senior Consultant, Booz Allen Hamilton
sign up for notifications @ conf.splunk.com
.conf19
October 21-24, 2019
Splunk University
October 19-21, 2019
Las Vegas, NV
The Venetian Sands
Expo
© 2019 SPLUNK INC.
Splunk Machine
Learning Advisory
Program
© 2019 SPLUNK INC.
Your
Logo
Here?
Get started on your specific use case with the guidance of Splunk Data Scientists
Consider the ML Advisory Program
© 2018 SPLUNK INC.
▶ Early access to new
and enhanced Machine
Learning features
▶ Opportunity to shape
the development of the
product
▶ Complimentary
assistance in
operationalizing a
production quality ML
model
What is the ML Advisory Program?
Complimentary support of Splunk data science resources to help build a ML use
case resulting in a public reference
© 2019 SPLUNK INC.© 2019 SPLUNK INC.
Thank You.

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (14)

Splunk und Multi-Cloud
Splunk und Multi-CloudSplunk und Multi-Cloud
Splunk und Multi-Cloud
 
Alle Neuigkeiten im letzten Plattform Release
Alle Neuigkeiten im letzten Plattform ReleaseAlle Neuigkeiten im letzten Plattform Release
Alle Neuigkeiten im letzten Plattform Release
 
Machine Learning in Action
Machine Learning in ActionMachine Learning in Action
Machine Learning in Action
 
Get more from your Machine Data with Splunk AI and ML
Get more from your Machine Data with Splunk AI and ML Get more from your Machine Data with Splunk AI and ML
Get more from your Machine Data with Splunk AI and ML
 
Make Your SOC Work Smarter, Not Harder
Make Your SOC Work Smarter, Not HarderMake Your SOC Work Smarter, Not Harder
Make Your SOC Work Smarter, Not Harder
 
Introduction into Security Analytics Methods
Introduction into Security Analytics Methods Introduction into Security Analytics Methods
Introduction into Security Analytics Methods
 
Turning Data into Business outcomes
Turning Data into Business outcomes Turning Data into Business outcomes
Turning Data into Business outcomes
 
Accelerate incident Response Using Orchestration and Automation
Accelerate incident Response Using Orchestration and Automation Accelerate incident Response Using Orchestration and Automation
Accelerate incident Response Using Orchestration and Automation
 
Vorausschauendes, proaktives und collaboratives Machine Learning mit Splunk ITSI
Vorausschauendes, proaktives und collaboratives Machine Learning mit Splunk ITSIVorausschauendes, proaktives und collaboratives Machine Learning mit Splunk ITSI
Vorausschauendes, proaktives und collaboratives Machine Learning mit Splunk ITSI
 
Adventures in Monitoring and Troubleshooting
Adventures in Monitoring and Troubleshooting Adventures in Monitoring and Troubleshooting
Adventures in Monitoring and Troubleshooting
 
Splunk Cloud and Splunk Enterprise 7.2
Splunk Cloud and Splunk Enterprise 7.2 Splunk Cloud and Splunk Enterprise 7.2
Splunk Cloud and Splunk Enterprise 7.2
 
Worst Splunk practices...and how to fix them
Worst Splunk practices...and how to fix them Worst Splunk practices...and how to fix them
Worst Splunk practices...and how to fix them
 
Exploring Frameworks of Splunk Enterprise Security
Exploring Frameworks of Splunk Enterprise Security Exploring Frameworks of Splunk Enterprise Security
Exploring Frameworks of Splunk Enterprise Security
 
How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...
How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...
How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...
 

Ähnlich wie Machine Learning in Action

Ähnlich wie Machine Learning in Action (20)

Machine Learning in Action
Machine Learning in ActionMachine Learning in Action
Machine Learning in Action
 
Predictive, Proactive, and Collaborative ML with iT Service Intelligence
Predictive, Proactive, and Collaborative ML with iT Service Intelligence Predictive, Proactive, and Collaborative ML with iT Service Intelligence
Predictive, Proactive, and Collaborative ML with iT Service Intelligence
 
Get more from your Machine Date with Splunk AI and ML
Get more from your Machine Date with Splunk AI and ML Get more from your Machine Date with Splunk AI and ML
Get more from your Machine Date with Splunk AI and ML
 
Mit Splunk Artificial Intelligence und Machine Learning mehr aus Ihren Daten ...
Mit Splunk Artificial Intelligence und Machine Learning mehr aus Ihren Daten ...Mit Splunk Artificial Intelligence und Machine Learning mehr aus Ihren Daten ...
Mit Splunk Artificial Intelligence und Machine Learning mehr aus Ihren Daten ...
 
Spliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics Methods
Spliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics MethodsSpliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics Methods
Spliunk Discovery Köln - 17-01-2020 - Intro to Security Analytics Methods
 
Get More From Your Data with Splunk AI + ML
Get More From Your Data with Splunk AI + MLGet More From Your Data with Splunk AI + ML
Get More From Your Data with Splunk AI + ML
 
SplunkLive! Paris 2018: Delivering New Visibility And Analytics For IT Operat...
SplunkLive! Paris 2018: Delivering New Visibility And Analytics For IT Operat...SplunkLive! Paris 2018: Delivering New Visibility And Analytics For IT Operat...
SplunkLive! Paris 2018: Delivering New Visibility And Analytics For IT Operat...
 
Accelerate incident Response Using Orchestration and Automation
Accelerate incident Response Using Orchestration and Automation Accelerate incident Response Using Orchestration and Automation
Accelerate incident Response Using Orchestration and Automation
 
SEC1671/ Attack range/Splunk SIEMulator splunkconf2019
SEC1671/ Attack range/Splunk SIEMulator splunkconf2019SEC1671/ Attack range/Splunk SIEMulator splunkconf2019
SEC1671/ Attack range/Splunk SIEMulator splunkconf2019
 
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident ResponseSplunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
Splunk Discovery Köln - 17-01-2020 - Accelerate Incident Response
 
Introduction into Security Analytics Methods
Introduction into Security Analytics Methods Introduction into Security Analytics Methods
Introduction into Security Analytics Methods
 
Sec1391
Sec1391Sec1391
Sec1391
 
Splunk Artificial Intelligence & Machine Learning Webinar
Splunk Artificial Intelligence & Machine Learning WebinarSplunk Artificial Intelligence & Machine Learning Webinar
Splunk Artificial Intelligence & Machine Learning Webinar
 
Common Machine Learning Solutions Everyone Needs to Know
Common Machine Learning Solutions Everyone Needs to KnowCommon Machine Learning Solutions Everyone Needs to Know
Common Machine Learning Solutions Everyone Needs to Know
 
SplunkLive! Stockholm 2019 - Customer presentation: Norlys
SplunkLive! Stockholm 2019 - Customer presentation: Norlys SplunkLive! Stockholm 2019 - Customer presentation: Norlys
SplunkLive! Stockholm 2019 - Customer presentation: Norlys
 
Adventures in Monitoring and Troubleshooting
Adventures in Monitoring and Troubleshooting Adventures in Monitoring and Troubleshooting
Adventures in Monitoring and Troubleshooting
 
SplunkLive! Frankfurt 2018 - Predictive, Proactive, and Collaborative ML with...
SplunkLive! Frankfurt 2018 - Predictive, Proactive, and Collaborative ML with...SplunkLive! Frankfurt 2018 - Predictive, Proactive, and Collaborative ML with...
SplunkLive! Frankfurt 2018 - Predictive, Proactive, and Collaborative ML with...
 
Exploring Frameworks of Splunk Enterprise Security
Exploring Frameworks of Splunk Enterprise SecurityExploring Frameworks of Splunk Enterprise Security
Exploring Frameworks of Splunk Enterprise Security
 
Splunk Enterprise Security
Splunk Enterprise SecuritySplunk Enterprise Security
Splunk Enterprise Security
 
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business OutcomesSplunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
Splunk Discovery Köln - 17-01-2020 - Turning Data Into Business Outcomes
 

Mehr von Splunk

Mehr von Splunk (20)

.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine
 
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
 
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica).conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
 
.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International
 
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett .conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
 
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär).conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
 
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu....conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
 
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever....conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
 
.conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex).conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex)
 
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
 
Splunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11y
 
Splunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go Köln
 
Splunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go Köln
 
Data foundations building success, at city scale – Imperial College London
 Data foundations building success, at city scale – Imperial College London Data foundations building success, at city scale – Imperial College London
Data foundations building success, at city scale – Imperial College London
 
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
 
SOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security Webinar
 
.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session
 
.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote
 
.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session
 
.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session
 

Kürzlich hochgeladen

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 

Machine Learning in Action

  • 1. © 2019 SPLUNK INC.© 2019 SPLUNK INC. Machine Learning in Action Anomaly Detection Methods and Applications
  • 2. © 2019 SPLUNK INC. During the course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2019 Splunk Inc. All rights reserved. Forward-Looking Statements
  • 7. © 2019 SPLUNK INC. Agenda
  • 8. © 2019 SPLUNK INC. 1. A bit theory first: perspectives on Anomalies 2. Why Anomalies Matter for Business 3. How to spot anomalies? 4. Demotime! 5. I want to learn more 6. Q&A Agenda
  • 9. © 2019 SPLUNK INC. A Bit Theory First
  • 10. © 2019 SPLUNK INC. Splunk Customers Want Answers from their Data ► Deviation from past behavior ► Deviation from peers ► (aka Multivariate AD or Cohesive AD) ► Unusual change in features ► ITSI MAD Anomaly Detection ► Predict Service Health Score Predicting Churn ► Predicting Events ► Trend Forecasting ► Detecting influencing entities ► Early warning of failure – predictive maintenance ► Identify peer groups ► Event Correlation ► Reduce alert noise ► Behavioral Analytics ► ITSI Event Analytics Anomaly detection Predictive Analytics Clustering
  • 11. © 2019 SPLUNK INC. ▶ From Latin anomalia, from Ancient Greek ἀνωμαλία (anōmalía, “irregularity, anomaly”), from ἀνώμαλος (anṓmalos, “irregular, uneven”), negating the meaning of ὁμαλός (homalós, “even”), from ὁμός (homós, “same”). ▶ A deviation from a rule or from what is regarded as normal; an outlier. Synonyms: abnormality, deviance, deviation, exception, inconsistency, irregularity, phenomenon ▶ In the natural sciences, especially in atmospheric and Earth sciences involving applied statistics, an anomaly is the deviation in a quantity from its expected value, e.g., the difference between a measurement and a mean or a model prediction. […] Perspectives on Anomalies https://en.wiktionary.org/wiki/anomaly and https://en.wikipedia.org/wiki/Anomaly and https://en.wikipedia.org/wiki/Anomaly_(natural_sciences)
  • 12. © 2019 SPLUNK INC. ▶ Only 72 pages ▶ A comprehensive report of most common classic methodologies and algorithmic approaches http://cucis.ece.northwestern.edu/projects/DMS/publications/AnomalyDetection.pdf
  • 13. © 2019 SPLUNK INC. Why Anomalies Matter
  • 14. © 2019 SPLUNK INC. • Network traffic • Access pattern • … • Service outages • Infrastructure problems • … • Equipment degradation • Preventative Maintenance • … • Fraud Detection • Insider Threats • … Interesting Anomalies Across Your Business Security – IT Operations – IoT/OT – Business Analytics
  • 15. © 2019 SPLUNK INC. How to Spot Anomalies
  • 16. © 2019 SPLUNK INC. ▶ “Can Splunk detect anomalies in my data?” ▶ “Can Splunk help me identify unknown things?” ▶ “Can Splunk find answers for questions that I don’t know?” ▶ Ask yourself what questions you are asking! Questions… there are so many questions…
  • 17. © 2019 SPLUNK INC. Search Processing Language (SPL) Machine Learning Toolkit (MLTK) Cheat Sheet for Anomaly Detection in Splunk Command Description analyzefields, af Analyze numerical fields for their ability to predict another discrete field. anomalies Computes an "unexpectedness" score for an event. anomalousvalue Finds and summarizes irregular, or uncommon, search results. anomalydetection Identifies anomalous events by computing a probability for each event and then detecting unusually small probabilities. cluster Clusters similar events together. kmeans Performs k-means clustering on selected fields. outlier Removes outlying numerical values. rare Displays the least common values of a field. Method / Algorithm Description DensityFunction The DensityFunction algorithm provides a consistent and streamlined workflow to create and store density functions and utilize them for anomaly detection… LocalOutlierFactor The LocalOutlierFactor algorithm measures the local deviation of density of a given sample with respect to its neighbors… OneClassSVM The OneClassSVM algorithm fits a model from a set of features or fields for detecting anomalies and outliers… Clustering Algorithms Spot point anomalies or anomaleous clusters. Inspect e.g. cluster_distance with KMeans, cluster=-1 with DBSCAN… Classifiers and Regressors Inspect strong residuals when applying your well fitted model to new incoming data points. ML SPL API Wrap your own algorithms of choice https://docs.splunk.com/Documentation/Splunk/7.2.5/SearchReference/Commandsbycategory#Find_anomalies https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms
  • 18. © 2019 SPLUNK INC. Demo time!
  • 19. © 2019 SPLUNK INC. consider your ML dataset’s dimensional and computational complexity computational complexity dimensional complexity Machine Learning Toolkit In general: for most common ML tasks: use MLTK + MLSPL API extensibility Case #1: need for specific algo / framework Case #2: need for distributed / gpu compute extensibility Recommendation Matrix
  • 20. © 2019 SPLUNK INC. I want to learn more!
  • 21. © 2019 SPLUNK INC. Where Can I Learn More About Anomaly Detection? 4 must read blog posts – don’t miss them!
  • 22. © 2019 SPLUNK INC. • DGA App for Splunk • Sec. Essentials • UBA • MLTK • ITSI • Splunk Essentials for Predictive Maintenance • Splunk Security Essentials for Fraud Detection Where to Find Ready Made Apps… … for my business area of interest? https://splunkbase.splunk.com
  • 23. © 2019 SPLUNK INC. 4 Days of Innovation 350 Education Sessions 20 Hours of Networking “Hands down the most beneficial and attendee focused conference I have attended!” – Michael Mills, Senior Consultant, Booz Allen Hamilton sign up for notifications @ conf.splunk.com .conf19 October 21-24, 2019 Splunk University October 19-21, 2019 Las Vegas, NV The Venetian Sands Expo
  • 24. © 2019 SPLUNK INC. Splunk Machine Learning Advisory Program
  • 25. © 2019 SPLUNK INC. Your Logo Here? Get started on your specific use case with the guidance of Splunk Data Scientists Consider the ML Advisory Program
  • 26. © 2018 SPLUNK INC. ▶ Early access to new and enhanced Machine Learning features ▶ Opportunity to shape the development of the product ▶ Complimentary assistance in operationalizing a production quality ML model What is the ML Advisory Program? Complimentary support of Splunk data science resources to help build a ML use case resulting in a public reference
  • 27. © 2019 SPLUNK INC.© 2019 SPLUNK INC. Thank You.