SlideShare ist ein Scribd-Unternehmen logo
1 von 46
Downloaden Sie, um offline zu lesen
Performance Monitoring
for Docker environments
Anomaly detection
Monitoring tools
Live demo
About me
@coscale
www.coscale.com
@spolfliet
stijn.polfliet@coscale.com
Performance Monitoring
for Docker environments
Anomaly detection
Anomaly: definition
Static alerts
TODO : more realistic business examples
!
!
!
?
seasonality
correlations
changing or dynamic
environment
Static alert limitations
Challenges
Machine learning perspective: unsupervised learning problem
? evaluation criteria ?
Challenges
statistical significance relevance
⇏
Simple technique: 3- rule
Holt-Winters
● seasonal exponential smoothing
● works quite well on ‘laboratory
data’
● calculation of prediction intervals
relies on normal distribution after
removal of seasonality
● => on our real world seasonal
data generates too many false
positives
Sliding window approach
model
evaluation
of new
data
Local outlier factor
Existing instance based machine learning technique (lazy,
~kNN)
Based on concept of local density
local outlier factor(A) =
density at point A
average density of kNN of point A
LOF >> 1 ⇒ outlier
en.wikipedia.org/wiki/Local_outlier_factor
Local outlier factor, no strong model assumption
spinning
process
Local outlier factor, no free lunch
Scaling: comparing apples and oranges
scale ⇒ distance ⇒ density ⇒ LOF-score
Autoscaling? (Mahalanobis distance) => enlarges
dimensions with low variance
“Curse of dimensionality”
dimensionality reduction preprocessing (e.g. PCA), but don’t throw
away the anomalies with the bathwater
Choosing cross-sections of data to analyze together, e.g.
different metric on same container
same metric on different containers
Load balance detector
Compare multiple signals (mean + variance) in load-balanced environment
Performance Monitoring
for Docker environments
Monitoring Docker
Orchestrator (services, volumes, replication controllers, …)
Resource usage (cpu, memory, disk, network, ...)
Container internals (application, database, caching, etc.)
Lightweight monitoring for lightweight microservices environment
What to monitor?
Docker Stats API
$ docker stats
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O
1285939c1fd3 0.07% 796 KiB / 64 MiB 1.21% 788 B / 648 B 3.568 MB / 512 KB
9c76f7834ae2 0.07% 2.746 MiB / 64 MiB 4.29% 1.266 KB / 648 B 12.4 MB / 0 B
d1ea048f04e4 0.03% 4.583 MiB / 64 MiB 6.30% 2.854 KB / 648 B 27.7 MB / 0 B
Docker API
docker run 
--volume=/:/rootfs:ro 
--volume=/var/run:/var/run:rw 
--volume=/sys:/sys:ro 
--volume=/var/lib/docker/:/var/lib/docker:ro 
--publish=8080:8080 
--detach=true 
--name=cadvisor 
google/cadvisor:latest
open http://<your-hostname>:8080/
CAdvisor
agent runs in 1 container
container resource usage
no container internals
no application monitoring
15$ / month / server
Scout
scoutapp.com
agent runs in 1 container or on host
container resource usage
basic application monitoring
15$ / month / server
Datadog
datadoghq.com
kernel module captures system calls
container resource usage
basic application monitoring
20$ / month / server
Sysdig
sysdig.com
deep application monitoring
too heavyweight for lightweight container environments
100+$ / month / server
APM vendors
Lightweight agent
• Server metrics from OS
• Container and cluster metrics from Kubernetes and Docker APIs
• Application metrics from log files and management interfaces
• Business & custom metrics from various sources
Contextual events
• Container lifecycle
• Deployments & software releases
• Infrastructure changes
• Custom events
CoScale approach
CoScale architecture
DEMO
Questions?
or contact me at stijn.
polfliet@coscale.com
@spolfliet

Weitere ähnliche Inhalte

Andere mochten auch

Netflix0SS Services on Docker
Netflix0SS Services on DockerNetflix0SS Services on Docker
Netflix0SS Services on Docker
Docker, Inc.
 
Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart
Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart
Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart
Docker, Inc.
 

Andere mochten auch (20)

cert
certcert
cert
 
Writing Docker monitoring agent with Go
Writing Docker monitoring agent with GoWriting Docker monitoring agent with Go
Writing Docker monitoring agent with Go
 
Discussing the difference between docker dontainers and virtual machines
Discussing the difference between docker dontainers and virtual machinesDiscussing the difference between docker dontainers and virtual machines
Discussing the difference between docker dontainers and virtual machines
 
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
Nagios Conference 2014 - Spenser Reinhardt - Detecting Security Breaches With...
 
Running Netflix OSS on Docker with Nirmata
Running Netflix OSS on Docker with NirmataRunning Netflix OSS on Docker with Nirmata
Running Netflix OSS on Docker with Nirmata
 
Docker Indy Meetup Monitoring 30-Aug-2016
Docker Indy Meetup Monitoring 30-Aug-2016Docker Indy Meetup Monitoring 30-Aug-2016
Docker Indy Meetup Monitoring 30-Aug-2016
 
Netflix and Containers: Not A Stranger Thing
Netflix and Containers:  Not A Stranger ThingNetflix and Containers:  Not A Stranger Thing
Netflix and Containers: Not A Stranger Thing
 
Microservices with Netflix OSS and Spring Cloud - Dev Day Orange
Microservices with Netflix OSS and Spring Cloud -  Dev Day OrangeMicroservices with Netflix OSS and Spring Cloud -  Dev Day Orange
Microservices with Netflix OSS and Spring Cloud - Dev Day Orange
 
NetflixOSS and ZeroToDocker Talk
NetflixOSS and ZeroToDocker TalkNetflixOSS and ZeroToDocker Talk
NetflixOSS and ZeroToDocker Talk
 
Microservices with Spring Cloud, Netflix OSS and Kubernetes
Microservices with Spring Cloud, Netflix OSS and Kubernetes Microservices with Spring Cloud, Netflix OSS and Kubernetes
Microservices with Spring Cloud, Netflix OSS and Kubernetes
 
Microservices and APIs
Microservices and APIsMicroservices and APIs
Microservices and APIs
 
Real world #microservices with Apache Camel, Fabric8, and OpenShift
Real world #microservices with Apache Camel, Fabric8, and OpenShiftReal world #microservices with Apache Camel, Fabric8, and OpenShift
Real world #microservices with Apache Camel, Fabric8, and OpenShift
 
Overseeing Ship's Surveys and Surveyors Globally Using IoT and Docker by Jay ...
Overseeing Ship's Surveys and Surveyors Globally Using IoT and Docker by Jay ...Overseeing Ship's Surveys and Surveyors Globally Using IoT and Docker by Jay ...
Overseeing Ship's Surveys and Surveyors Globally Using IoT and Docker by Jay ...
 
Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...
Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...
Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...
 
Securing the Container Pipeline at Salesforce by Cem Gurkok
Securing the Container Pipeline at Salesforce by Cem Gurkok   Securing the Container Pipeline at Salesforce by Cem Gurkok
Securing the Container Pipeline at Salesforce by Cem Gurkok
 
Docker in Production, Look No Hands! by Scott Coulton
Docker in Production, Look No Hands! by Scott CoultonDocker in Production, Look No Hands! by Scott Coulton
Docker in Production, Look No Hands! by Scott Coulton
 
Build Fast, Deploy Fast: Innovating in the Enterprise by Imran Raja and Andy Lim
Build Fast, Deploy Fast: Innovating in the Enterprise by Imran Raja and Andy LimBuild Fast, Deploy Fast: Innovating in the Enterprise by Imran Raja and Andy Lim
Build Fast, Deploy Fast: Innovating in the Enterprise by Imran Raja and Andy Lim
 
Netflix0SS Services on Docker
Netflix0SS Services on DockerNetflix0SS Services on Docker
Netflix0SS Services on Docker
 
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
Using the SDACK Architecture on Security Event Inspection by Yu-Lun Chen and ...
 
Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart
Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart
Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart
 

Ähnlich wie Performance Monitoring for Docker Environments - Docker Amsterdam June Meetup

How to accelerate docker adoption with a simple and powerful user experience
How to accelerate docker adoption with a simple and powerful user experienceHow to accelerate docker adoption with a simple and powerful user experience
How to accelerate docker adoption with a simple and powerful user experience
Docker, Inc.
 

Ähnlich wie Performance Monitoring for Docker Environments - Docker Amsterdam June Meetup (20)

Performance monitoring for Docker - Lucerne meetup
Performance monitoring for Docker - Lucerne meetupPerformance monitoring for Docker - Lucerne meetup
Performance monitoring for Docker - Lucerne meetup
 
Cluster management with Kubernetes
Cluster management with KubernetesCluster management with Kubernetes
Cluster management with Kubernetes
 
Be a better developer with Docker (revision 3)
Be a better developer with Docker (revision 3)Be a better developer with Docker (revision 3)
Be a better developer with Docker (revision 3)
 
Check the version with fixes. Link in description
Check the version with fixes. Link in descriptionCheck the version with fixes. Link in description
Check the version with fixes. Link in description
 
Docker as an every day work tool
Docker as an every day work toolDocker as an every day work tool
Docker as an every day work tool
 
Monitoring in Motion: Monitoring Containers and Amazon ECS
Monitoring in Motion: Monitoring Containers and Amazon ECSMonitoring in Motion: Monitoring Containers and Amazon ECS
Monitoring in Motion: Monitoring Containers and Amazon ECS
 
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
OpenNebulaConf 2016 - Measuring and tuning VM performance by Boyan Krosnov, S...
 
Handson Oracle Management Cloud with Application Performance Monitoring and L...
Handson Oracle Management Cloud with Application Performance Monitoring and L...Handson Oracle Management Cloud with Application Performance Monitoring and L...
Handson Oracle Management Cloud with Application Performance Monitoring and L...
 
H2O World - Building a Smarter Application - Tom Kraljevic
H2O World - Building a Smarter Application - Tom KraljevicH2O World - Building a Smarter Application - Tom Kraljevic
H2O World - Building a Smarter Application - Tom Kraljevic
 
Stephan Ewen - Experiences running Flink at Very Large Scale
Stephan Ewen -  Experiences running Flink at Very Large ScaleStephan Ewen -  Experiences running Flink at Very Large Scale
Stephan Ewen - Experiences running Flink at Very Large Scale
 
Cloudy Open Source and DevOps
Cloudy Open Source and DevOpsCloudy Open Source and DevOps
Cloudy Open Source and DevOps
 
this-is-garbage-talk-2022.pptx
this-is-garbage-talk-2022.pptxthis-is-garbage-talk-2022.pptx
this-is-garbage-talk-2022.pptx
 
Scale Container Operations with AIOps
Scale Container Operations with AIOpsScale Container Operations with AIOps
Scale Container Operations with AIOps
 
"Offline mode for a mobile application, redux on server and a little bit abou...
"Offline mode for a mobile application, redux on server and a little bit abou..."Offline mode for a mobile application, redux on server and a little bit abou...
"Offline mode for a mobile application, redux on server and a little bit abou...
 
Rx Swift
Rx SwiftRx Swift
Rx Swift
 
(WEB301) Operational Web Log Analysis | AWS re:Invent 2014
(WEB301) Operational Web Log Analysis | AWS re:Invent 2014(WEB301) Operational Web Log Analysis | AWS re:Invent 2014
(WEB301) Operational Web Log Analysis | AWS re:Invent 2014
 
Docker and Cloud - Enables for DevOps - by ACA-IT
Docker and Cloud - Enables for DevOps - by ACA-ITDocker and Cloud - Enables for DevOps - by ACA-IT
Docker and Cloud - Enables for DevOps - by ACA-IT
 
'DOCKER' & CLOUD: ENABLERS For DEVOPS
'DOCKER' & CLOUD:  ENABLERS For DEVOPS'DOCKER' & CLOUD:  ENABLERS For DEVOPS
'DOCKER' & CLOUD: ENABLERS For DEVOPS
 
How to accelerate docker adoption with a simple and powerful user experience
How to accelerate docker adoption with a simple and powerful user experienceHow to accelerate docker adoption with a simple and powerful user experience
How to accelerate docker adoption with a simple and powerful user experience
 
Containers explained as for cook and a mecanics
 Containers explained as for cook and a mecanics  Containers explained as for cook and a mecanics
Containers explained as for cook and a mecanics
 

Kürzlich hochgeladen

%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
VictorSzoltysek
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
masabamasaba
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
masabamasaba
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
masabamasaba
 

Kürzlich hochgeladen (20)

Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastArchitecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?
 
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
 
Introducing Microsoft’s new Enterprise Work Management (EWM) Solution
Introducing Microsoft’s new Enterprise Work Management (EWM) SolutionIntroducing Microsoft’s new Enterprise Work Management (EWM) Solution
Introducing Microsoft’s new Enterprise Work Management (EWM) Solution
 
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
 
tonesoftg
tonesoftgtonesoftg
tonesoftg
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park %in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
 
Harnessing ChatGPT - Elevating Productivity in Today's Agile Environment
Harnessing ChatGPT  - Elevating Productivity in Today's Agile EnvironmentHarnessing ChatGPT  - Elevating Productivity in Today's Agile Environment
Harnessing ChatGPT - Elevating Productivity in Today's Agile Environment
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
 

Performance Monitoring for Docker Environments - Docker Amsterdam June Meetup