SlideShare ist ein Scribd-Unternehmen logo
1 von 21
Downloaden Sie, um offline zu lesen
github.com/gianarb/orbiter
Gianluca Arbezzano
@gianarb
Building clouds at InfluxDB (SRE)
Why?
● To create a use case around metrics, InfluxDB and Docker.
● Docker Swarm at the moment doesn’t provider autoscale capabilities.
● To design an application easy to manage and well integrated in Docker Swarm.
Orbiter is a general-purpose multi vendor autoscaler
Orbiter is a general-purpose multi vendor autoscaler
Orbiter is a Docker Swarm autoscaler
swarm cluster
manager
get metrics:
● cadvidor
● telegraf
● ….
alert:
● custom
● kapacitor
● alert
manager
Why Docker Swarm doesn’t have autoscaling?
● I don’t know.
● As you can see the architecture required is not small.
● Probably it will have autoscaling at some point.
Orbiter is designed to be Only-Swarm
● To make it easy to start
● To remove abstractions
● Because less is more! cit unknown but for sure a lazy person
What does it?
● It lists all your services and it gets only who has label orbiter=true
● Expose and HTTP API to trigger up and down scale event
○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/up
○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/down
● It uses Docker Swarm Api to keep in sync the managed services
● You can configure a service via label orbiter.up=3, orbiter.donw=2, orbiter.cooldown = 2
Requirement:
● Deploy it in a manager.
Deploy it as service in
Docker Swarm
version: '3'
services:
orbiter:
image: gianarb/orbiter
command: daemon
ports:
- 8081:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints:
- node.role == manager
mode: replicated
replicas: 1
How Orbiter communicate with Swarm
func NewSwarmProvider(c map[string]string) (autoscaler.Provider, error) {
var p autoscaler.Provider
client, err := docker.NewEnvClient()
if err != nil {
logrus.WithField("error", err).Warn("problem to communicate with docker")
return p, err
} else {
logrus.Info("Successfully connected to a Docker daemon")
}
p = SwarmProvider{
dockerClient: client,
}
return p, nil
}
Scale a service - 1.Find the service
service, _, err := p.dockerClient.ServiceInspectWithRaw(ctx, serviceId, types.ServiceInspectOptions{
InsertDefaults: true,
})
if err != nil {
logrus.WithFields(logrus.Fields{
"error": err.Error(),
"provider": "swarm",
}).Debugf("Service %s didn't scale. We didn't get it from docker.", serviceId)
return err
}
filters := filters.NewArgs()
filters.Add("service", serviceId)
tasks, err := p.dockerClient.TaskList(ctx, types.TaskListOptions{Filters: filters})
if err != nil { … }
err = p.isAcceptable(tasks, target, direction)
Scale a service - 2.Update the Spec
spec := service.Spec
var ptrFromSystem uint64
base := p.calculateActiveTasks(tasks)
if direction == true {
ptrFromSystem = uint64(base + target)
} else {
ptrFromSystem = uint64(base - target)
}
spec.Mode.Replicated.Replicas = &ptrFromSystem
_, err = p.dockerClient.ServiceUpdate(ctx, serviceId, service.Version, spec, types.ServiceUpdateOptions{})
if err != nil {...}
Background job to sync services with Docker
services, err := dockerClient.ServiceList(ctx, types.ServiceListOptions{})
if err != nil {
logrus.WithField("error", err).Debug("Bad communication with Docker.")
return
}
prov, _ := provider.NewSwarmProvider(map[string]string{})
for _, service := range services {
s, err := getAutoscalerByService(prov, service.Spec.Annotations)
if err != nil {
continue
}
c.Autoscalers[fmt.Sprintf("autoswarm/%s", service.Spec.Annotations.Name)] = s
}
Is orbiter just a wrapper on Docker Swarm API?
Let’s put that in a cooler way.
Orbiter provides easy to use rest API to
scale your services up and down.
T-shirt time
Demo
Thanks a lot
https://twitter.com/gianarb
https://github.com/gianarb
https://gianarb.it

Weitere ähnliche Inhalte

Was ist angesagt?

A dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenarioA dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenarioGioia Ballin
 
CLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.jsCLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.jsForrest Norvell
 
Callbacks and control flow in Node js
Callbacks and control flow in Node jsCallbacks and control flow in Node js
Callbacks and control flow in Node jsThomas Roch
 
Asynchronous handlers in asp.net
Asynchronous handlers in asp.netAsynchronous handlers in asp.net
Asynchronous handlers in asp.netAbhishek Sur
 
Using akka streams to access s3 objects
Using akka streams to access s3 objectsUsing akka streams to access s3 objects
Using akka streams to access s3 objectsMikhail Girkin
 
Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...Love Nyberg
 
Asynchronní programování
Asynchronní programováníAsynchronní programování
Asynchronní programováníPeckaDesign.cz
 
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2MongoDB
 
Javascript internals
Javascript internalsJavascript internals
Javascript internalsAyush Sharma
 
Reactive mistakes - ScalaDays Chicago 2017
Reactive mistakes -  ScalaDays Chicago 2017Reactive mistakes -  ScalaDays Chicago 2017
Reactive mistakes - ScalaDays Chicago 2017Petr Zapletal
 
Unity 3D Runtime Animation Generation
Unity 3D Runtime Animation GenerationUnity 3D Runtime Animation Generation
Unity 3D Runtime Animation GenerationDustin Graham
 
Converting a naive flow to akka streams
Converting a naive flow to akka streamsConverting a naive flow to akka streams
Converting a naive flow to akka streamsGal Topper
 
Workshop 22: React-Redux Middleware
Workshop 22: React-Redux MiddlewareWorkshop 22: React-Redux Middleware
Workshop 22: React-Redux MiddlewareVisual Engineering
 
Functional Programming with JavaScript
Functional Programming with JavaScriptFunctional Programming with JavaScript
Functional Programming with JavaScriptMark Shelton
 

Was ist angesagt? (20)

A dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenarioA dive into akka streams: from the basics to a real-world scenario
A dive into akka streams: from the basics to a real-world scenario
 
CLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.jsCLS & asyncListener: asynchronous observability for Node.js
CLS & asyncListener: asynchronous observability for Node.js
 
Callbacks and control flow in Node js
Callbacks and control flow in Node jsCallbacks and control flow in Node js
Callbacks and control flow in Node js
 
Asynchronous handlers in asp.net
Asynchronous handlers in asp.netAsynchronous handlers in asp.net
Asynchronous handlers in asp.net
 
Using akka streams to access s3 objects
Using akka streams to access s3 objectsUsing akka streams to access s3 objects
Using akka streams to access s3 objects
 
Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...Using SaltStack to orchestrate microservices in application containers at Sal...
Using SaltStack to orchestrate microservices in application containers at Sal...
 
Go With The Flow
Go With The FlowGo With The Flow
Go With The Flow
 
Elixir at Evercam (By Milos Mosic)
Elixir at Evercam (By Milos Mosic)Elixir at Evercam (By Milos Mosic)
Elixir at Evercam (By Milos Mosic)
 
Asynchronní programování
Asynchronní programováníAsynchronní programování
Asynchronní programování
 
Neighborly nagios
Neighborly nagiosNeighborly nagios
Neighborly nagios
 
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
MongoDB World 2016: Implementing Async Networking in MongoDB 3.2
 
Javascript internals
Javascript internalsJavascript internals
Javascript internals
 
Reactive mistakes - ScalaDays Chicago 2017
Reactive mistakes -  ScalaDays Chicago 2017Reactive mistakes -  ScalaDays Chicago 2017
Reactive mistakes - ScalaDays Chicago 2017
 
Unity 3D Runtime Animation Generation
Unity 3D Runtime Animation GenerationUnity 3D Runtime Animation Generation
Unity 3D Runtime Animation Generation
 
Converting a naive flow to akka streams
Converting a naive flow to akka streamsConverting a naive flow to akka streams
Converting a naive flow to akka streams
 
Async Web QA
Async Web QAAsync Web QA
Async Web QA
 
Workshop 22: React-Redux Middleware
Workshop 22: React-Redux MiddlewareWorkshop 22: React-Redux Middleware
Workshop 22: React-Redux Middleware
 
Functional Programming with JavaScript
Functional Programming with JavaScriptFunctional Programming with JavaScript
Functional Programming with JavaScript
 
Gearman & PHP
Gearman & PHPGearman & PHP
Gearman & PHP
 
Hadoop Map Reduce Arch
Hadoop Map Reduce ArchHadoop Map Reduce Arch
Hadoop Map Reduce Arch
 

Ähnlich wie Orbiter and how to extend Docker Swarm

ILM - Pipeline in the cloud
ILM - Pipeline in the cloudILM - Pipeline in the cloud
ILM - Pipeline in the cloudAaron Carey
 
Easy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp NomadEasy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp NomadBram Vogelaar
 
Docker swarm introduction
Docker swarm introductionDocker swarm introduction
Docker swarm introductionEvan Lin
 
Building a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless frameworkBuilding a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless frameworkLuciano Mammino
 
Microservices with docker swarm and consul
Microservices with docker swarm and consulMicroservices with docker swarm and consul
Microservices with docker swarm and consulNguyen Sy Thanh Son
 
DCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless WorldDCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless WorldDocker, Inc.
 
Lightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just RightLightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just Rightmircodotta
 
Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...Codemotion
 
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...European Collaboration Summit
 
Dockerizing Rails
Dockerizing RailsDockerizing Rails
Dockerizing RailsiGbanam
 
Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!Rafał Leszko
 
Build your operator with the right tool
Build your operator with the right toolBuild your operator with the right tool
Build your operator with the right toolRafał Leszko
 
Docker in production service discovery with consul - road to opscon 2015
Docker in production  service discovery with consul - road to opscon 2015Docker in production  service discovery with consul - road to opscon 2015
Docker in production service discovery with consul - road to opscon 2015Giovanni Toraldo
 
Altitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architecturesAltitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architecturesFastly
 
Serverless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPandaServerless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPandaPaul Dykes
 
Docker 進階實務班
Docker 進階實務班Docker 進階實務班
Docker 進階實務班Philip Zheng
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 introTerry Cho
 

Ähnlich wie Orbiter and how to extend Docker Swarm (20)

ILM - Pipeline in the cloud
ILM - Pipeline in the cloudILM - Pipeline in the cloud
ILM - Pipeline in the cloud
 
Easy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp NomadEasy Cloud Native Transformation using HashiCorp Nomad
Easy Cloud Native Transformation using HashiCorp Nomad
 
Docker swarm introduction
Docker swarm introductionDocker swarm introduction
Docker swarm introduction
 
Building a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless frameworkBuilding a serverless company on AWS lambda and Serverless framework
Building a serverless company on AWS lambda and Serverless framework
 
Microservices with docker swarm and consul
Microservices with docker swarm and consulMicroservices with docker swarm and consul
Microservices with docker swarm and consul
 
DCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless WorldDCEU 18: Docker Containers in a Serverless World
DCEU 18: Docker Containers in a Serverless World
 
Lightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just RightLightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just Right
 
Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...Higher order infrastructure: from Docker basics to cluster management - Nicol...
Higher order infrastructure: from Docker basics to cluster management - Nicol...
 
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
ECS19 - Damir Dobric - Designing and Operating modern applications with Micro...
 
Dockerizing Rails
Dockerizing RailsDockerizing Rails
Dockerizing Rails
 
Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!Build Your Kubernetes Operator with the Right Tool!
Build Your Kubernetes Operator with the Right Tool!
 
Build your operator with the right tool
Build your operator with the right toolBuild your operator with the right tool
Build your operator with the right tool
 
Docker in production service discovery with consul - road to opscon 2015
Docker in production  service discovery with consul - road to opscon 2015Docker in production  service discovery with consul - road to opscon 2015
Docker in production service discovery with consul - road to opscon 2015
 
Altitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architecturesAltitude SF 2017: Nomad and next-gen application architectures
Altitude SF 2017: Nomad and next-gen application architectures
 
Serverless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPandaServerless APIs with JavaScript - Matt Searle - ChocPanda
Serverless APIs with JavaScript - Matt Searle - ChocPanda
 
Swarm mode
Swarm modeSwarm mode
Swarm mode
 
Docker 進階實務班
Docker 進階實務班Docker 進階實務班
Docker 進階實務班
 
Why Grails?
Why Grails?Why Grails?
Why Grails?
 
Why Grails
Why GrailsWhy Grails
Why Grails
 
Kubernetes #1 intro
Kubernetes #1   introKubernetes #1   intro
Kubernetes #1 intro
 

Mehr von Gianluca Arbezzano

OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringGianluca Arbezzano
 
Value of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observabilityValue of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observabilityGianluca Arbezzano
 
InfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestratorInfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestratorGianluca Arbezzano
 
Overview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca ArbezzanoOverview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca ArbezzanoGianluca Arbezzano
 
Monitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and PrometheusMonitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and PrometheusGianluca Arbezzano
 
Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017Gianluca Arbezzano
 
Security Tips to run Docker in Production
Security Tips to run Docker in ProductionSecurity Tips to run Docker in Production
Security Tips to run Docker in ProductionGianluca Arbezzano
 
Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017Gianluca Arbezzano
 
Monitor your application and sleep
Monitor your application and sleepMonitor your application and sleep
Monitor your application and sleepGianluca Arbezzano
 
Tick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleepTick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleepGianluca Arbezzano
 
Docker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in ProductionDocker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in ProductionGianluca Arbezzano
 
DockerDublin Meetup - News about Docker 1.13
DockerDublin Meetup -  News about Docker 1.13DockerDublin Meetup -  News about Docker 1.13
DockerDublin Meetup - News about Docker 1.13Gianluca Arbezzano
 
Time Series Database and Tick Stack
Time Series Database and Tick StackTime Series Database and Tick Stack
Time Series Database and Tick StackGianluca Arbezzano
 
Queue System and Zend\Queue implementation
Queue System and Zend\Queue implementationQueue System and Zend\Queue implementation
Queue System and Zend\Queue implementationGianluca Arbezzano
 
ZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everythingZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everythingGianluca Arbezzano
 

Mehr von Gianluca Arbezzano (19)

Kubernetes debug like a pro
Kubernetes debug like a proKubernetes debug like a pro
Kubernetes debug like a pro
 
OSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoringOSDC 2018 - Distributed monitoring
OSDC 2018 - Distributed monitoring
 
Value of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observabilityValue of your metrics: goodbye monitoring, welcome observability
Value of your metrics: goodbye monitoring, welcome observability
 
InfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestratorInfluxCloudi craft container orchestrator
InfluxCloudi craft container orchestrator
 
Overview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca ArbezzanoOverview and Opentracing in theory by Gianluca Arbezzano
Overview and Opentracing in theory by Gianluca Arbezzano
 
Monitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and PrometheusMonitoring Pull vs Push, InfluxDB and Prometheus
Monitoring Pull vs Push, InfluxDB and Prometheus
 
Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017Open Tracing, to order and understand your mess. - ApiConf 2017
Open Tracing, to order and understand your mess. - ApiConf 2017
 
Security Tips to run Docker in Production
Security Tips to run Docker in ProductionSecurity Tips to run Docker in Production
Security Tips to run Docker in Production
 
Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017Jenkins in the real world - DevOpsCon 2017
Jenkins in the real world - DevOpsCon 2017
 
Monitor your application and sleep
Monitor your application and sleepMonitor your application and sleep
Monitor your application and sleep
 
Tick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleepTick Stack - Listen your infrastructure and please sleep
Tick Stack - Listen your infrastructure and please sleep
 
Docker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in ProductionDocker Novosibirsk Meetup #3 - Docker in Production
Docker Novosibirsk Meetup #3 - Docker in Production
 
DockerDublin Meetup - News about Docker 1.13
DockerDublin Meetup -  News about Docker 1.13DockerDublin Meetup -  News about Docker 1.13
DockerDublin Meetup - News about Docker 1.13
 
Docker 1.12 and SwarmKit
Docker 1.12 and SwarmKitDocker 1.12 and SwarmKit
Docker 1.12 and SwarmKit
 
Time Series Database and Tick Stack
Time Series Database and Tick StackTime Series Database and Tick Stack
Time Series Database and Tick Stack
 
Queue System and Zend\Queue implementation
Queue System and Zend\Queue implementationQueue System and Zend\Queue implementation
Queue System and Zend\Queue implementation
 
ZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everythingZfDayIt 2014 - There is a module for everything
ZfDayIt 2014 - There is a module for everything
 
Vagrant - PugMI
Vagrant - PugMIVagrant - PugMI
Vagrant - PugMI
 
Silex, iniziamo
Silex, iniziamoSilex, iniziamo
Silex, iniziamo
 

Kürzlich hochgeladen

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 

Kürzlich hochgeladen (20)

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 

Orbiter and how to extend Docker Swarm

  • 3. Why? ● To create a use case around metrics, InfluxDB and Docker. ● Docker Swarm at the moment doesn’t provider autoscale capabilities. ● To design an application easy to manage and well integrated in Docker Swarm.
  • 4. Orbiter is a general-purpose multi vendor autoscaler
  • 5. Orbiter is a general-purpose multi vendor autoscaler Orbiter is a Docker Swarm autoscaler
  • 6. swarm cluster manager get metrics: ● cadvidor ● telegraf ● …. alert: ● custom ● kapacitor ● alert manager
  • 7. Why Docker Swarm doesn’t have autoscaling? ● I don’t know. ● As you can see the architecture required is not small. ● Probably it will have autoscaling at some point.
  • 8. Orbiter is designed to be Only-Swarm ● To make it easy to start ● To remove abstractions ● Because less is more! cit unknown but for sure a lazy person
  • 9. What does it? ● It lists all your services and it gets only who has label orbiter=true ● Expose and HTTP API to trigger up and down scale event ○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/up ○ curl -X POST -v http://$(IP):8081/v1/orbiter/handle/autoswarm/stack_micro/down ● It uses Docker Swarm Api to keep in sync the managed services ● You can configure a service via label orbiter.up=3, orbiter.donw=2, orbiter.cooldown = 2
  • 10. Requirement: ● Deploy it in a manager.
  • 11. Deploy it as service in Docker Swarm version: '3' services: orbiter: image: gianarb/orbiter command: daemon ports: - 8081:8000 volumes: - /var/run/docker.sock:/var/run/docker.sock deploy: placement: constraints: - node.role == manager mode: replicated replicas: 1
  • 12. How Orbiter communicate with Swarm func NewSwarmProvider(c map[string]string) (autoscaler.Provider, error) { var p autoscaler.Provider client, err := docker.NewEnvClient() if err != nil { logrus.WithField("error", err).Warn("problem to communicate with docker") return p, err } else { logrus.Info("Successfully connected to a Docker daemon") } p = SwarmProvider{ dockerClient: client, } return p, nil }
  • 13.
  • 14. Scale a service - 1.Find the service service, _, err := p.dockerClient.ServiceInspectWithRaw(ctx, serviceId, types.ServiceInspectOptions{ InsertDefaults: true, }) if err != nil { logrus.WithFields(logrus.Fields{ "error": err.Error(), "provider": "swarm", }).Debugf("Service %s didn't scale. We didn't get it from docker.", serviceId) return err } filters := filters.NewArgs() filters.Add("service", serviceId) tasks, err := p.dockerClient.TaskList(ctx, types.TaskListOptions{Filters: filters}) if err != nil { … } err = p.isAcceptable(tasks, target, direction)
  • 15. Scale a service - 2.Update the Spec spec := service.Spec var ptrFromSystem uint64 base := p.calculateActiveTasks(tasks) if direction == true { ptrFromSystem = uint64(base + target) } else { ptrFromSystem = uint64(base - target) } spec.Mode.Replicated.Replicas = &ptrFromSystem _, err = p.dockerClient.ServiceUpdate(ctx, serviceId, service.Version, spec, types.ServiceUpdateOptions{}) if err != nil {...}
  • 16. Background job to sync services with Docker services, err := dockerClient.ServiceList(ctx, types.ServiceListOptions{}) if err != nil { logrus.WithField("error", err).Debug("Bad communication with Docker.") return } prov, _ := provider.NewSwarmProvider(map[string]string{}) for _, service := range services { s, err := getAutoscalerByService(prov, service.Spec.Annotations) if err != nil { continue } c.Autoscalers[fmt.Sprintf("autoswarm/%s", service.Spec.Annotations.Name)] = s }
  • 17. Is orbiter just a wrapper on Docker Swarm API? Let’s put that in a cooler way. Orbiter provides easy to use rest API to scale your services up and down.
  • 19. Demo
  • 20.