More developers on DevOps with Docker orchestration
•Als PPTX, PDF herunterladen•
0 gefällt mir•673 views
Few things you have to Know in Order to use docker with Orchestration in Production like I did, but not really like i did, because I cried, I really cried hard! Three way devops, pipeline...
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie More developers on DevOps with Docker orchestration
Ähnlich wie More developers on DevOps with Docker orchestration (20)
Mehr von Giulio De Donato
Mehr von Giulio De Donato (13)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
More developers on DevOps with Docker orchestration
- 1. moreDEVsonDEVOPSwith DOCKERORCHESTRATION All the Things you have to Know in OrdertousedockerwithOrchestrationinProductionlikeIdidbutnotreallylikeididitbecauseIcried,Ireallycriedhard,sopleasedonotcrybecauselifeisshortandyoushouldspendyourtimeenjoyingit.
- 2. 2. @liuggio Giulio De Donato @liuggiowelcometothebundle.com
- 3. moreDEVsonDEVOPSwith DOCKERORCHESTRATION All the Things you have to Know in OrdertousedockerwithOrchestrationinProductionlikeIdidbutnotreallylikeididitbecauseIcried,Ireallycriedhard,sopleasedonotcrybecauselifeisshortandyoushouldspendyourtimeenjoyingit.
- 4. @liuggio Giulio De Donato 1 Docker Container!==
- 5. @liuggio Giulio De Donato Virtualization !== ContainerContainer are mostly syscalls2
- 6. @liuggio Giulio De Donato 3 It is all about while true; do mkdir x; cd x; done bomb() { bomb | bomb & }; bomb ISOLATION
- 7. @liuggio Giulio De Donato 4 DevOps is ...
- 10. ● VELOCITY ● Variation ● VisualizationLEAD TIME Threewaydevops:
- 11. @liuggio Giulio De Donato Infrastructure like a managed application 5 Infrastructure like a managed application
- 12. @liuggio Giulio De Donato 6 Use the FORCE One process per container
- 13. @liuggio Giulio De Donato 7 Use the FORCE log to StdErr and StdOut
- 14. @liuggio Giulio De Donato Can’t touch this8
- 15. @liuggio Giulio De Donato 9 Use the FORCE Env. Variable all the things /etc/php5/fpm/pool.d/www.conf clear_env = no [www] env[MY_ENV_VAR_1] = 'value1' env[MY_ENV_VAR_2] = 'value2'
- 16. @liuggio Giulio De Donato 10 Containers communication is easy
- 17. NGINX PHP-FPM Data (code) php-cli @liuggio Giulio De Donato Cache DB 10 Containers communication is easy Backend expose port 80 ...Services
- 18. NGINX PHP-FPM Data (code) php-cli @liuggio Giulio De Donato Cache DB 10 Containers communication is easy N N N N PHP PHP PHP CLI expose port 80 Load balancer ...Scale
- 19. NGINX PHP-FPM Data (code) php-cli @liuggio Giulio De Donato 10 Containers communication is easy N N N N PHP PHP PHP CLI expose port 80 Load balancer NODE 3 NODE 2 NODE 1 ...Nodes
- 20. NGINX PHP-FPM Data (code) php-cli @liuggio Giulio De Donato 10 Containers communication is easy Cache DB ...SubNet
- 21. @liuggio Giulio De Donato 10 Containers communication is easy ...Microservices NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NX FPM ode) cli Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FP Data (cod php-cli NGINX PHP-FPM Data (code) php-cli Cache DB Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli Cach DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli Cache DB NGINX PHP-FPM Data (code) php-cli C
- 22. @liuggio Giulio De Donato 10 Containers communication is easy NOT FOR HUMAN!!!!
- 23. @liuggio Giulio De Donato 11 Orchestration !== Automation
- 24. ● Kubernetes ● DockerCloud ● DOCKERUCP ● MESOS ● Provisioningtool(BASHISM/…) ● … DockerOrchestrationframeworks/tools
- 25. @liuggio Giulio De Donato DEMO
- 26. Devops: The last human action happens when you commit? @liuggio Giulio De Donato
- 27. @liuggio Giulio De Donato 12 Understand your processes CODE TEST DEPLOY
- 28. @liuggio Giulio De Donato 13 Blue Green Deployment Load Balancer Web Blue Web Green
- 29. @liuggio Giulio De Donato 12 My pipeline (contd) CODE BUILD CHOOSE NODES Register Services RUN Containers (blue-green) Integration tests Configure Proxy with color Post Integration test PUSH to registry TEST PULL CONTAINER YOUR CUSTOMERS ARE HAPPY
- 30. @liuggio Giulio De Donato 14 Container registry and service discovery
- 31. CONSUL
- 32. @liuggio Giulio De Donato 12 My pipeline (contd) CODE BUILD CHOOSE NODES Register Services RUN Containers (blue-green) Integration tests Configure Proxy with color Post Integration test PUSH to registry TEST PULL CONTAINER YOUR CUSTOMERS ARE HAPPY
- 33. @liuggio Giulio De Donato 15 Graceful Deployment (contd) Load Balancer Web New Web Feature
- 34. @liuggio Giulio De Donato 16 You’ll need a lot of disk space
- 35. @liuggio Giulio De Donato 17 Resources ...
- 36. @liuggio Giulio De Donato 18 CONTAINER SERVICE A CONTAINER SERVICE A CONTAINER SERVICE A CONTAINER SERVICE B CONTAINER SERVICE B CONTAINER SERVICE B
- 37. @liuggio Giulio De Donato 18 DNS SRV is your friend CONTAINER SERVICE A CONTAINER SERVICE A CONTAINER SERVICE A CONTAINER SERVICE B CONTAINER SERVICE B CONTAINER SERVICE B
- 38. @liuggio Giulio De Donato 18 DNS SRV is your friend $ dig @192.168.99.100 bash_server.service.dc1.consul. SRV
- 39. @liuggio Giulio De Donato $consulHost = "bash_server.service.dc1.consul."; $results = dns_get_record($consulHost, DNS_SRV); // $result["host"] // $result["port"] // $result["pri"] // $result["weight"] // $result["class"] // $result["ttl"] // $result["type"] // $result["target"] DNS SRV is your friend18
- 40. @liuggio Giulio De Donato 19 Use a log handler -visualize-
- 41. @liuggio Giulio De Donato 20 Fast is better a b c Small Images eg. Alpine ~5mb DockerFile use layer cache wisely Build could be slow (if you don’t follow the docker-way) d Bad performance on large files
- 42. WinningKeys DEMO --- THE PERFECT ORCHESTRATION EasilyDEPLOY (LASTFEATUREPUSHED gracefully) EasilyRollback (LASTPUSHEDFEATUREgracefully) Visualize Share/Test/Commit/Merge Infrastructure liuggio
- 43. THE PERFECT ORCHESTRATION Processes respect your needs and: - The Lead time is SHORT - The Feedback is FAST - The Improvement loop is INFINITE @liuggio Giulio De Donato
- 44. fewthings you have to Know in Ordertousedockerwith OrchestrationinProductionlikeIdid,butnotreallylikeidid, becauseIcried,Ireallycriedhard,sopleasedonotcrybecauselifeis shortandyoushouldspendyourtimeenjoyingit. ThankYou @liuggiojoind.in/talk/4b24e
- 45. @liuggio Giulio De Donato ● http://www.infoq.com/articles/continuous-deployment-containers ● devops 2.0 the book ● www.welcometothebundle.com/isolate-a-process-with-no-container-like-docker ● https://github.com/opencontainers/runtime-spec/blob/master/config-linux.md#namespaces ● https://www.opencontainers.org/news/faqs/who-will-be-initial-technical-leadership ● http://www.cyberciti.biz/faq/unix-linux-chroot-command-examples-usage-syntax/ ● http://s0.cyberciti.org/uploads/faq/2013/01/bash-chroot-ls-demo.gif ● https://www.flockport.com/lxc-vs-docker/ ● http://ramirose.wix.com/ramirosen ● https://lwn.net/Articles/532593/ ● https://lwn.net/Articles/531114/ ● https://unsplash.com/photos/6wQId4r0uA4 CREDITS
Hinweis der Redaktion
- This talk is for developers that know what docker is, And maybe they already use in dev, Or in test env. How many of you is using dokcer in dev? And in production? This would be a gentle introduction to Docker ----------- 6 Fino ll’1 exl Fino al devops 12min
- It is 2016 and Docker is everywhere, in the workshop, in the conferences, startup sites, including Amazon and services in google app engine with kubernetes, got billion on big fund, all the big companies are moving to this trend. Can you read the real title This talk is about all the thinks I think you have to know in order to start working in production with docker... A lot things to know Did you read the title? Did you carefully read the title? The docker ecosystem is very loudy, this has lead to much hype, and some disillusion, around this space. We are here to cut through some of this confusion, and explains how containers are actually being used within the enterprise. Is organized in things to know…
- The docker ecosystem is very loudy, this has lead to much hype, and some disillusion, around this space. We are here to cut through some of this confusion, and explains how containers are actually being used within the enterprise. The developer worked hard to integrate … Things to know n1. Docker wasn’t a new technology... But the container story start really a lot of years ago Since 2002 this technology has made steps forward, thanks to different sponsors (IBM, Ubuntu, Google etc) Un po di storia... API! TODO storia So docker didn’t invent the container technology They added great marketing, great api, they added values and simplify the workflow for a developer Docker is a company is an open source organization that pushed forward allof this they used golang :) ---------- I held a workshop about docker and an attendee told me oh yes Docker i know the virtualization thing ------5min to devops
- Days ago when I talked to my friend about the workshop on docker I held, he said "ah the virtualization think?" … i answered yes but this association between virtualization and containers is conceptually similar but technically so far, Prima di arrivare alla ragione avete mai visto questa immagine, ovunque in tutti i talk si vede questa immagine, è semplice per simulare un isolamento non serve creare un altro systema operativo sopra al systema operativo esistente, eh no, Basta usare la tecnologia dei container che non sono altro che patch e syscall, Quindi non è virtualizzazione sono chiamate a sistema, l’isolamento è il problema. In 2012 Glauber Costa tells at the LinuxCon Europe: "I once heard That hypervisors are the living proof of operating system's incompetence" - Glauber Costa's - LinuxCon Europe 2012 Don’t you think that buying any hardware or create a sub-operating system (VM) only to defend against a fork bomb is a little too much?
- The bigger value Ci interessa veramente avere un sistema operativo diverso come host? Ci è sembrato che le VM potessero essere una idea geniale perche posso installare qualsiasi OS dentro un altro OS, ma il mercato ha dimostrato che la funzionalità e i casi di uso richeisti erano sopratutto basati sull’isolamento... Pensate a container come a scatole che eseguono uno o piu processi e che lo fanno in maniera isolata.
- Moving devs to devops I hope you know what development means :) Move to devops definition What is devops in order to move devs on devops We need to understand what devops is non c’è una risposta definitiva, cisno molte opinioni su cosa c’è sotto il tetto del devops e cosa no. è una cultura? è una job title? è un modo di organizzarsi o solo un modo di pensare, Possiamo anche pensare ad un movimento che sia in evoluzione, quindi non fermiamoci troppo nel dare una defizione adesso, Invece possiamo parlare di temi comuni, strumenti e idee. Nato con l’idea di migliorare il service delivery agility, il devops enfatizza la comunicazione, la collaborazione e l’integrazione tra software developer e IT operations. Piu che guardare questi in silos separati e paralleli, il DevOps riconosce che l’interdipendenza del software dev e IT operation e aiuta una organizzazione al fine di render Two silos : dev and ops Dev increase entropy Ops increase stability ----- devops fino (docker-way incluso) 14min 9 min fino a docker-way excl
- Moving devs to devops I hope you know what development means :) Move to devops definition What is devops in order to move devs on devops We need to understand what devops is non c’è una risposta definitiva, cisno molte opinioni su cosa c’è sotto il tetto del devops e cosa no. è una cultura? è una job title? è un modo di organizzarsi o solo un modo di pensare, Possiamo anche pensare ad un movimento che sia in evoluzione, quindi non fermiamoci troppo nel dare una defizione adesso, Invece possiamo parlare di temi comuni, strumenti e idee. Nato con l’idea di migliorare il service delivery agility, il devops enfatizza la comunicazione, la collaborazione e l’integrazione tra software developer e IT operations. Piu che guardare questi in silos separati e paralleli, il DevOps riconosce che l’interdipendenza del software dev e IT operation e aiuta una organizzazione al fine di render Two silos : dev and ops Dev increase entropy Ops increase stability
- A perfect storm of converging adjacent methodology including Agile, Operations Management (Systems Thinking & Dynamics), Theory of Constraints, LEAN and IT Service management came together in 2009 through a smattering of conferences, talks and Twitter (#devops) debates worldwide that eventually became the philosophy behind DevOps. Agile software development paved the way, steering away from the waterfall method of software development toward a continuous development cycle. Ma senza includere le operation side so while development could be continuous, deployment was still waterfall-oriented. In a DevOps environment, cross functionality, shared responsibilities and trust are promoted. DevOps essentially extends the continuous development goals of the Agile movement to continuous integration and release. In order to accommodate continuous releases, DevOps encourages automation of the change, configuration and release processes.
- An arrow with direction
- With the 3 way devops in mind in order to increase velocity, How do you handle your application? Do you use a versiong tool? Do you share easily? Can you rollback feature? Do you test it? Etc Quindi le regole di base è quella di disaccoppiare l’infrastruttura dall’applicazione e Gestire l’infrastruttura come una applicazione distinta Sharing the file, Reproduce Merge it Testing it…
- Tthe real difference with the Lcx (excluding the API) Is that on docker you have one process per container, Example in a webstack nginx-php-mysql you are not forced to it but is a Good practice Some in the community also says chat if you are migrating to docker Gentle migration but be careful because if you use orch. Tool It will default think that you are doing it.. Maybe at the beginning to this new methodologies But at the end you will understand that all the ecosystem will think about A container == a process .------------------ 7min to communication excl
- Docker the company really pushed the logging feature, You shouldn’t login to watch the error log because You will have thousand of container You will add variation manual activity Container are immutable ...
- Container immutable is not properly in the docker best practices but is on the devops best practices, Would be great if on github when you merge a new feature, the container like a box could be compiled Then this box is tested so you know is sane, And then you can move this box to production, that is tested, if you need to change it Why not you create a new feature and you create a new container so you can test it? If is immutable you can scale it, move across nodes (node for me are machines) Eg. don’t change, don’t upgrade version, don’t edit config file,
- Env in the container worlds are really important are used to pass property to different layers parameters (db etc) Is not a docker things is more a cloud platform as a services… In php you ST=1; echo $ST ST=2 echo $ST; #is still 1 :( ST=3 env | grep ST # is 3 WTF
- I container comincano solo in due modi, tramite i volumi e tramite le reti Docker pushes a lot of effort on improvmenton NETworks and volumes As we saw container should be read only immutible So easy in dev envonrment you have your simple way you have one container per service… Docker automatically modify /etc/hosts -----6 min fino a orch excl
- Let’s talk about a simple backend db and cache tier application You want to follow container per process you need nginx serves port 80, php-fpm ….. But this are services thatmeans that you can scale services with a randon container number A service is a group of container with the same image
- Container are great on scaling,
- Talking about nodes, per semplicità abbiamo tolto L’imporatante è avere il codice uno per nodo perche php-fpm and cli need the data on the filesystem in order to work, They need to mount it, The other services could be anywhere but they need to know where their dependences are… So we need to control where container are, each container will be
- Quindi abbiamo nginx che espose the 80 port to the external, we have php-fpm that communicate with nginx and must have access to the code, the db and the cache But we also have the commmand line interface the php commands we use it for migration, for cronjobs and the same way the C-L-I must communicate with cache, db and data… Can you recognize it? This is a mess! Eh immaginate invece di dividerli in container quante sottoreti e dati si devono colorare…
- Quindi abbiamo nginx che espose the 80 port to the external, we have php-fpm that communicate with nginx and must have access to the code, the db and the cache But we also have the commmand line interface the php commands we use it for migration, for cronjobs and the same way the C-L-I must communicate with cache, db and data… Can you recognize it? This is a mess! Eh immaginate invece di dividerli in container quante sottoreti e dati si devono colorare…
- Serve un qualcosa per automatizzare, la nostra regola madre ci dice che dobbiamo autmentare la velocity, diminuendo la variation Can you image handle all the subnet manually? Handle the Ip and change the ip on the load balance manually? Automation is the key… Orchestration is the key here… We need a tool that help us with infrastructures
- Overused word So we need something to automate and orchestrate our infrostracture OMG did I used in the same sentence orch e automate? Is bad CLICK … automation is not orchestration… Orchestration is when on domain processes Automation in about tasks So we need something to orchestrate the domain processes … In order to orchestrate our processes we need to understand what processes are...
- And there are some on the ecosystem using docker, This talk is intended to be orchestration agnostic, The community is an evolution Ma forse dobbiamo vedere il concetto un po piu da lontano forse l’orchestrazione cosa dovrebbe fare? Si abbiamo detto i processi aziendali ma quali? ---- Let’s dream for a moment on how would be the perfect pipeline… would be great when you finish to code a feature and you want to push that feature …
- Let’s dream for a moment on how would be the perfect pipeline… would be great when you finish to code a feature and you want to push that feature … https://asciinema.org/a/44936 Would be great to have the perfect orchestration, The perfect orchestration is that does all for you? Many of us have already experimented with Docker — for example, running one of the pre-built images from Docker Hub. It is possible that your team might have recognized the benefits that Docker, with experimentation, provides in building microservices and the advantages the technology could bring to development, testing, integration, and, ultimately, production. However, we must create a comprehensive build pipeline before deploying any containers into a live environment. Image we have an orchestration tool You are a developer so you create a new feature ... --------------------------------------------------------------------------------------------------------------- Last human action is when you push your code to a commit.
- Last human action is when you push your code to a commit. If you postpone the process and, for example, run it at the end of a sprint, neither testing nor deployment would be continuous but it depends from your company… But one things to say, postponing testing and deployment to production, you postpone the discovery of potential problems and as a result increase the effort required to correct them. Without testing, we have no guarantee that the service works. Without building it, there is nothing to deploy. Without deploying it, our users can’t benefit from the new release. Test: -> The tests would have a different complex infrastructure like mongo and other dep. You may want to extend your dockerfile for a test, and a docker-compose only for test purpose Build-> this is the simpler, you should compile and push to a container registry Deploy->
- It depends maybe starts from the processes Last human action is when you push your code to a commit. If you postpone the process and, for example, run it at the end of a sprint, neither testing nor deployment would be continuous. By postponing testing and deployment to production, you postpone the discovery of potential problems and as a result increase the effort required to correct them. Without testing, we have no guarantee that the service works. Without building it, there is nothing to deploy. Without deploying it, our users can’t benefit from the new release. Test: -> The tests would have a different complex infrastructure like mongo and other dep. You may want to extend your dockerfile for a test, and a docker-compose only for test purpose Build-> this is the simpler, you should compile and push to a container registry Deploy->
- Ricordo quando si facevano link simboli per risolvere l aggiunta di codice Provisioning tool like capistrano capifony… Ricordate quando i server venivano dati il nome perche un server era per sempre... Beh con i tool di provisioning un po...
- Ok eravamo al deploy quindi introducendo…
- QUalsiasi orchestratore usiate sicuramente se siete nell’abito di ambiente distribuito avrete a che fare con un service discovery/service registry Dato ch
- Cosa dovrebe fare? Perche è cosi utile? Perche proprio consul? È distribuito
- Ok eravamo al deploy quindi introducendo…
- Non ha garbace collector e non sa che cosa è da cancellare lo sai tu … Se si USA un frameworkdi astrazione è questo utilizza nodi attenzione a prendere nodi con poco disco distribuzione delle immagini
- Carefully set explicitly resources If you need at least 1gb of ram do no run thousand of container
- We said that communication is about net And identify a running container is ip:port is difficult How do you connect this? How to do you let container know where is the best container in the nearby?
- Do you know a protocl that convert name to ips? We said that communication is about net And identify a running container is ip:port is difficult How do you connect this? How to do you let container know where is the best container in the nearby?
- We said that communication is about net And identify a running container is ip:port is difficult You can avoid using a reverse proxy
- The developer worked hard to integrate ...hh
- The developer worked hard to integrate … You’ll need to know which container is slowing down You’ll need the output, and when You will need also the graph and a simple Yes is ok and no is bad. And for sure when you will evaluate grouping regexing… etc.
- Orchestrazione puo coinvolgere tante immagini e il build deve essere veloce Un solo servizio
- The developer worked hard to integrate ...