This talk was presented in October at the Docker Beijing Meetup, in the VMware offices.
It presents some of the latest features of Docker, discusses orchestration possibilities with Docker, then gives a briefing about the performance of containers; and finally shows how to use volumes to decouple components in your applications.
2. Jérôme Petazzoni (@jpetazzo)
Grumpy French DevOps
- Go away or I will replace you with a very small shell script
Wrote dotCloud PAAS deployment tools
- EC2, LXC, Puppet, Python, Shell, ØMQ...
Docker contributor
- Security, networking...
Runs all kinds of crazy things in Docker
- Docker-in-Docker, VPN-in-Docker,
KVM-in-Docker, Xorg-in-Docker...
3. Outline
Some new features that you should know about
The Docker orchestration flowchart
Measuring and optimizing container performance
You should use volumes
5. Docker 0.11
SELinux integration
(works better with CentOS)
DNS integration for links
(access linked containers by hostname)
docker run --net
- use host networking for high speed
- share network of another container
13. Orchestration
There's more than one way to do it
- describe your stack in files
(Fig, Maestro-NG, Ansible and other CMs)
- submit requests through an API
(Mesos, Kubernetes, Helios...)
- implement something that looks like a PAAS
(Flynn, Deis, OpenShift...)
- OpenStack (because OpenStack can do everything!)
15. Do you (want to) use OpenStack?
Yes
- if you are building a PAAS, keep an eye on Solum
(and consider contributing)
- if you are moving VM workloads to containers, use Nova
(that's probably what you already have; just enable the Docker driver)
- otherwise, use Heat
(and use Docker resources in your Heat templates)
No
- go to next slide
17. Good question: to PAAS or not to PAAS?
PAAS does not solve problems
- PAAS puts all* your problems in one place
- now you have N identical problems instead of N different problems
All your applications must be standardized
- so that they all have the same problem (instead of different ones)
It's much harder to operate a PAAS than a single app
- in other words: PAAS is great if you have many apps
*Well, not all your problems, but things like database failover, high
availability, scaling...
19. Are you looking for a PAAS?
Yes
- CloudFoundry (Ruby, but increasing % Go)
- Deis (Python, Docker-ish, runs on top of CoreOS)
- Dokku (A few 100s of line of Bash!)
- Flynn (Go, bleeding edge)
- Tsuru (Go, more mature)
- OpenShift geard (Go again!)
Choose wisely (or go to the next slide)
- http://blog.lusis.org/blog/2014/06/14/paas-for-realists/
“I don’t think ANY of the current private PaaS solutions are a fit right now.”
20. If you have only one host
Fig (www.fig.sh)
fig.yml:
web:
build: .
command: python app.py
links:
- db
ports:
- "8000:8000"
db:
image: postgres
21. If you have a few hosts (10s)
Maestro-NG
(https://github.com/signalfuse/maestro-ng)
- fig-like YAML file
- can talk to multiple hosts
- manual placement
Your favorite Configuration Management system
- Ansible, Chef, Puppet, Salt: have Docker modules
- use CM to deploy hosts and start containers
- use Dockerfiles to deploy code & dependencies, libraries, packages
22. If you have many hosts (100s)
Helios
- Java
- needs ZK, a master server, and one agent per host
<empty spot>
<empty spot>
<empty spot>
Hmmm... There might be a start-up opportunity there
23. If you have many many hosts (1000s)
Mesos
- C++
- needs ZK, a master server, and one agent per host
- and probably a few other standby servers for HA
- and frameworks; e.g.:
https://github.com/VoltFramework/volt
https://github.com/mesosphere/marathon
Kubernetes
- work in progress
25. Gathering metrics
cgroups give us per-container...
- CPU usage
- memory usage (fine-grained: cache and resident set size)
- I/O usage (per device, reads vs writes, in bytes and in ops)
cgroups don't give us...
- network metrics (have to do tricks with network namespaces)
https://github.com/google/cadvisor
http://jpetazzo.github.io/2013/10/08/docker-containers-metrics/
27. I/O performance
Working set should be on a volume
Volume performance is native in all benchmarks
28. Memory performance
Memory control group has an overhead
Overhead happens when memory is given by the
kernel to the container, or reclaimed back
Overhead is not related to memory allocations
Disabling the memory control group = native speed
But it is a global operation (affects all containers)
… And requires a reboot
29. Network performance
Linux bridge = overhead
IPTables = overhead
docker run --net host = native speed
- but loss of isolation
SR/IOV and macvlan = almost native speed
- better performance than VMs
- maintain isolation
31. What is a volume?
Special directory in a container
Mapped to normal directory on the host
Can be shared by multiple containers
32. When should we use volumes?
Bypass copy-on-write system
- fast I/O path with zero overhead
- keep data across container upgrades
Use specific storage device in container
- e.g. SAN, or fast SSD RAID for database...)
Share data between containers
- this is cool, and let's see why!
33. Logging with volumes
Write log files to a volume
docker run --name logs -v /var/log busybox true
docker run --volumes-from logs myapp
Inspect logs
docker run --rm --volumes-from logs ubuntu bash
Ship logs to something else (logstash, syslog...)
docker run --volumes-from logs pipestash
34. Backups with volumes
Data files should be in a volume
docker run --name mysqldata -v /var/lib/mysql busybox true
docker run --volumes-from mysqldata mysql
Run backup job in a separate container
docker run --rm --volumes-from mysqldata mysqlbackup
tar -cJf- /var/lib/mysql | stream-it-to-the-cloud.py
Of course, you can use anything fancier than tar
(e.g. rsync, tarsnap...)
35. Moving containers and volumes around
If the container is stateless (web app...):
- get the image to the new machine
- start the new container
- reconfigure load balancers
If the container is stateful (DB...):
- Flocker
- Flocker
- Flocker
- or move volumes around and do the network plumbing yourself
36. More information about volumes
Docker Docs:
https://docs.docker.com/userguide/dockervolumes/
Additional insights:
http://blog.docker.com/2014/06/why-you-dont-need-to-run-sshd-in-docker/
37. Not an actual book (yet) Thank you!
Docker
advanced concepts
Containers, containers everywhere!
Questions?
www.docker.com
@docker
@jpetazzo