"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzoni, dotCloud

привет!
меня зовут Джером
Я не говорю по России
(к сожалению)

Lightweight Virtualization
with
Linux Containers
and
Docker
Yet another Conference – Moscow, 2013
Jérôme Petazzoni, dotCloud Inc.

Яндекс:
спасибо большое!

Outline
● Why Linux Containers?
● What are Linux Containers exactly?
● What do we need on top of LXC?
● Why Docker?
● What is Docker exactly?
● Where is it going?

Why Linux Containers?
What are
we trying
to solve?

Many payloads
● backend services (API)
● databases
● distributed stores
● webapps

Many payloads
● Go
● Java
● Node.js
● PHP
● Python
● Ruby
● …

Many payloads
● CherryPy
● Django
● Flask
● Plone
● ...

Many payloads
● Apache
● Gunicorn
● uWSGI
● ...

Many targets
● your local development environment
● your coworkers' developement environment
● your Q&A team's test environment
● some random demo/test server
● the staging server(s)
● the production server(s)
● bare metal
● virtual machines
● shared hosting

Many targets
● BSD
● Linux
● OS X
● Windows

Many targets
● BSD
● Linux
● OS X
● Windows
Not yet

The Matrix From Hell
Static website ? ? ? ? ? ? ?
Web frontend ? ? ? ? ? ? ?
background workers ? ? ? ? ? ? ?
User DB ? ? ? ? ? ? ?
Analytics DB ? ? ? ? ? ? ?
Queue ? ? ? ? ? ? ?
DevelopmentVM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’s laptop Customer Servers

Real-world analogy:
containers

Many products
● clothes
● electronics
● raw materials
● wine
● …

Many transportation methods
● ships
● trains
● trucks
● ...

Another matrix from hell
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?

Solution to the transport problem:
the intermodal shipping container

Solution to the transport problem:
the intermodal shipping container
● 90% of all cargo now shipped in a standard container
● faster and cheaper to load and unload on ships
(by an order of magnitude)
● less theft, less damage
● freight cost used to be >25% of final goods cost, now <3%
● 5000 ships deliver 200M containers per year

Solution to the deployment problem:
the Linux container

Linux containers...
● run everywhere
– regardless of kernel version
– regardless of host distro
– (but container and host architecture must match*)
● run anything
– if it can run on the host, it can run in the container
– i.e., if it can run on a Linux kernel, it can run
*Unless you emulate CPU with qemu and binfmt

What are Linux Containers exactly?

High level approach:
it's a lightweight VM
● own process space
● own network interface
● can run stuff as root
● can have its own /sbin/init
(different from the host)
« Machine Container »

Low level approach:
it's chroot on steroids
● can also not have its own /sbin/init
● container = isolated process(es)
● share kernel with host
● no device emulation (neither HVM nor PV)
« Application Container »

Separation of concerns:
Dmitry the Developer
● inside my container:
– my code
– my libraries
– my package manager
– my app
– my data

Separation of concerns:
Oleg the Ops guy
● outside the container:
– logging
– remote access
– network configuration
– monitoring

How does it work?
Isolation with namespaces
● pid
● mnt
● net
● uts
● ipc
● user

pid namespace
jpetazzo@tarrasque:~$ ps aux | wc -l
212
jpetazzo@tarrasque:~$ sudo docker run -t -i ubuntu bash
root@ea319b8ac416:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 18044 1956 ? S 02:54 0:00 bash
root 16 0.0 0.0 15276 1136 ? R+ 02:55 0:00 ps aux
(That's 2 processes)

mnt namespace
jpetazzo@tarrasque:~$ wc -l /proc/mounts
32 /proc/mounts
root@ea319b8ac416:/# wc -l /proc/mounts
10 /proc/mounts

net namespace
root@ea319b8ac416:/# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
22: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 2a:d1:4b:7e:bf:b5 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.3/24 brd 10.1.1.255 scope global eth0
inet6 fe80::28d1:4bff:fe7e:bfb5/64 scope link

uts namespace
jpetazzo@tarrasque:~$ hostname
tarrasque
root@ea319b8ac416:/# hostname
ea319b8ac416

ipc namespace
jpetazzo@tarrasque:~$ ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes
nattch status
0x00000000 3178496 jpetazzo 600 393216 2
dest
0x00000000 557057 jpetazzo 777 2778672 0
0x00000000 3211266 jpetazzo 600 393216 2
dest
root@ea319b8ac416:/# ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes

user namespace
● no « demo » for this one... Yet!
● UID 0→1999 in container C1 is mapped to
UID 10000→11999 in host;
UID 0→1999 in container C2 is mapped to
UID 12000→13999 in host; etc.
● required lots of VFS and FS patches (esp. XFS)
● what will happen with copy-on-write?
– double translation at VFS?
– single root UID on read-only FS?

How does it work?
Isolation with cgroups
● memory
● cpu
● blkio
● devices

memory cgroup
● keeps track pages used by each group:
– file (read/write/mmap from block devices; swap)
– anonymous (stack, heap, anonymous mmap)
– active (recently accessed)
– inactive (candidate for eviction)
● each page is « charged » to a group
● pages can be shared (e.g. if you use any COW FS)
● Individual (per-cgroup) limits and out-of-memory killer

cpu and cpuset cgroups
● keep track of user/system CPU time
● set relative weight per group
● pin groups to specific CPU(s)
– Can be used to « reserve » CPUs for some apps
– This is also relevant for big NUMA systems

blkio cgroups
● keep track IOs for each block device
– read vs write; sync vs async
● set relative weights
● set throttle (limits) for each block device
– read vs write; bytes/sec vs operations/sec
Note: earlier versions (pre-3.8) didn't account async correctly.
3.8 is better, but use 3.10 for best results.

devices cgroups
● controls read/write/mknod permissions
● typically:
– allow: /dev/{tty,zero,random,null}...
– deny: everything else
– maybe: /dev/net/tun, /dev/fuse

If you're serious about security,
you also need…
● capabilities
– okay: cap_ipc_lock, cap_lease, cap_mknod, cap_net_admin,
cap_net_bind_service, cap_net_raw
– troublesome: cap_sys_admin (mount!)
● think twice before granting root
● grsec is nice
● seccomp (very specific use cases); seccomp-bpf
● beware of full-scale kernel exploits!

Efficiency: almost no overhead
● processes are isolated, but run straight on the host
● CPU performance
= native performance
● memory performance
= a few % shaved off for (optional) accounting
● network performance
= small overhead; can be optimized to zero overhead

Efficiency: storage-friendly
● unioning filesystems
(AUFS, overlayfs)
● snapshotting filesystems
(BTRFS, ZFS)
● copy-on-write
(thin snapshots with LVM or device-mapper)
This is now being integrated with low-level LXC tools as well!

Efficiency: storage-friendly
● provisioning now takes a few milliseconds
● … and a few kilobytes
● creating a new base image (from a running container) takes a
few seconds (or even less)

What can Docker do?
● Open Source engine to commoditize LXC
● using copy-on-write for quick provisioning
● allowing to create and share images
● standard format for containers
(stack of layers; 1 layer = tarball+metadata)
● standard, reproducible way to easily build trusted images
(Dockerfile)

Docker: authoring images
● you can author « images »
– either with « run+commit » cycles, taking snapshots
– or with a Dockerfile (=source code for a container)
– both ways, it's ridiculously easy
● you can run them
– anywhere
– multiple times

Dockerfile example
FROM ubuntu
RUN apt-get -y update
RUN apt-get install -y g++
RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...
RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...
RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-
RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]nport = 8101nbind_address = 0.0.0.0" >
/usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101
CMD ["/usr/local/bin/couchdb"]

Yes, but...
● « I don't need Docker;
I can do all that stuff with LXC tools, rsync, some scripts! »
● correct on all accounts;
but it's also true for apt, dpkg, rpm, yum, etc.
● the whole point is to commoditize,
i.e. make it ridiculously easy to use

What this really means…
● instead of writing « very small shell scripts » to
manage containers, write them to do the rest:
– continuous deployment/integration/testing
– orchestration
● = use Docker as a building block
● re-use other people images (yay ecosystem!)

Docker: sharing images
● you can push/pull images to/from a registry
(public or private)
● you can search images through a public index
● dotCloud maintains a collection of base images
(Ubuntu, Fedora...)
● satisfaction guaranteed or your money back

Docker: not sharing images
● private registry
– for proprietary code
– or security credentials
– or fast local access
● the private registry is available
as an image on the public registry
(yes, that makes sense)

Typical workflow
● code in local environment
(« dockerized » or not)
● each push to the git repo triggers a hook
● the hook tells a build server to clone the code and run « docker
build » (using the Dockerfile)
● the containers are tested (nosetests, Jenkins...),
and if the tests pass, pushed to the registry
● production servers pull the containers and run them
● for network services, load balancers are updated

Hybrid clouds
● Docker is part of OpenStack « Havana »,
as a Nova driver + Glance translator
● typical workflow:
– code on local environment
– push container to Glance-backed registry
– run and manage containers using OpenStack APIs

What's Docker exactly?
● rewrite of dotCloud internal container engine
– original version: Python, tied to dotCloud's internal stuff
– released version: Go, legacy-free
● the Docker daemon runs in the background
– manages containers, images, and builds
– HTTP API (over UNIX or TCP socket)
– embedded CLI talking to the API
● Open Source (GitHub public repository + issue tracking)
● user and dev mailing lists

Docker: the community
● Docker: >160 >170 contributors
● latest milestone (0.6): 40 contributors
● GitHub repository: >600 >680 forks

Docker roadmap
● Today: Docker 0.6
– LXC
– AUFS
● Tomorrow: Docker 0.7
– LXC
– device-mapper thin snapshots (target: RHEL)
● The day after: Docker 1.0
– LXC, libvirt, qemu, KVM, OpenVZ, chroot…
– multiple storage back-ends
– plugins

Docker: the ecosystem
● Cocaine (PAAS; has Docker plugin)
● CoreOS (full distro based on Docker)
● Deis (PAAS; available)
● Dokku (mini-Heroku in 100 lines of bash)
● Flynn (PAAS; in development)
● Maestro (orchestration from a simple YAML file)
● OpenStack integration (in Havana, Nova has a Docker driver)
● Shipper (fabric-like orchestration)

Cocaine integration
● what's Cocaine?
– Open Source PaaS from Yandex
– modular: can switch logging, storage, etc. without changing apps
– infrastructure abstraction layer + service discovery
– monitoring: metrics collection; load balancing
● why Docker?
– Cocaine initially used cgroups
– wanted to add LXC for better isolation and resource control
– heard about Docker at the right time

device-mapper thin snapshots
(aka « thinp »)
● start with a 10 GB empty ext4 filesystem
– snapshot: that's the root of everything
● base image:
– clone the original snapshot
– untar image on the clone
– re-snapshot; that's your image
● create container from image:
– clone the image snapshot
– run; repeat cycle as many times as needed

AUFS vs THINP
AUFS
● easy to see changes
● small change =
copy whole file
● ~42 layers
● patched kernel
(Debian, Ubuntu OK)
● efficient caching
● no quotas
THINP
● must diff manually
● small change =
copy 1 block (100k-1M)
● unlimited layers
● stock kernel (>3.2)
(RHEL 2.6.32 OK)
● duplicated pages
● FS size acts as quota

Misconceptions about THINP
● « performance degradation »
no; that was with « old » LVM snapshots
● « can't handle 1000s of volumes »
that's LVM; Docker uses devmapper directly
● « if snapshot volume is out of space,
it breaks and you lose everything »
that's « old » LVM snapshots; thinp halts I/O
● « if still use disk space after 'rm -rf' »
no, thanks to 'discard passdown'

Other features in 0.7
● links
– linked containers can discover each other
– environment variable injection
– allows to expose remote services thru containers
(implements the ambassador pattern)
– side-effect: container naming
● host integration
– we ♥ systemd

0.8 and beyond
● beam
– introspection API
– based on Redis protocol
(i.e. all Redis clients work)
– works well for synchronous req/rep and streams
– reimplementation of Redis core in Go
– think of it as « live environment variables »,
that you can watch/subscribe to
● and much more

Thank you! Questions?
http://docker.io/
https://github.com/dotcloud/docker
@docker
@jpetazzo

"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzoni, dotCloud

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (6)

Ähnlich wie "Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzoni, dotCloud

Ähnlich wie "Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzoni, dotCloud (20)

Mehr von Yandex

Mehr von Yandex (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzoni, dotCloud