Docker containers provide significantly lower resource usage and higher density than traditional virtual machines when running multiple workloads concurrently on a server.
When booting 15 Ubuntu VMs with MySQL sequentially, Docker containers boot on average 3.5 seconds compared to 5.8 seconds for KVMs. During steady state operation of 15 active VMs, Docker uses on average 0.2% CPU and 49MB RAM per container, while KVMs use 1.9% CPU and 292MB RAM each. Docker maintains low 1-minute load averages of 0.15, while KVMs average 35.9% under load.
4. Increasing Revenue: Do More With Less
Reduce Total Cost of Ownership (TCO) and increase Return On Investment (ROI)
6/11/2014 4
Category Factors Scope
CAPEX
Hardware costs - VM density (consolidation ratio)
- Soft device integration
- Broad vendor compatibility
- Hypervisor
- Cloud manager
Software licensing costs - Software purchase price
- Support contracts
- Hypervisor
- Cloud manager
OPEX
Disaster recovery - Hypervisor
- Cloud manager
Upgrade / maintenance expenses - Hypervisor
- Cloud manager
Power & cooling costs - Reduced HW footprint - Hypervisor
- Cloud manager
Administration efficiency - Automated operations
- Performance / response time
- Hypervisor
- Cloud manager
Support & training costs - Hypervisor
- Cloud manager
AGILITY
Application delivery time - Workflow complexity
- Toolset costs
- Skillset
- Hypervisor
- Cloud manager
Planned / unplanned downtime - Hypervisor
- Cloud manager
*Not a complete or extensive list
5. About This Benchmark
Use case perspective
– As an OpenStack Cloud user I want a Ubuntu based VM with MySQL… Why would I choose
docker LXC vs a traditional hypervisor?
OpenStack “Cloudy” perspective
– LXC vs. traditional VM from a Cloudy (OpenStack) perspective
– VM operational times (boot, start, stop, snapshot)
– Compute node resource usage (per VM penalty); density factor
Guest runtime perspective
– CPU, memory, file I/O, MySQL OLTP, etc.
Why KVM?
– Exceptional performance
DISCLAIMERS
The tests herein are semi-active litmus tests – no in depth tuning,
analysis, etc. More active testing is warranted. These results do not
necessary reflect your workload or exact performance nor are they
guaranteed to be statistically sound.
6/11/2014 5
6. Docker in OpenStack
Havana
– Nova virt driver which integrates with docker REST API on backend
– Glance translator to integrate docker images with Glance
Icehouse
– Heat plugin for docker
Both options are still under development
6/11/2014 6
nova-docker virt driver docker heat plugin
DockerInc::Docke
r::Container
(plugin)
7. Benchmark Environment Topology @ SoftLayer
6/11/2014 7
glance api / reg
nova api / cond / etc
keystone
…
rally
nova api / cond / etc
cinder api / sch / vol
docker lxc
dstat
controller compute node
glance api / reg
nova api / cond / etc
keystone
…
rally
nova api / cond / etc
cinder api / sch / vol
KVM
dstat
controller compute node
+
Awesome!
+
Awesome!
8. Benchmark Specs
6/11/2014 8
Spec Controller Node (4CPU x 8G RAM) Compute Node (16CPU x 96G RAM)
Environment Bare Metal @ SoftLayer Bare Metal @ SoftLayer
Mother Board SuperMicro X8SIE-F Intel Xeon QuadCore SingleProc SATA
[1Proc]
SuperMicro X8DTU-F_R2 Intel Xeon HexCore DualProc [2Proc]
CPU Intel Xeon-Lynnfield 3470-Quadcore [2.93GHz] (Intel Xeon-Westmere 5620-Quadcore [2.4GHz]) x 2
Memory (Kingston 4GB DDR3 2Rx8 4GB DDR3 2Rx8 [4GB]) x2 (Kingston 16GB DDR3 2Rx4 16GB DDR3 2Rx4 [16GB]) x 6
HDD (LOCAL) Digital WD Caviar RE3 WD5002ABYS [500GB]; SATAII Western Digital WD Caviar RE4 WD5003ABYX [500GB]; SATAII
NIC eth0/eth1 @ 100 Mbps eth0/eth1 @100 Mbps
Operating System Ubuntu 12.04 LTS 64bit Ubuntu 12.04 LTS 64bit
Kernel 3.5.0-48-generic 3.8.0-38-generic
IO Scheduler deadline deadline
Hypervisor tested NA - KVM 1.0 + virtio + KSM (memory deduplication)
- docker 0.10.0 + go1.2.1 + commit dc9c28f + AUFS
OpenStack Trunk master via devstack Trunk master via devstack. Libvirt KVM nova driver / nova-docker
virt driver
OpenStack Benchmark
Client
OpenStack project rally NA
Metrics Collection NA dstat
Guest Benchmark Driver NA - Sysbench 0.4.12
- mbw 1.1.1.-2
- iibench (py)
- netperf 2.5.0-1
- Blogbench 1.1
- cpu_bench.py
VM Image NA - Scenario 1 (KVM): official ubuntu 12.04 image + mysql
snapshotted and exported to qcow2 – 1080 MB
- Scenario 2 (docker): guillermo/mysql -- 381.5 MB
Hosted @
9. STEADY STATE VM PACKING
OpenStack Cloudy Benchmark
6/11/2014 9
10. Cloudy Performance: Steady State Packing
Benchmark scenario overview
– Pre-cache VM image on compute node prior to test
– Boot 15 VM asynchronously in succession
– Wait for 5 minutes (to achieve steady-state on the
compute node)
– Delete all 15 VMs asynchronously in succession
Benchmark driver
– cpu_bench.py
High level goals
– Understand compute node characteristics under
steady-state conditions with 15 packed / active VMs
6/11/2014 10
0
2
4
6
8
10
12
14
16
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47
ActiveVMs
Time
Benchmark Visualization
VMs
17. Cloudy Performance: Serial VM Boot
Benchmark scenario overview
– Pre-cache VM image on compute node prior to test
– Boot VM
– Wait for VM to become ACTIVE
– Repeat the above steps for a total of 15 VMs
– Delete all VMs
Benchmark driver
– OpenStack Rally
High level goals
– Understand compute node characteristics under
sustained VM boots
6/11/2014 17
0
2
4
6
8
10
12
14
16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
ActiveVMs
Time
Benchmark Visualization
VMs
18. Cloudy Performance: Serial VM Boot
6/11/2014 18
3.529113102
5.781662448
0
1
2
3
4
5
6
7
docker KVM
TimeInSeconds
Average Server Boot Time
docker
KVM
25. SERIAL VM SOFT REBOOT
OpenStack Cloudy Benchmark
6/11/2014 25
26. Cloudy Performance: Serial VM Reboot
Benchmark scenario overview
– Pre-cache VM image on compute node prior to test
– Boot a VM & wait for it to become ACTIVE
– Soft reboot the VM and wait for it to become ACTIVE
• Repeat reboot a total of 5 times
– Delete VM
– Repeat the above for a total of 5 VMs
Benchmark driver
– OpenStack Rally
High level goals
– Understand compute node characteristics under sustained VM reboots
6/11/2014 26
0
1
2
3
4
5
6
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55
ActiveVMs
Time
Benchmark Visualization
Active VMs
27. Cloudy Performance: Serial VM Reboot
6/11/2014 27
2.577879581
124.433239
0
20
40
60
80
100
120
140
docker KVM
TimeInSeconds
Average Server Reboot Time
docker
KVM
28. Cloudy Performance: Serial VM Reboot
6/11/2014 28
3.567586041
3.479760051
0
0.5
1
1.5
2
2.5
3
3.5
4
docker KVM
TimeInSeconds
Average Server Delete Time
docker
KVM
32. SNAPSHOT VM TO IMAGE
OpenStack Cloudy Benchmark
6/11/2014 32
33. Cloudy Performance: Snapshot VM To Image
Benchmark scenario overview
– Pre-cache VM image on compute node prior to test
– Boot a VM
– Wait for it to become ACTIVE
– Snapshot the VM
– Wait for image to become ACTIVE
– Delete VM
Benchmark driver
– OpenStack Rally
High level goals
– Understand cloudy ops times from a user perspective
6/11/2014 33
34. Cloudy Performance: Snapshot VM To Image
6/11/2014 34
36.88756394
48.02313805
0
10
20
30
40
50
60
docker KVM
TimeInSeconds
Average Snapshot Server Time
docker
KVM
45. Cloud Management Impacts on docker LXC
0.17
3.529113102
0
0.5
1
1.5
2
2.5
3
3.5
4
docker cli nova-docker
Seconds
Docker: Boot Container - CLI vs Nova Virt
docker cli
nova-docker
6/11/2014 45
Cloud management often caps true ops performance of LXC
46. Ubuntu MySQL Image Size
381.5
1080
0
200
400
600
800
1000
1200
docker kvm
SizeInMB
Docker / KVM: Ubuntu MySQL
docker
kvm
6/11/2014 46
Out of the box JeOS images for docker are lightweight
47. In Summary
Near bare metal performance in the guest
Fast operations in the Cloud
– Often capped by Cloud management framework
Reduced resource consumption (CPU, MEM) on the compute
node – greater density
Out of the box smaller image footprint
6/11/2014 47
Let me start off by saying– it’s very exciting to be here at the 1st dockercon and I hope this is the start of many more
Welcome, before getting started a little about me and containers and in particular about docker
Boden Russell , IBM GTS – advanced cloud solutions & innovation team
SL engagenments including customre PoCs, managed and as a service realizations
One of my favoritate parts of this job – nextgen technology evals and recommend to broader IBM community
In about nov of last year we started evaluating LXC
Tried vairous lxc user toolsets.. .kept coming back to docker
Since then we’ve done some other research with LXC including SAP HANA for education purposes as well as other things
Looking across the industry appeared to be a gap in docs talking about LXC from a Cloud perspective vs hypers
I set out to do some semi-active testing using OpenStack with KVM and docker --- the results we’ll talk about today
Before getting started on the technicals, lets take a mintue to step back and consider why these results are import
What motivates me from a technology / industry perspective…
Consider myself a technologist / scientist, as a result of this I strive to work on projects which have a certain degree of awesomness… obviously docker has a massive degress of awesomeness
I strive for projects and technologies which allow me to use creativity and innovation
Revenue is important to me in that I must support my family
However, I’m willing to consider less revenue as long as I can support my familiy to work on something which is innovate / creative / exciting
Why am I telling you this…. I believe there are a number of people in the community and even in this room who follow these values.. They prioritize working on things they are passionate about above revenue (to a degree).
So what motivates larger companies say those who are making key tech decisions in the enterprise space??...
What do you think movitvates technical decisions in industry?
If you ask and transparent exec making key tech decisions in enterprise, they will tell you: revenue, revenue, revenue
You might argue otherwise – our goal is provide the best use experience possible, or all the features our customers wants, etc..
I would argue All of these are directly related to revenue.
So, how can we increase revenue in this space??
In a nutshell – do more with less
More specifically you will see the benefits of virt and cloud discussed within the context of reducing TCO and increasing the ROI
There are various aspects which impact TCO and ROI and this chart briefly outlines some of the more common categories.
Let’s just cover a few of these which I believe