This is the talk i gave at Future of Paas Workshop which is part of IC2E IEEE conference 2014 at Boston University on March 11 2014

1. IEEEFutureofPaas2014
Virtualization vs
Containerization to support PaaS
RajdeepDua
VMWare Email:rajdeepd@vmware.com
AReddyRaja
IIITHyderabad,India
DharmeshKakadia
IIITEmail:dharmesh.kakadia@research.iiit.ac.in

2. IEEEFutureofPaas2014
About Myself
• Working on Distributed Systems / Server side platforms
for more than 15 years
• Some companies I worked for
VMware
Google
Microsoft
Amdocs
• vSphere, OpenStack, Cloud Foundry, Google App
Engine…
• Twitter : @rajdeepdua

3. IEEEFutureofPaas2014
Agenda
• Introduction
• Definition : VMs and Containers
• Paas Requirements
• Understanding Containment Principles
• Existing Container Implementation and Comparison
• Choosing the right container for a Paas
• Conclusion

4. IEEEFutureofPaas2014
Introduction
• Platform as a Service has brought developer
productivity to the forefront
• Infrastructure as a Service is masked from a
developer
• Opportunity for more efficient use of resources
• In this paper, we explore various container
technologies and how they are used by Paas
Implementation

5. IEEEFutureofPaas2014
Comparing VMs and Containers

6. IEEEFutureofPaas2014
Virtual Machine and Containers
• Virtual Machine
• Allow Multiple Guest OS to run together
on a single machine.
• Each Guest OS abstracts Compute, Storage
and Network Components.
• Hypervisor itself could run on bare-metal
(ESXi) or be part of an OS (KVM).
• Guest ISA is translated to Host ISA using
multiple techniques like Hardware
Virtualization or Binary Translation.

7. IEEEFutureofPaas2014
Virtual Machine and Containers
Container
• light weight containment system
running
• Runs instructions native to the core
CPU :
• Container shares the Kernel and
Process scheduler with the Host OS
• No Binary Translation or VM exits
• Containers can be run either directly
on the Host OS or in a Guest VM :

8. IEEEFutureofPaas2014
Virtual Machine and Containers
Parameter Virtual Machines Containers
Guest OS
Each VM runs on a hypervisor and
Kernel is loaded into in its own
memory region
All the guests share same OS and
Kernel. Kernel image is loaded into
the physical memory
Networking
Can be linked to virtual or Physical
switches. Hypervisors have buffers
for IO performance improvement,
NIC bonding etc
Leverages standard IPC mechanisms
like Signals, pipes, sockets etc.
Advanced features like NIC bonding
etc still not available.
Security Complete Isolation Isolation using techniques like
namespaces

9. IEEEFutureofPaas2014
Virtual Machine and Containers
Parameter Virtual Machines Containers
Performance
Suffer from a small overhead as the
Machine instructions are translated
from Guest to Host OS.
Provide near native performance
as compared to the underlying
Host OS.
Isolation
Higher level of Isolation –Need special
techniques for file sharing
Subdirectories can be
transparently mounted and can
be shared
Startup time VMs take a few minutes to boot up
Containers can be booted up in a
few seconds as compared to VMs
Storage
VMs take much more storage as the
whole OS kernel and its associated
programs have to installed and run
Containers take lower amount of
storage as the base OS is shared

10. IEEEFutureofPaas2014
Paas Requirements

11. IEEEFutureofPaas2014
Paas Requirements
• Paas focusses on developer productivity and abstracts outs
the underlying infrastructure
• 3 Key requirements of Paas from Underlying Infrastructure are
• [1] Provision and manage lifecycle of Networking, compute and
storage programatically
• [2] Provide Highly reliable infrastructure which can be efficicently
used
• [3] Ability to add applications/services and bind them to external
routers/ dns servers

12. IEEEFutureofPaas2014
Paas Requirements : VM
• VMs are very suitable for requirements [1] and [2]
• Applications could be hosted either directly in VMs or on Containers
inside VMs.
• Containers are more efficient for [3] as they are light weight and allow
better leverage of the hardware and resources

13. IEEEFutureofPaas2014
Container Overview
• Containers leverage the underlying features in
the Linux Kernel to implement Containment
• chroot : Unix command which changes the root directory for the
existing process and its child
• Control Groups (cgroup) :
• cgroup is a method to put processes into groups.
• Implemented as pseudo filesystem.
• Grouping can be done by a unit of thread.
• Many functions are implemented as “subsystem”
• e.g CPU Sets , Fake NUMA, Freezer, device

14. IEEEFutureofPaas2014
Container Overview : Kernel
Namespaces
• The purpose of each namespace is
to wrap a particular global system
resource in an abstraction
• Makes it appear to the processes
within the namespace that they
have their own isolated instance of
the global resource
• Namespaces and cgroups are used in
conjunction
Pic reference : http://www.linuxfoundation.jp/jp_uploads/seminar20081119/CgroupMemcgMaster.pdf

15. IEEEFutureofPaas2014
Namespaces Implemented in Linux
• Mount namespace : Isolate the set of file system mount
points seen by a group of processes
• PID namespace : Isolate the process ID number space. In
other words, processes in different PID namespaces can
have the same PID
• UTS namespace : Isolate two system identifiers—node-
name and domain-name
• Network namespace : Each network namespace has its
own network devices, IP addresses, IP routing tables,
/proc/net directory, port numbers,

16. IEEEFutureofPaas2014
Containers Current State

17. IEEEFutureofPaas2014
Linux Containers
• Linux Containers (LXC) are light weight kernel
containment implementation
• Use cgroups for resource allocation and namespaces for
isolation
• Key Characteristics of the Linux Containers are,
• Resource Allocation Uses cgroups for CPU, Memory and Device
allocation
• Process and Network Isolation Using namespaces : PID and net
• File System Isolation Each container gets a private file system by
leveraging chroot

18. IEEEFutureofPaas2014
Linux Containers

19. IEEEFutureofPaas2014
Warden Container
• Warden container provides a kernel independent
containment implementation on top of LXC
• Used by Cloud Foundry project to host
applications.
• Key attributes of Warden container :
• Uses namespaces for isolation of Process Ids and
Network namespaces
• Uses cgroups concept from Linux to allocate resources
• Each container is managed by a Warden Daemon

20. IEEEFutureofPaas2014
Docker
• Docker is a daemon which provides ability to manage Linux containers as
self contained images.
• Utilizes LXC (Linux Containers) for the container implementation and adds
image management and Union File System capability to it.
• Key attribute of Docker Containers are
• Resource Isolation - Same as LXC
• Network and Process Isolation - Leverages LXC functionality
• File System Isolation Leverages LXC functionality
• Container Lifecycle - Managed using a daemon and command line tool
• Container State - Docker allows ability to store and retrieve container state

21. IEEEFutureofPaas2014
Docker – Improving Security
• High level Goals of Docker project to improve
security (which are limitations of Linux
Containers)
• Map root user of the container to non-root user of
docker
• Make docker daemon run as a non root user

22. IEEEFutureofPaas2014
Google lmctfy
• Google lmctfy provides a resource configuration API which simplifies
container management as compared to LXC
• Provides intent based configuration without the need to understand
cgroups and removes the difficulties of unstable LXC APIs.
• Improves the resource sharing and can support performance guarantees.
• Key attribute of lmctfy Containers are
Uses only cgroups and customer kernel patches for resource isolation
• Resource Isolation - Resources isolation using cgroups.
• Network Isolation - Plan to use net namespace in next version CL2
• File System Isolation - Plan to use mnt namespace in next version CL2
• Container Lifecycle - Plan to support freeze, chck- point/restore and disk
import/export.
• Monitoring - To be supported through higher level c APIs

23. IEEEFutureofPaas2014
Open VZ
• OpenVZ uses a modified Linux Kernel with a set of extensions.
• very similar conceptually, its functionality is not part of the core
Linux Kernel
• It additionally provides templates that help in pre- created Virtual
environments.
• Resource Management - Manages resource sharing using Bean Counters, Fair
Share CPU Scheduler ..
• Network Isolation - Uses net namespace File System Isolation - Provides isolation
to Application Files, System Libraries etc
• Container Lifecycle – managed using vzctl tool
• Container State - Provides Check pointing feature for storing and recovering the
last known state.

24. IEEEFutureofPaas2014
Container Comparison

25. IEEEFutureofPaas2014
Container Comparison
Parameter LXC Warden Docker OpenVZ
Process Iso-
lation
Uses pid names-
pace
Uses pid names-
pace
Uses pid names-
pace
Uses pid names-
pace
Resource
Isolation
Uses cgroups Uses cgroups Uses cgroups Uses cgroups
Network
Isolation
Uses net names-
pace
Uses net names-
pace
Uses net names-
pace
Uses net names-
pace
Filesystem
Isolation
Using chroot
Overlay File
system using
overlayfs
Using chroot Using chroot
Container
Lifecycle
Tools lxc-create,
lxc-stop, lxc-start
to create, start,
stop a container
Containers are
managed by
running com-
mands on a
warden client
which talks to
warden server
Uses Docker
daemon and a
client to manage
the containers
uses vzctl to
manage
container
lifecycle

26. IEEEFutureofPaas2014
Container usage in a Paas Example

27. IEEEFutureofPaas2014
Container adoption in a Open
Source Paas : Cloud Foundry
• Cloud Foundry uses
Warden Container to
isolate each
application.
• Multiple containers
are run on a single
VM for efficient
Resource usage
• Currently tested on
on Ubuntu. Supports
AWS, vSphere and
OpenStack Clouds

28. IEEEFutureofPaas2014
Choosing the Right Container for
your Paas
• Choosing the right container for most of the PaaS
implementations will be based on the following
factors:
• Ecosystem for prebuilt containers
• Hardened layer for isolation of Process, Network, CPU
and File system
• Tools to manage lifecycle of a container
• Ability to migrate containers between hosts
• Support for multiple OS and kernels

29. IEEEFutureofPaas2014
Next Generation Paas Support
• Some of the features listed below would help
improve adoption of containers in PaaS platforms
• Standardization :
• Need for a standard container file format – similar to OVF
• Security :
• Containers need to be more secure both from the perspective of file
system, network and memory isolation.
• Most popular container implementations like LXC and Docker seem to
be lacking in this regard.
• OS Independence
• Containers should have abstraction layer so that they are not tied to a
particular Kernel or its user space.
• Warden Container seems to be in the right direction in this regard, but
we doubt it will get wide adoption beyond Cloud Foundry.

30. IEEEFutureofPaas2014
Conclusion
• Containers have an inherent advantage over VMs for PAAS use case
• Multiple flavors of container implementations available, all of them
open source.
• Common Linux containment principles like chroot, cgroups or
namespaces are being used by all of them.
• Containers have a bright future specially in the PaaS use case provided
there is more standardization and abstraction from the underlying
kernel and Host OS.