Cloud Computing Workshop 2013, ITU

1. 14: Operating Systems and Virtualization
Zubair Nabi
zubair.nabi@itu.edu.pk
April 21, 2013
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 1 / 24

2. Outline
1 Operating Systems
2 Virtualization
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 2 / 24

3. Outline
1 Operating Systems
2 Virtualization
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 3 / 24

4. Introduction
Linux is the default choice
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 4 / 24

5. Introduction
Linux is the default choice
Due to its performance, modularity, power efficiency, scalability, open
source nature, and ubiquity
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 4 / 24

6. Introduction
Linux is the default choice
Due to its performance, modularity, power efficiency, scalability, open
source nature, and ubiquity
Deployed by everyone, from Google to Dell and from Amazon to IBM
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 4 / 24

7. Introduction
Linux is the default choice
Due to its performance, modularity, power efficiency, scalability, open
source nature, and ubiquity
Deployed by everyone, from Google to Dell and from Amazon to IBM
But which distribution?
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 4 / 24

8. Introduction
Linux is the default choice
Due to its performance, modularity, power efficiency, scalability, open
source nature, and ubiquity
Deployed by everyone, from Google to Dell and from Amazon to IBM
But which distribution?
All distros have different features
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 4 / 24

9. Distributions
Distributions overlap in most of their functionality
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 5 / 24

10. Distributions
Distributions overlap in most of their functionality
Many share a common package manager
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 5 / 24

11. Distributions
Distributions overlap in most of their functionality
Many share a common package manager
CentOS, Red Hat, and Fedora use RPM packages
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 5 / 24

12. Distributions
Distributions overlap in most of their functionality
Many share a common package manager
CentOS, Red Hat, and Fedora use RPM packages
Ubuntu and Debian use APT or .deb packages
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 5 / 24

13. Distributions
Distributions overlap in most of their functionality
Many share a common package manager
CentOS, Red Hat, and Fedora use RPM packages
Ubuntu and Debian use APT or .deb packages
Almost all distributions will run any type of software
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 5 / 24

14. Ubuntu
Designed for ease of use across desktops as well as servers
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 6 / 24

15. Ubuntu
Designed for ease of use across desktops as well as servers
Latest software versions are rolled out regularly
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 6 / 24

16. Ubuntu
Designed for ease of use across desktops as well as servers
Latest software versions are rolled out regularly
Two different versions available:
1 LTS (Long Term Support) version with stable packages
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 6 / 24

17. Ubuntu
Designed for ease of use across desktops as well as servers
Latest software versions are rolled out regularly
Two different versions available:
1 LTS (Long Term Support) version with stable packages
2 Normal version with cutting-edge packages
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 6 / 24

18. Ubuntu
Designed for ease of use across desktops as well as servers
Latest software versions are rolled out regularly
Two different versions available:
1 LTS (Long Term Support) version with stable packages
2 Normal version with cutting-edge packages
Due to their stability, LTS versions are deployed on production servers
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 6 / 24

19. CentOS
Emphasizes reliability
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 7 / 24

20. CentOS
Emphasizes reliability
Well-suited to production environments
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 7 / 24

21. CentOS
Emphasizes reliability
Well-suited to production environments
Packages have been tuned over time to work out as many bugs and
security flaws as possible
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 7 / 24

22. CentOS
Emphasizes reliability
Well-suited to production environments
Packages have been tuned over time to work out as many bugs and
security flaws as possible
On the downside, software versions are rarely the latest
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 7 / 24

23. Red Hat Enterprise Linux (RHEL)
Targets enterprise-level servers
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 8 / 24

24. Red Hat Enterprise Linux (RHEL)
Targets enterprise-level servers
Stable and handles heavy loads well
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 8 / 24

25. Red Hat Enterprise Linux (RHEL)
Targets enterprise-level servers
Stable and handles heavy loads well
Requires a small licensing fee for non-free software components and
updates
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 8 / 24

26. Red Hat Enterprise Linux (RHEL)
Targets enterprise-level servers
Stable and handles heavy loads well
Requires a small licensing fee for non-free software components and
updates
Useful if the target software/workload supports it; enterprise software
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 8 / 24

27. Red Hat Enterprise Linux (RHEL)
Targets enterprise-level servers
Stable and handles heavy loads well
Requires a small licensing fee for non-free software components and
updates
Useful if the target software/workload supports it; enterprise software
If not, better off using CentOS
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 8 / 24

28. Gentoo
By default forces the compilation of software rather than grabbing
precompiled packages
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 9 / 24

29. Gentoo
By default forces the compilation of software rather than grabbing
precompiled packages
Achieves great system optimizations if compiler options for the target
environment known
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 9 / 24

30. Gentoo
By default forces the compilation of software rather than grabbing
precompiled packages
Achieves great system optimizations if compiler options for the target
environment known
Useful if fine-grained control over each aspect of the system is required
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 9 / 24

31. Arch
A distribution for experienced sysadmins
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 10 / 24

32. Arch
A distribution for experienced sysadmins
Useful if good low-level control over program execution required
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 10 / 24

33. Outline
1 Operating Systems
2 Virtualization
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 11 / 24

34. Virtual Machine Monitor or Hypervisor
Sits between one or more operating systems and the hardware
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 12 / 24

35. Virtual Machine Monitor or Hypervisor
Sits between one or more operating systems and the hardware
Multiplexes the hardware across OSes
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 12 / 24

36. Virtual Machine Monitor or Hypervisor
Sits between one or more operating systems and the hardware
Multiplexes the hardware across OSes
Major advantage is server consolidation
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 12 / 24

37. Xen
Hypervisor which enables many instances of different operating
systems to run atop the same physical host
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 13 / 24

38. Xen
Hypervisor which enables many instances of different operating
systems to run atop the same physical host
Used as the basis for a large number of applications: server and
desktop virtualization, IaaS, security, etc.
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 13 / 24

39. Xen
Hypervisor which enables many instances of different operating
systems to run atop the same physical host
Used as the basis for a large number of applications: server and
desktop virtualization, IaaS, security, etc.
Powering some of the largest clouds in production
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 13 / 24

40. Features
Small footprint (1MB in size) due to its microkernel design
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 14 / 24

41. Features
Small footprint (1MB in size) due to its microkernel design
Main device drivers for a system can be run within a VM to ensure
isolation
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 14 / 24

42. Features
Small footprint (1MB in size) due to its microkernel design
Main device drivers for a system can be run within a VM to ensure
isolation
Paravirtualized guests optimized to run as VMs enabling greater
performance
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 14 / 24

43. Architecture
The hypervisor runs directly atop the hardware
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 15 / 24

44. Architecture
The hypervisor runs directly atop the hardware
In charge of handling CPU, memory, and I/O
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 15 / 24

45. Architecture
The hypervisor runs directly atop the hardware
In charge of handling CPU, memory, and I/O
Running instance of a VM is called domU or guest
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 15 / 24

46. Architecture
The hypervisor runs directly atop the hardware
In charge of handling CPU, memory, and I/O
Running instance of a VM is called domU or guest
A special domain, called dom0, contains the drivers for all devices in
the system
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 15 / 24

47. Architecture
The hypervisor runs directly atop the hardware
In charge of handling CPU, memory, and I/O
Running instance of a VM is called domU or guest
A special domain, called dom0, contains the drivers for all devices in
the system
Also contains the interface for VM creation, termination, and
configuration
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 15 / 24

48. Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 16 / 24

49. Xen-enabled operating systems
dom0 requires a Xen-enabled kernel
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 17 / 24

50. Xen-enabled operating systems
dom0 requires a Xen-enabled kernel
Similarly, paravirtualized guests also require a PV-enabled kernel
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 17 / 24

51. Xen-enabled operating systems
dom0 requires a Xen-enabled kernel
Similarly, paravirtualized guests also require a PV-enabled kernel
Most Linux distributions are Xen-enabled and by default contain the
Xen tool chain
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 17 / 24

52. I/O
dom0 is in charge of all hardware
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 18 / 24

53. I/O
dom0 is in charge of all hardware
Based on the configuration of a domU, dom0 exports a subset of the
devices into that domU
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 18 / 24

54. I/O
dom0 is in charge of all hardware
Based on the configuration of a domU, dom0 exports a subset of the
devices into that domU
Devices are exported as “class devices”
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 18 / 24

55. I/O
dom0 is in charge of all hardware
Based on the configuration of a domU, dom0 exports a subset of the
devices into that domU
Devices are exported as “class devices”
A block device or a network device, not as a specific hardware device
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 18 / 24

56. I/O
dom0 is in charge of all hardware
Based on the configuration of a domU, dom0 exports a subset of the
devices into that domU
Devices are exported as “class devices”
A block device or a network device, not as a specific hardware device
dom0 runs the backend of the device which is connected to the
frontend in the domU
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 18 / 24

57. I/O
dom0 is in charge of all hardware
Based on the configuration of a domU, dom0 exports a subset of the
devices into that domU
Devices are exported as “class devices”
A block device or a network device, not as a specific hardware device
dom0 runs the backend of the device which is connected to the
frontend in the domU
netback and netfront in case of network devices and blockback and
blockfront in case of block devices
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 18 / 24

58. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

59. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Instructions are run native to the core CPU
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

60. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Instructions are run native to the core CPU
In contrast to virtualization, no overhead of instruction-level emulation
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

61. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Instructions are run native to the core CPU
In contrast to virtualization, no overhead of instruction-level emulation
Applications presented the illusion of running on a separate machine
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

62. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Instructions are run native to the core CPU
In contrast to virtualization, no overhead of instruction-level emulation
Applications presented the illusion of running on a separate machine
A number of underlying resources are shared
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

63. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Instructions are run native to the core CPU
In contrast to virtualization, no overhead of instruction-level emulation
Applications presented the illusion of running on a separate machine
A number of underlying resources are shared
For instance, all containers share the same underlying kernel
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

64. Linux Containers (LXC)
Partition the resources managed by the OS into individual silos
Instructions are run native to the core CPU
In contrast to virtualization, no overhead of instruction-level emulation
Applications presented the illusion of running on a separate machine
A number of underlying resources are shared
For instance, all containers share the same underlying kernel
Part of the main kernel tree
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 19 / 24

65. Bridging and Filesystem
Linux bridge code is used to link the container to the network
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 20 / 24

66. Bridging and Filesystem
Linux bridge code is used to link the container to the network
Each virtual interface has two pairs: one inside the container, the other
end connected to the bridge
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 20 / 24

67. Bridging and Filesystem
Linux bridge code is used to link the container to the network
Each virtual interface has two pairs: one inside the container, the other
end connected to the bridge
Also possible to create a filesystem for each container
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 20 / 24

68. Kernel-based Virtual Machine (KVM)
Merges the hypervisor with the kernel
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 21 / 24

69. Kernel-based Virtual Machine (KVM)
Merges the hypervisor with the kernel
Reduces redundancy and speeds up execution times
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 21 / 24

70. Kernel-based Virtual Machine (KVM)
Merges the hypervisor with the kernel
Reduces redundancy and speeds up execution times
KVM driver acts as the middleware between the kernel and a
userspace VM
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 21 / 24

71. Kernel-based Virtual Machine (KVM)
Merges the hypervisor with the kernel
Reduces redundancy and speeds up execution times
KVM driver acts as the middleware between the kernel and a
userspace VM
Scheduling of processes and memory management is handled by the
kernel
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 21 / 24

72. QEMU
Manages I/O and enables a virtual home for the guest OS
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 22 / 24

73. QEMU
Manages I/O and enables a virtual home for the guest OS
QEMU itself runs as a user-space process
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 22 / 24

74. QEMU
Manages I/O and enables a virtual home for the guest OS
QEMU itself runs as a user-space process
Virtual processors within a virtual machine simply execute as threads
in the host process
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 22 / 24

75. Advantages
Reuse of existing software and infrastructure
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 23 / 24

76. Advantages
Reuse of existing software and infrastructure
Seamlessly works across all platforms: from servers to embedded
devices
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 23 / 24

77. Advantages
Reuse of existing software and infrastructure
Seamlessly works across all platforms: from servers to embedded
devices
Integrates with the existing Linux scheduler, I/O stack, and available
filesystems
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 23 / 24

78. References
1 Linux: The Operating System of the Cloud:
http://www.linuxfoundation.org/sites/main/
files/publications/linuxincloud.html
2 Choosing a Linux Distribution:
http://www.rackspace.com/knowledge_center/
article/choosing-a-linux-distribution
3 Xen Overview:
http://wiki.xenproject.org/wiki/Xen_Overview
4 LXC: Linux container tools: http://www.ibm.com/
developerworks/linux/library/l-lxc-containers/
5 Kernel-based virtualization with KVM:
http://www.linux-magazine.com/content/download/
62512/485179/file/Kernel_Based_Virtualization_
With_KVM.pdf
Zubair Nabi 14: Operating Systems and Virtualization April 21, 2013 24 / 24