2. Outline
⢠Classifications
⢠Processor virtualization
Two main Software-based solutions
⢠Challenges to virtualize Intel x86(software-only)
⢠Hardware-based Virtualization
â˘
Intel VT-x : x86
⢠Intel VT-I :Itanium (X)
⢠Intel EPT/AMD NPT
⢠AMD-V
⢠Sun SPARC (X)
⢠ARM Virtualization Extensions(X)
⢠IBM Power(X)
â˘
⢠I/O virtualization
⢠Intel VT-d
⢠AMD IOMMU(AMD-V)
⢠Intel VT-c
⢠Dose these techniques work?
2
3. Classifications
⢠VMM(virtual machine monitor) = hypervisor
⢠By techniques
Full Virtualization
⢠Paravirtualization
⢠Hardware Assisted Virtualization
â˘
⢠Robert P. Goldberg(Harvard University,1973)
â˘
Type 1/native/bare metal hypervisors
Hypervisors run directly on the host's hardware to control the hardware and to manage
guest operating systems.
⢠Oracle VM Server for SPARC, the Citrix XenServer, KVM, VMware ESX/ESXi, and Microsoft
Hyper-V hypervisor.
â˘
â˘
Type 2/hosted hypervisors
Hypervisors run within a conventional operating system environment.
⢠VMware Workstation/player, Microsoft Virtual PC and VirtualBox
â˘
3
6. Two main Software-based
solutions(1)
⢠Full virtualization using binary translation
â˘
Transforming guest OS binaries on-the-fly
â˘
â˘
Guest applications donât use privilege instructions
Pros
Support unmodified OSs (The only way of pure-software solutions)
⢠Offer best isolation and security
⢠Simplify migration and portability of guest OS
â˘
Cons: low performance
⢠Examples: VMware, MS Virtual PC, Virtual box
â˘
â˘
Disable HW virtualization
6
8. â˘
Two main Software-based
OS assisted virtualization or paravirtualization
solutions(2)ď guest OSs help the VMM
OS assisted virtualization
â˘
â˘
â˘
paravirtualization refers to communication between the guest OS and the VMM to
improve performance and efficiency
Modify the guest OS to cooperate with the VMM
â˘
Modify the OS kernel to replace non-virtualizable instructions with hypercalls(the
functions provided by the VMM)
Pros: Offer higher performance
⢠Cons: Need the source code of an OS
⢠Example: Xen, KVM(*), VMware(*)
⢠(*) Vmware tool uses some paravirtualization techniques
â˘
optimize virtual device drivers
⢠time synchronization
⢠logging and guest shutdown.
⢠Vmxnet is a paravirtualized I/O device driver
â˘
8
11. Challenges to virtualize Intel
x86(software-only)(1/3)
⢠Ring Aliasing
â˘
Guest-OSes run at the Ring3
â˘
â˘
Original: OS:Ring 0, APP:Ring 3 (Ring0>ring3)
A guest OS can know its run level
⢠Address-Space Compression
VMM must use some of the guestâs virtual-address space to manage
transition between guest OS and VMM
⢠VMMâs address spaces must be protected
â˘
â˘
Guest could detect that it is running in a VM
11
12. Challenges to virtualize Intel
x86(software-only)(2/3)
⢠Non-Faulting Access to Privileged State
â˘
Some instructions should be intercepted by VMM do not involve faults
⢠Adverse Impact on Guest System Calls
â˘
VMM must emulate every system calls
⢠Interrupt Virtualization
A VMM may manage external interrupts and deny guest to control
interrupt masking
⢠Some OS frequently mask and unmask
â˘
â˘
VMM must process these requests.
12
13. Challenges to virtualize Intel
x86(software-only) (3/3)
⢠Ring Compression
â˘
Guest OS runs at the same privilege level as applications
â˘
The guest OS canât protect guest applications
⢠Frequent Access to Privileged Resources
â˘
VMM should deny the accesses
⢠Address translation
â˘
Guest OS doesnât know the physical address, so the VMM must
intercepted guest page table updates
13
14. Intel VT-x overview(1/4)
⢠VT=virtualization technology
â˘
Two new form of CPU operation
VMX root operation : for VMM
⢠VMX non-root operation: for guest-software
⢠Both forms of operation support all four privilege levels(Ring0~Ring3)
â˘
â˘
Guest OS can run at its intended privilege level
14
16. Intel VT-x overview(2/4)
⢠Two new transitions
⢠VM entry
⢠VMX root operation (VMM)ď non-root operation(VM)
⢠VM exit
⢠VMX non-root operation (VM)ď root operation (VMM)
⢠Under VMX non-root operation, many
instructions/events cause VM exits
⢠configurable
16
17. Intel VT-x overview(3/4)
⢠VMCS (Virtual Machine Control Structure)
A new data structure includes guest-state area and host-state area
⢠VM entry: load the guest-state area and save the host-state area
⢠VM exit : load the host-state area and save the guest-state area
⢠The exiting conditions controlled by the VM-execution fields
⢠Switch the structure will switch the address space
â˘
17
18. Intel VT-x overview(4/4)
⢠VMCS supports interrupt virtualization
â˘
Determine the conditions of VM to cause VM exit
â˘
â˘
â˘
â˘
â˘
All interrupt
Whenever guest OS is ready to receive interrupts
Which exception?
Which port access attempts?
Which Model Specific Register access attempts?
18
19. Intel EPT / AMD NPT(1)
⢠EPT (Extended Page Tables)
â˘
â˘
âEPT provides performance gains of up to 48% for MMU-intensive
benchmarks and up to 600% for MMU-intensive microbenchmarks.â â
VMware
AMDâs nested page table (NPT) is similar to EPT
⢠A.k.a Rapid Virtualization Indexing (RVI)
â˘
âRVI provides performance gains of up to 42% for MMU-intensive
benchmarks and up to 500% for MMU-intensive microbenchmarks.â -VMware
19
20. Intel EPT / AMD NPT(2)
⢠Software MMU (software-only)
Hardware uses the shadow page table
⢠VMM must maintain the shadow page table
â˘
20
21. Intelmaintains guest page tables
EPT / AMD NPT(3)
Guest-OS
⢠Hardware MMU
â˘
VMM maintains PPN->MPN mappings in an additional level of page tables
⢠The hardware will find the LPN->MPN with the two pages
â˘
21
22. AMD-V(1/2)
⢠Tagged TLB
Add the ASID
⢠Hardware features that facilitate efficient switching between virtual
machines for better application responsiveness
â˘
â˘
Two new form of CPU operation
Host mode : for VMM (similar to Intelâs VMX root operation)
⢠Guest mode : for guest software (similar to Intelâs VMX non-root operation)
⢠new instructions
â˘
â˘
â˘
â˘
â˘
vmrun : host mode ď guest mode
exit : guest mode ď host mode
vmcall: it lets the operating system and VMM communicate directly
A new structure
â˘
Virtual Machine Control Block (VMCB)
â˘
Similar to Intelâs VMCS
22
23. AMD-V(2/2)
⢠Nested page table (NPT)/ Rapid Virtualization Indexing (RVI)
⢠VMM migration
â˘
Use the CPUID to identify the ability of the processor where the VMM
runs, and the VMM use the supported functions.
23
24. Hardware-base solution with VTx(1/2)
⢠Address-Space Compression
â˘
VM Exits / VM Entries change the linear address space
⢠Ring Aliasing
â˘
& Ring Compression
VT-x allows guest OS to run at its intended privilege level
⢠Nonfaulting Access to Privileged State
Either causes transition to VMM
⢠Or becomes unimportant to VMM
â˘
24
25. Hardware-base solution with VTx(2/2)
⢠Guest System Calls
â˘
a guest OS can run at privilege level 0
⢠Frequent Access to Privileged Resources
â˘
VT-x provides TPR shadow. VMM is only involved when the value drops
below the thresholdď VMM only processes the situation it cares.
25
28. Current I/O virtualization
techniques
⢠Emulation
The VMM supports virtual devices that guest OS can recognize
⢠The virtual device models are responsible to translate commands and data.
⢠Pros. No requirement to modify guest-OSs
⢠Cons. Low performance
â˘
⢠Paravirtualization
Modify the guest software (driver)to enhance the performance
⢠Pros. better performance
⢠Cons. Limited applicability. (modifyď need the source code)
â˘
⢠Direct assignment
Bind a specify device to a VM
⢠VMM allow the owning VM to connect directly
⢠Issue command (go)ď low overhead
⢠DMA? (back)
â˘
28
29. DMA on a virtualizing system
⢠DMA
Driver issue a packet consists of command, physical address, etc.
⢠DMA controller read/write data from/to the physical address
⢠Challenge?
â˘
A physical address that a Guest-OS knows is not really physical !
⢠The really physical address space is managed by the VMM
⢠The DMA controller will incorrectly write data to an address.
â˘
29
30. Intel VT-d(1/2)
⢠Need the support of the North bridge
⢠Two functions
â˘
Bind devices to a specify VM
â˘
â˘
DMA remapping
Interrupt virtualization
â˘
Interrupt remapping
⢠DMA remapping
DVA (DMA Virtual Address), GPA(Guest Physical Address), HPA(Host
Physical Address)
⢠A guest-OS issue a DMA request with DVA(=GPA)
⢠The VT-d hardware will translate the DVA to HPA
â˘
â˘
The concept: lookup tables
30
32. Intel VT-d (2/2)
â˘
Interrupt Remapping
⢠Assign an interrupt attribute
â˘
â˘
Destination processor, vector, etc.
A VMM enables the interrupt requests from the I/O device to target the
physical CPUs running the appropriate virtual CPUs of the legacy VM
⢠AMD IOMMU is similar to Intel VT-d
32
33. Intel VT-c
⢠Virtualization Technology for Connectivity
â˘
Virtualization on devices
⢠A collection of technologies that improve the performance of
network I/O on a virtualized system
⢠VT-c is comprised of two components
â˘
VMDq (Virtual Machine Device Queues)
A hardware-base enhancement
⢠Target: throughput
â˘
â˘
VMDc (Virtual Machine Direct Connect)
Virtualizing physical I/O ports of a network controller into multiple virtual I/O
ports, and then to map the virtual ports to individual VMs
⢠Target :VT-x + VT-d + VT-c ď nearly native performance
â˘
33
49. Summarization of Hardware
Assisted Virtualization
⢠Hardware provides some mechanisms to reduce overheads of
virtualization to improve performance
⢠Pros.
The highest performance in theory (a counter example, 2006 VMware)
⢠Support unmodified Oss
⢠Simplify the development of VMM
â˘
⢠Cons.
â˘
Need newer processors
⢠Example
â˘
KVM(basic requirements)
49
50. References
â˘
Performance Evaluation of Intel EPT Hardware Assist, VMware
â˘
I/O Virtualization and AMD's IOMMU
â˘
â˘
Processor-Based Virtualization, AMD64 Style, Part I
â˘
â˘
http://developer.amd.com/documentation/articles/pages/630200614.aspx
Processor-Based Virtualization, AMD64 Style, Part II
â˘
â˘
http://developer.amd.com/documentation/articles/pages/892006101.aspx
http://developer.amd.com/documentation/articles/pages/630200615.aspx
Intel technology Journal, vol 10, issue 3, 2006
Intel virtualization technology: Hardware Support for Efficient processor virtualization
⢠Intel virtualization technology for Directed I/O
â˘
â˘
ARM virtualization Extension Architecture Specification
â˘
A Comparison of software and hardware techniques for x86 virtualization,Vmware
â˘
http://www.intel.com/network/connectivity/solutions/vmdc.htm
â˘
http://www.intel.com/network/connectivity/solutions/vmdq.htm
â˘
http://software.intel.com/en-us/blogs/2009/09/30/understanding-vt-c-virtualizationtechnology-for-connectivity/
50
51. References
⢠Ubuntu 11.10: Xen vs. KVM vs. VirtualBox
http://www.phoronix.com/scan.php?page=article&item=ubuntu_11
10_xenkvm&num=1
⢠Ubuntu 12.04 KVM/Xen Virtualization: Intel vs. AMD
http://www.phoronix.com/scan.php?page=article&item=ubuntu_12
04_virt&num=1
⢠Intel Ivy Bridge Linux Virtualization Performance
http://www.phoronix.com/scan.php?page=article&item=intel_iv
y_virtualization&num=5
⢠http://en.wikipedia.org/wiki/Hypervisor
51