During the last few months of 2011 the Xen Community started an effort to port Xen to ARMv7 with virtualization extensions, using the Cortex A15 processor as reference platform. The new Xen port is exploiting this set of hardware capabilities to run guest VMs in the most efficient way possible while keeping the ARM specific changes to the hypervisor and the Linux kernel to a minimum. Developing the new port we took the chance to remove legacy concepts like PV or HVM guests and only support a single kind of guests that is comparable to "PVH" in the Xen X86 world. Linux 3.7 was the first kernel release to run on Xen on ARM as Dom0 and DomU. Xen 4.3, out in July 2013, is the first hypervisor release to support ARMv7 with virtualization extensions and ARMv8. This talk will explain why ARM virtualization is set to be increasingly relevant for the automotive industry in the coming years. We will go on to describe how Xen exploits the strengths of the hardware to meet the requirements of the industry. We will illustrate the early design choices and we will evaluate whether they were proven successful or a failure.

1. Xen on ARM
Stefano Stabellini

2. What is Xen?
● a type-1 hypervisor
● small footprint (less than 90K LOC)

3. Xen: Open Source
GPLv2 with DCO (like Linux)
Diverse contributor community

4. Xen: Open Source
source:
Mike Day
http://code.ncultra.org

5. Xen: the gears of the cloud
● large user base
more than 10 million individuals users
● power the largest clouds in
production
● not just for servers

6. Xen Architecture
Dom0
DomU
DomU
DomU
PV backends
PV Frontends
PV Frontends
PV Frontends
HW drivers
Xen
Hardware

7. Xen Architecture:
driver domains
Toolstack
Disk Driver
Domain
Network
Driver Domain
DomU
BlockBack
Dom0
NetBack
BlockFront
Disk Driver
Network Driver
NetFront
Xen
Hardware

8. Xen Architecture:
driver domains
● disaggregation and componentization
● security and isolation
● resilience
● hardware vendors can run their drivers in
separate VMs
○ could run in a RTOS environment
○ hidden from the user
■ media codecs, crypto keys, etc.

9. Xen on ARM: not just a port
● a lean and simple architecture
○
○
○
○
○
removed cruft accumulated during the years
no emulation, no QEMU
use PV drivers for IO as early as possible
one type of guest
exploit the hardware as much as possible
● a very good match for the hardware
● clean architecture = small code base

10. Xen on ARM architecture

11. Xen on ARM architecture

12. Code size
sometimes smaller is better
Common
ARMv7
ARMv8
Total
xen/arch/arm
5,122
1,969
821
7,912
C
5,023
406
344
5,773
99
1,563
477
2,139
2,315
563
666
3,544
2,532 1,487
11,456
ASM
xen/include/asm-arm
TOTAL
7,437
● X86_64-bit: ~120,000LOC (~4,000 ASM)
● ARM code ~= 1/10 x86_64 code

13. Achievements of one year
Xen support for ARM
upstream in Linux 3.7
Xen support for ARM64
upstream in Linux 3.11
Xen 64-bit
on ARM64
You are here
11/11
08/12 09/12
Part-time Xen ARM
hacking starts
11/12
Xen running on real
ARM hardware
First Xen on ARM talk at
Xen Summit 2012
01/13
03/13
Citrix announces
that will be joining
Linaro
06/13
07/13
Xen 4.3 released with
ARM and ARM64
support

14. A growing community
Xen-devel ARM traffic from August 2012:
● 4685 emails: 360 emails per month!
● 39% of which are not from Citrix

15. Hardware support
Upstream:
● Versatile Express Cortex A15
● Arndale board
● ARMv8 FVP
In progress:
●
●
●
●
●
Cubieboard2
Calxeda “Midway”
Applied Micro “Mustang”
Broadcom Brahma-B15
OMAP5

16. Porting Xen to a new board
● Xen only relies on GIC and GT
● platform specific code in Xen is reduced to:
○ secondary cpus bring up
○ UART drivers
○ any platform specific bootup quirks (ideally none)

17. Upstream features
Xen v4.3:
● basic lifecycle operations
● memory ballooning
● scheduler configurations and vcpu pinning
Linux v3.11:
● dom0 and domU support
● 32-bit and 64-bit support
● SMP support
● PV disk, network and console

18. Android on Xen on ARM
● Android is based on the Linux kernel
○ Jelly Bean uses Linux 3.4, when it updates to 3.7 will
get Xen on ARM support out of the box
● Additional work needed to support client
devices (compass, GPS, etc.) on multiple
VMs
○ easy to export 1 device to 1 VM
○ otherwise each type of device needs a PV drivers
pair

19. PV Protocols
Backend
Request Producer
Frontend
Request Consumer
Response
Consumer
Response Producer

20. PV Protocols
●
●
●
●
●
shared ring protocol
software interrupts AKA event channels
consensual memory sharing: grant table
easy to write
plenty of examples
○ network, block, console, PCI, keyboard, mouse,
framebuffer, sound, SCSI, USB, …

21. Porting other OSes to Xen
on ARM
● No invasive modifications needed
● only some new drivers:
○ grant table, xenbus and event channels
○ PV drivers for network, block, console, etc.
● BSD drivers already exist in NetBSD and
FreeBSD, can they be reused?
● FreeBSD port to Xen on ARM in progress

22. Coming in Xen 4.4
● 64-bit guest support
● live-migration
● SWIOTLB

23. A look into the future
●
●
●
●
IOMMU support in Xen
device assignment
UEFI booting
ACPI support

24. Xen for automotive: why?
● type-1
● small footprint
○ small codebase
○ no QEMU, no emulation
● driver domains / service VMs
○ componentization
○ security
○ support for legacy drivers
● supports Linux out of the box
● easy to port other OSes to Xen on ARM

25. More Information
● http://www.xenproject.org
● Xen on ARM @wiki.xenproject.org goo.
gl/FKNXe
● http://lists.xen.org/mailman/listinfo/xen-devel

26. Questions?