Linaro has published VM System specification for ARM Processors, which provides a set of guidelines for both guest OS and hypervisor implementations, such that building OS images according to these guidelines guarantees that those images can also run on hypervisors compliant with this specification. One of the spec requirements is that the hypervisor must provide an emulated PL011 UART as a serial console which meets the minimum requirements in SBSA UART as defined in ARM Server Base Architecture Document. In this presentation, Bhupinder Thakur will cover the motivation behind VM System spec requirements and the work done for adding support of PL011 emulation in Xen.

1. SBSA UART Emulation in Xen on 64-bit ARM
Server Processor
Bhupinder Thakur
Principal Engineer
Qualcomm Datacenter Technologies
Xen Developer and Design Summit
July 11-13 2017, Budapest Hungary

2. ▪ Working as a Linaro assignee from Qualcomm Datacenter
Technologies
▪ Working on VM spec compliance for ARM based servers
▪ First open source contribution
▪ SBSA UART emulation in Xen
About The Speaker

3. Challenges in implementing ARM as a new ISA
▪ ARM processors are built into a large variety of systems
▪ Feature requirements vary from chip to chip
▪ This variability in the architecture impacts the software system development
▪ x86 relies on PC standards
▪ Challenge: How to provide software portability across ARM
systems ?

4. Enabling ARM as a First Class Architecture
▪ SBSA (Server Base System Architecture)
▪ It defines minimum requirements from the hardware such as CPU, PCIe, IOMMU, UART, timers and
interrupt
▪ SBBR (Server Base Boot Requirements)
▪ Firmware must be UEFI/ACPI compliant
▪ UEFI for server booting
▪ ACPI for describing the hardware resources that are installed
▪ Any compliant software is guaranteed to run on any SBSA/SBBR compliant hardware
▪ The specifications can be downloaded from
▪ http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0029/index.html

5. Challenges in Enabling VM System for ARM Processors
▪ SBSA/SBBR specs were defined for the physical systems
▪ Similar standardization was required for the virtual systems
▪ How should the VM bring up the guest OS ?
▪ How should the guest OS discover the devices ?
▪ What are mandatory peripherals to be supported by the hypervisor ?
▪ Linaro published a specification to standardize the requirements from the hypervisor/OS/firmware
▪ http://www.linaro.org/app/resources/WhitePaper/VMSystemSpecificationForARM-v2.0.pdf

6. ▪ Virtual Firmware
▪ UEFI compliant virtual firmware
▪ UEFI binary must be executed as the first software by the VM which then executes the EFI
application
▪ VM Image Format
▪ The raw disk format must be partitioned with a GUID table
▪ The bootable software must reside in a EFI partition and must be an EFI application
▪ Hardware Description
▪ The virtual UEFI firmware must implement ACPI and DT tables
VM System Specification for ARM Processors

7. ▪ Mandatory Hypervisor Peripherals
▪ SBSA UART
▪ Support for both GICv2 and GICv3 (Generic Interrupt Controller)
▪ Virtual timer and counter system register must be available to the VM
▪ Virtual bus that supports block and network devices hotplug
▪ The guest OS
▪ must include console driver for SBSA UART
▪ must include support for both GICv2 and GICv3
▪ strongly recommended to include support for block, network, console, balloon, virtio and Xen PV
drivers
VM System Specification for ARM Processors

8. SBSA UART
▪ A simple UART which offers basic facility for software bring up
▪ It is a subset of ARM PL011 UART
▪ Supports only a subset of registers to handle data, status and interrupts
▪ No support for features such as DMA, hardware flow control etc.
▪ It is defined in Appendix B of the SBSA specification

9. Benefits of Emulating SBSA UART in Xen
▪ Compliant with the VM specification
▪ The guest OS can access the console even if it does not include PV console driver
▪ Can be used for debugging if PV console is not working for some reason

10. Options Available for Emulating SBSA UART
▪ Emulating it in Dom0 using QEMU
▪ Emulating it in the Xen hypervisor

11. Challenges in Emulating SBSA UART in Xen
▪ Xen/ARM does not use QEMU
▪ Typically devices are not emulated in Xen/ARM (GIC is an exception)

12. Current Design for Emulating SBSA UART
App App
Kernel
DomU
XencosoleApp
Kernel
Dom0
EL0
EL1
EL2
SBSA UART Emulation Logic
▪ Emulate SBSA UART in the Xen
hypervisor

13. Current Design for Emulating SBSA UART
App App
Kernel
DomU
XencosoleApp
Kernel
Dom0
EL0
EL1
EL2
SBSA UART Emulation Logic
▪ Emulate SBSA UART in the Xen
hypervisor
▪ Reuse the PV console backend to
minimize the changes in Dom0

14. Current Design for Emulating SBSA UART
App App
Kernel
DomU
Xenco
sole
App
Kernel
Dom0
Xen
EL0
EL1
EL2
SBSA
UART
Emulation
Logic
▪ Emulate SBSA UART in the Xen
hypervisor
▪ Reuse the PV console backend to
minimize the changes in Dom0
▪ Future: Move emulation to Xen
non-privileged (EL0) mode

15. Xen
DomUDom0
User I/O
Events
IN
OUT
PV Console in Xen

16. Implemenation of SBSA UART Emulation in
Xen
DomUDom0
User I/O
MMIO Traps
IN ring buffer
OUT ring buffer
Device Tree
SBSA UART
emulation
IRQ
Events
DOMCTL
▪ Changes in the Xen hypervisor
▪ MMIO trap handling
▪ IRQ injection
▪ Event handling
Xen

17. Implementation of SBSA UART Emulation in
Xen
DomUDom0
User I/O
MMIO Traps
IN ring buffer
OUT ring buffer
Device Tree
SBSA UART
emulation
IRQ
Events
DOMCTL
▪ Changes in the toolstack
▪ New DOMCTL API to enable emulation in
Xen
▪ New SBSA UART node added in the
xenstore
▪ New SBSA UART node added to the
device tree
Xen

18. Implementation of SBSA UART Emulation in
Xen
Device Tree ▪ Changes in Xenconsole
▪ Multiple console support added
▪ New option “vuart” added in DomU.cfg
file to enable sbsa uart for a domain
▪ xl console command enhanced to allow
the user to connect to vuart
DomUDom0
User I/O
MMIO Traps
IN ring buffer
OUT ring buffer
SBSA UART
emulation
IRQ
Events
DOMCTL
Xen

19. Using SBSA UART
Sample DomU.cfg file
vuart = “sbsa_uart”
extra = “console=ttyAMA0 …”
Start the domain with SBSA UART enabled
xl create domU.cfg
Connect to SBSA UART
xl console –t vuart <domain name>

20. TBD Items
▪ SBSA UART support in UEFI firmware
▪ Currently UEFI firmware uses PV console for input/output
▪ Add support for SBSA UART
▪ How should the guest OS select the default console ?
▪ Currently PV console is the default console
▪ Add SBSA UART support in ACPI

21. Upstream Status
▪ The SBSA UART patch set is under review on the xendev mailing list
▪ It has gone through five revisions
▪ The changes have matured over these revisions
▪ Hopefully it should be included in 4.10
▪ The code is available @ https://git@git.linaro.org:/people/bhupinder.thakur/xen.git branch: vpl011_v5

22. Thank You!

23. Xenconsole Changes
Handle_ring_read()
DomU
Xen
PV Ring buffer
write and event
raised
PL011 ring buffer
write and event
raised
Buffer_append()PV buffer
SBSA UART
buffer
Handle_tty_write()
Write to PV
tty client
Handle_tty_read()
DomU
Xen
PV Ring buffer write and
event raised
SBSA UART Ring
buffer write and event
raised
Read from PV tty
client
Read from PV tty
client
PV console logs SBSA UART
console logs
console_create_ring()
console_create_tty()
create_console_log()
console_cleanup()
…
Write to
SBSA UART
tty client