This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional. Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings. The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.

1. © Copyright 2019 Xilinx
Stefano Stabellini – Xen Developer Summit July 2019
Xen Dom0-less

2. © Copyright 2019 Xilinx
Traditional Xen System Configuration and Boot
>> 2
U-Boot
Xen
Dom0
DomU 1 DomU 2
CPU0 CPU1 CPU2
boots
xl

3. © Copyright 2019 Xilinx
Traditional Xen System Configuration and Boot
>> 3
U-Boot
Xen
Dom0
DomU 1 DomU 2
CPU0 CPU1 CPU2
boots
xl
quit

4. © Copyright 2019 Xilinx
The problem
˃ Boot Time
total > xen + dom0_kernel + dom0_user
˃ Safety Certifications
Dom0 doesn’t have to be Linux but it typically is
non-Linux Dom0
exit Dom0 after boot
˃ Complexity
build-time complexity
‒ Yocto rootfs build
runtime flexibility
‒ Monitoring
‒ VMs restart

5. © Copyright 2019 Xilinx
Dom0-less

6. © Copyright 2019 Xilinx
Dom0-less System Configuration and Boot
>> 6
U-Boot
Xen
Dom0 DomU 1 DomU 2
CPU CPU CPU
loads into memoryloads into memory

7. © Copyright 2019 Xilinx
Dom0-less System Configuration and Boot
>> 7
U-Boot
Xen
Dom0 DomU 1 DomU 2
CPU CPU CPU
boots
boots

8. © Copyright 2019 Xilinx
Dom0-less System Configuration and Boot
>> 8
U-Boot
Xen
Dom0 DomU 1 DomU 2
CPU CPU CPU
Dom0DomU 3
CPU
xl
Frontend Backend
Dom0-less VMs

9. © Copyright 2019 Xilinx
U-Boot + Device Tree protocol
˃ Load all the required binaries via U-Boot commands
tftpb 0x4000000dom0less/mpsoc.dtb
tftpb 0x80000 dom0less/Image-dom0
tftpb 0x5000000dom0less/uXen
tftpb 0xd000000 dom0less/dom0-ramdisk.cpio.uboot
tftpb 0xa000000 dom0less/Image-domU
tftpb 0xb000000 dom0less/domU-ramdisk.cpio
bootm 0x5000000 0xd000000 0x4000000

10. © Copyright 2019 Xilinx
U-Boot + Device Tree protocol
˃ Advertise and configure Dom0-less VMs via Device Tree
domU1 {
compatible = "xen,domain";
memory = <0x20000>;
cpus = 1;
vpl011;
module@a000000 {
compatible = "multiboot,kernel", "multiboot,module";
reg = <0xa000000 0xffffff>;
bootargs = "console=ttyAMA0";
};
module@b0000000 {
compatible = "multiboot,ramdisk", "multiboot,module";
reg = <0xb000000 0xffffff>;
};
};

11. © Copyright 2019 Xilinx
Dom0-less Device Assignment
>> 11
U-Boot
Xen
Dom0 DomU 1 DomU 2
CPU CPU CPU
boots
boots
Network
Card
Other
Devices

12. © Copyright 2019 Xilinx
Dom0-less Device Assignment
˃ Configured via a nested device tree snippet
domU1 {
[…]
module@a000000 {
compatible = "multiboot,kernel", "multiboot,module";
reg = <0xa000000 0xffffff>;
bootargs = "console=ttyAMA0";
};
module@b0000000 {
compatible = "multiboot,ramdisk", "multiboot,module";
reg = <0xb000000 0xffffff>;
};
module@c000000 {
compatible = "multiboot,device-tree", "multiboot,module";
reg = <0xc0000000xffffff>;
};
};

13. © Copyright 2019 Xilinx
Dom0-less Device Assignment
˃ Configured via a nested device tree snippet
the device tree node of the device to assign
same as for regular DomUs
Special properties:
‒ interrupts: interrupts to remap
‒ interrupt-parent: special reference to GIC parent
(65000)
‒ xen,path: path to the device node in the main DT
‒ xen,reg: memory to remap
/dts-v1/;
/ {
#address-cells = <0x2>;
#size-cells = <0x1>;
passthrough {
compatible = "simple-bus";
ranges;
#address-cells = <0x2>;
#size-cells = <0x1>;
ethernet@ff0e0000 {
compatible = "cdns,zynqmp-gem";
status = "okay";
interrupt-parent = <0xfde8>;
interrupts = <0x0 0x3f 0x4 0x0 0x3f 0x4>;
xen,path = "/amba/ethernet@ff0e0000";
xen,reg= <0x0 0xff0e0000 0x1000 0x0 0xff0e0000>;
reg = <0x0 0xff0e0000 0x1000>;
clock-names = "pclk", "hclk", "tx_clk", "rx_clk";
#address-cells = <0x1>;
#size-cells = <0x0>;
[…]

14. © Copyright 2019 Xilinx
True Dom0-less
>> 14
U-Boot
Xen
DomU 1 DomU 2
CPU CPU
boots
boots
Network
Card
PL Block
boots

15. © Copyright 2019 Xilinx
Dom0-less and safety critical applications
>> 15
U-Boot
Xen
Dom0
DomU 1
Safety Critical
DomU 2
Safety Critical
CPU CPU CPU
boots
boots
DomU 3
Non-Critical
CPU
xl
Network
Card

16. © Copyright 2019 Xilinx
Dom0-less and PV drivers
>> 16
U-Boot
Xen
DomU 1 DomU 2
CPU CPU
boots
boots
Network
Card
Netfront
boots
Netback
Doesn’t
work
today!

17. © Copyright 2019 Xilinx
Dom0-less Pros & Cons
Pros:
˃ Much faster startup times
total ~= xen + domU
˃ Enable true Dom0-less configurations
Excellent for small systems
Easier to certify
˃ Lower Complexity
No need for the Xen tools
Does not require Yocto, just cross-build Xen
No need for Xen support in Dom0-less VMs, no need for CONFIG_XEN
Cons:
˃ No monitoring and restarting DomUs without Dom0
˃ No PV frontends/backends without Dom0

18. © Copyright 2019 Xilinx
Status & TODO
˃ DONE
basic Dom0-less booting upstream in Xen 4.12
device assignment implemented and sent to the list (not upstream)
˃ TODO
True Dom0-less
Shared memory and interrupts for VM-to-VM communications
PV frontends/backends drivers for Dom0-less VMs

19. © Copyright 2019 Xilinx
Demo

20. © Copyright 2019 Xilinx
Dom0-less and PVCalls
>> 20
U-Boot
Xen
Dom0
DomU 1 DomU 2
CPU CPU CPU
boots
boots
DomU 3
CPU
xl
Frontend Backend
Network
Card

21. © Copyright 2019 Xilinx
Dom0-less and PVCalls
>> 21
U-Boot
Xen
Dom0
DomU 1 DomU 2
CPU CPU CPU
boots
DomU 3
CPU
xl
Frontend Backend
Network
Card
Private Network

22. © Copyright 2019 Xilinx
Questions?

23. © Copyright 2019 Xilinx
Adaptable.
Intelligent.