Containers are extremely convenient to package applications and deploy them quickly across the data center. They enable microservices oriented approaches to the development of complex apps. These technologies are benefiting the data center, but are struggling to find their place in embedded environments. As embedded developers, we would like the convenience of containers for deployment, while retaining real-time capabilities and supporting mixing and matching of applications with different safety and criticality profiles on the same board. This talk will introduce ViryaOS, a new OSS project to create the ideal platform for secure containers in embedded. It will discuss the design goals and the key runtime components. It will cover the new Virya cross-architecture build system, based on Docker containers, and designed to build and assemble multiple domains into a single target image.

1. Stefano Stabellini @stabellinist
ViryaOS: Secure Containers for Embedded and IoT
A proposal for a new Xen Project sub-project

2. Problem #1:
Multi-domains builds and
deployments are difficult

3. Embedded and IoT use patterns
Typically users know all the VMs they need beforehand
They still need to:
• build them all, plus Xen and Dom0
• install all images on target
• partition the hardware using device assignment
– edit the Dom0 device tree
– generate appropriate device trees for DomUs with device nodes
• plan for image upgrades and security fixes

4. It’s a lot of work!

5. You think this is bad enough...
...then you try disaggregation

6. Current status
Everybody has their own scripts and handcrafted solutions
– They are limited
– Only target one use-case
– Limited support for driver domains and service domains
– Only support one hardware platform
→ We would all benefit from a unifying effort

7. Problem #2:
Users want Docker in IoT,
Why?

8. The problem
Package applications for the target
Contain all dependencies
Easy to update, Independent lifecycle
Run applications on the target
Run in isolation
No interference between applications

9. The problem
Package applications for the target
Contain all dependencies
Easy to update, Independent lifecycle
Run applications on the target
Run in isolation
No interference between applications

10. The problem
Package applications for the target
Contain all dependencies
Easy to update, Independent lifecycle
Run applications on the target
Run in isolation
No interference between applications

11. Packaging vs. Runtime
OCI Image Spec vs. OCI Runtime Spec

12. Linux Kernel
Linux Namespaces
Containers != Linux Namespaces
Docker Registry
Linux Namespaces
Cloud-Native App
Linux Namespaces
Cloud-Native App
App binaries
App libraries
Cloud Native App
(rootfs +
manifest)
App binaries
App libraries
Docker

13. The problem with Linux namespaces
Cloud-native
App
Malicious App
Linux kernel
POSIX
Cloud-native
App
Large surface of attack
On average, 3 privilege
escalation vulnerabilities per
Linux release!

14. Linux Namespaces: very embedded problems
Mixed-criticality workloads are not supported
Limits on resources utilization hard to enforce
Real-time support is difficult
Certifications are very difficult

16. Sta 1-Xen

17. Containers in Embedded and IoT
Docker is a fantastic tool
Containers are a good packaging format
We would all benefit from a technology to help us deploying
them securely in embedded and IoT environments

18. ViryaOS
A proposal for a new Xen Project
sub-project

19. ViryaOS
A Flexible build system
A Secure Xen based runtime
Containers supported natively
A turnkey solution
Supports ARM64 and x86_64
Targeted at Embedded and IoT

20. ViryaOS: Build
A multi-domain build system
Builds multiple domains in one go
Creates a runnable SD Card image from multiple domain builds
Each domain build is independent and runs in a container
Pre-configured device passthrough to guests
Made for disaggregated architectures

21. ViryaOS Build
Image
Builder
SD
Card
Image
apks
Image.
gz
Dom0 -
Alpine Linux
Dom0 kernel
- Yocto
Dom1 DriverD
- Yocto
Rootfs
Dom2 DriverD -
Alpine Linux apks

22. ViryaOS Build
Everything builds in a container
Supports cross-builds (ARM64 targets on x86) with qemu-user
Supports any build systems for domain builds
– enables mixed Alpine Linux / Yocto environments
– rootfs and kernel can be built independently
Supports multiple DomU build output formats
The DomU output itself is stored in a container
Anything can be pull to the Docker Hub to speed up the build

23. ViryaOS Runtime
Supports VMs and containers
Containers are run securely, transparently as Xen VMs
1 Kubernetes Pod per VM
See stage1-xen
Measured Boot
Software Updates
Uses disaggregation, service domains and driver domains

24. ViryaOS Partitions Scheme and Updates
P1: Xen + Dom0 kernel + Dom0 rootfs
P2: fallback / “P1-next”
LVM volumes for regular VMs, no redundancy
FAT32 partition for UEFI booting: bootloader EFI binary

25. ViryaOS Runtime
Xen
Dom0
Containers/
DomUs
Hardware
CRI/ContainerD + Stage1-Xen
Kubernetes

26. Xen
Dom0
Domain
Manager
Kubernetes
DomUs
Hardware
CRI/Containerd
+ Stage1-Xen
Internal API

27. Xen
Dom0
Domain
Manager
Kubernetes
DomUs
Hardware TPM
Network
Manager
NIC
TPM
Manager
CRI/ContainerD
+ Stage1-Xen
Internal API

28. Current status
Temporary mailing list
GitLab repositories
Early stages of development
Initial implementation available for:
• SDK
• Containers-driven build
• Yocto kernel build
Alpine Linux Dom0 rootfs and ImageBuilder in progress

29. Questions?

30. We are hiring!