After a short introduction to the goals and approach of the Superfluidity EU research project, we present the proposed extensions to OpenVIM to support ClickOS Unikernels and Xen.
We have implemented a scenario that can combines Unikernels and regular VMs in the same Network Service or VNF extending OpenVIM.We describe how we have extended the ETSI NFV models and OpenVIM. In particular, we provide the details of the OpenVIM descriptor extensions to support Unikernels.
As a background information, we discuss the Unikernels and their orchestration aspects. Unikernel technology allows to build tiny VMs with memory footprint in the order of hundreds of KBs and boot time in the order of milliseconds. We focus on ClickOS Unikernels. We have adapted 3 VIMs (OpenStack, Nomad, OpenVIM) to support ClickOS Unikernels and report a performance evaluation of the VM instantiation time.
Extending OpenVIM R3 to support Unikernels (and Xen)
1. Extending OpenVIM R3 to support Unikernels (and Xen)
Paolo Lungaroni (1), Claudio Pisa(2), Stefano Salsano(2,3), Giuseppe Siracusano(3), Francesco Lombardo(2)
(1)Consortium GARR, Italy; (2)CNIT, Italy; (3)Univ. of Rome Tor Vergata, Italy
Stefano Salsano
Project coordinator – Superfluidity project
Univ. of Rome Tor Vergata, Italy / CNIT, Italy
ETSI OSM-Mid-Release#4 meeting, February 8th, Roma, Italy
A super-fluid, cloud-native, converged edge system
2. Outline
• Superfluidity project goals and approach
• Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers)
• Unikernels orchestration over OpenStack, OpenVIM and Nomad –
Performance evaluation
• Extending ETSI NFV Release 2 models (IFA011, IFA014) and OpenVIM to support
Unikernels orchestration – Live demo
• Details of OpenVIM extensions for Unikernels support (proposal for a patch…)
2
3. Superfluidity project
Superfluidity Goals
• Instantiate network functions and services on-the-fly
• Run them anywhere in the network (core, aggregation, edge), across
heterogeneous infrastructure environments (computing and networking), taking
advantage of specific hardware features, such as high performance accelerators,
when available
Superfluidity Approach
• Decomposition of network components and services into elementary and
reusable primitives (“Reusable Functional Blocks – RFBs”)
• Platform-independent abstractions, permitting reuse of network functions
across heterogeneous hardware platforms
3
4. The Superfluidity vision
4
Current NFV
technology
Granularity
Time scale
Superfluid
NFV
technology
Days, Hours Minutes Seconds Milliseconds
Big VMs
Small
components
Micro
operations • From VNF
Virtual Network Functions
to RFB
Reusable Functional Blocks
• Heterogeneous RFB execution
environments
- Hypervisors
- Modular routers
- Packet processors
…
5. Outline
• Superfluidity project goals and approach
• Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers)
• Unikernels orchestration over OpenStack, OpenVIM and Nomad –
Performance evaluation
• Extending ETSI NFV Release 2 models (IFA011, IFA014) and OpenVIM to support
Unikernels orchestration – Live demo
• Details of OpenVIM extensions for Unikernels support (proposal for a patch…)
5
6. Extending the ETSI NFV models to support Unikernels
6
• In the NFV models, a Virtual Network Function (VNF) is decomposed in
Virtual Deployment Units (VDU)
• We extended the VDU information elements in the model to support
Unikernel VDUs (based on the ClickOS Unikernel)
• “Regular” VDUs based on traditional VMs and Unikernel VDUs can
coexist in the same VNF Descriptor
7. Working prototype (see the live demo!)
7
Orchestrator
prototype
RDCL 3D
VIM
OpenVIM
XEN
We configured XEN to support
both regular VMs (HVM) and
Click Unikernels
NSD
NSD
NSD
ETSI release 2
descriptors
NSD
NSD
VNFD
Our orchestrator
prototype
(RDCL 3D) uses the
enhanced VDU
descriptors and
interacts with
OpenVIM
OpenVIM has been
enhanced to support
XEN and Unikernels
8. Working prototype (see the live demo!)
8
This is a regular
VM (XEN HVM)
These are 3 Unikernel
VMs
(ClickOS)
9. Regular
VM
(Alpine)
Unikernels Chaining Proof of Concept
9
OpenVSwitch
ICMP
responder
(ClickOS)
Firewall
(ClickOS)
OpenVIM
Firewall
Descriptor
ICMP
Responder
Descriptor
VLAN
Encapsulator/
Decapsulator
Descriptor
VLAN
Encap/
Decap
(ClickOS)
“Regular” Linux
Alpine VM
Descriptor
3 Unikernel VMs1 “regular” VM
Extended ETSI NFV Release 2 models
Extended OpenVIM YAML descriptors
RDCL 3D
10. Some details of the working prototype
10
RDCL 3D GUI
VIM
OpenVIM
XEN
NSDNSDNSD
ETSI release 2
descriptorsNSDNSDVNFD
ClickOS images are prepared
“on the fly” by the RDCL 3D agent
using the Click Configuration files
RDCL 3D
Agent
libvirt
ClickOS
images
NSDNSDClick
Click
Configurations
11. Unikernel Chaining Proof of Concept
• Regular VM
– Pings towards the ICMP responder over a VLAN
• VLAN Encapsulator/Decapsulator
– Decapsulates the VLAN header (and re-encapsulates in the return path)
• Firewall
– Lets through only ARP and IP packets with TOS == 0xcc
• ICMP Responder
– Responds to ARP and ICMP echo requests
09/02/2018 CNIT 11
14. Status checks after VM startup
09/02/2018 CNIT 14
After the completion of the VM startup, we can check the status via the Libvirt and Xen command line
tools in the target compute node
• On Libvirt CLI:
$ virsh -c xen:/// list
Id Name State
----------------------------------------------------
105 vm-clickos-ping2_56c0edb0-5b4c-11e7-ad8f-0cc47a7794be running
• On Xen console:
$ sudo xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 10238 8 r----- 96646.2
vm-clickos-ping2_56c0edb0-5b4c-11e7-ad8f-0cc47a7794be 105 8 1 r----- 227.6
16. Outline
• Superfluidity project goals and approach
• Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers)
• Unikernel orchestration over OpenStack, OpenVIM and Nomad –
Performance evaluation
• Extending ETSI NFV Release 2 models (IFA011, IFA014) and OpenVIM to support
Unikernels orchestration – Live demo
• Details of OpenVIM extensions for Unikernels support (proposal for a patch…)
16
17. OpenVIM extensions
1. Extension to OpenVIM to support Xen and Unikernel VMs
2. Extension to OpenVIM for a different networking model (multiple
OvS bridges)
1709/02/2018 CNIT
18. OpenVIM extension 1 (Xen/Unikernels)
Extension to OpenVIM to support Unikernel VMs
• Xen hypervisor support
– Unikernel support (in particular, ClickOS Unikernels)
– Full HVM machines support
– Coexistence on the same compute node of Unikernels and HVM VMs
In order to specify that we are using the Xen hypervisor and the Unikernels, the
configuration is extended by adding a new tags in the object descriptor files.
• No changes in configuration openvim.cfg file.
This extension works in “development” mode and in “normal” mode.
1809/02/2018 CNIT
19. OpenVIM extension 1 (Xen/Unikernels)
• The patch extends the behavior of OpenVIM enabling the support for Xen :
– Orchestrate a unikernel machine such as ClickOS
– Orchestrate a standard Virtual Machines with Xen hypervisor
• Backward compatibility is granted with the original OpenVIM’s modes
(“normal”, “test”, “host only”, “OF only”, “development”)
• NB: We execute our experiments in “development” mode, to run them with
hardware not meeting all the requirements for “normal” OpenVIM mode
1909/02/2018 CNIT
20. Extension 1 : New descriptor tags
Server (VMs) descriptor new tags:
• hypervisor [kvm|xen-unik|xenhvm] defines which hypervisor is used. “kvm”
reflects the original mode, while "xen-unik" and "xenhvm" start xen with support
for Unikernels and full VM respectively.
• osImageType: [clickos] defines the type of Unikernel image to start. It is
mandatory if hypervisor = xen-unik. Currently, only ClickOS Unikernel are
supported, but this tag allows future support of different types of Unikernels.
Host (Compute Nodes) descriptor new flag:
• hypervisors (comma separated list of kvm,xen-unik,xenhvm) defines the
hypervisors supported by the compute node. NB: in a compute node kvm and
xen* are mutually exclusive, while xenhvm and xen-unik can coexist.
2009/02/2018 CNIT
21. Extension 1 : Scheduling enhancements
• The Compute Node is now selected based on the available resources
AND the type of hypervisor.
• If a specific Compute Node is requested for a Server (using the
“hostId” tag), a consistency check between the requested hypervisor
type and the supported hypervisor type in the Compute Node is
performed. An error is returned if the hypervisor type is not
supported.
09/02/2018 CNIT 21
22. Extension 2: OpenVIM Networking enhancements
NB This extension is independent from the previous one, we used it to
support the VNF chaining in the proposed example
• Networking enhancements
– Additional networking model: a separate OVS datapath (within the same OVS
instance) is associated to each OpenVIM network
• It allows transparent L2 networking instead of VLAN based
• It could be extended to work across multiple compute nodes (with VXLAN
tunneling)
2209/02/2018 CNIT
23. Extension 2: An additional Networking Model
09/02/2018 CNIT 23
Open vSwitch
Bridge 1
VNF 1
VNF 2
VNF 3
VNF N
Open vSwitch
Bridge 2
Open vSwitch
Bridge M
External
Network
Open vSwitch
Bridge α
VNF a
VNF b
VNF z
Open vSwitch
Bridge β
Open vSwitch
Bridge ω
VXLAN Tunnel
Compute NodeCompute Node
24. OpenVIM Instantiation sequence
09/02/2018 CNIT 24
OpenVIM deamon
Compute Node
VNF Descriptors files
OpenVIM CLI tool
Libvirt XML
descriptor
POST
create_server
./openvim vm-create clickos-ping.yaml
OpenVIM supports an OpenStack-like REST API on
Northbound side.
A CLI tool called openvim sends command over the
REST APIs to OpenVIM deamon. This tool convert
YAML descriptor to JSON format and sends it via REST.
30. Installation of the extended OpenVIM (R2, R3)
• Download the extended version of OpenVIM in your system from our repository:
$ git clone https://github.com/superfluidity/openvim4unikernels.git
• Install OpenVIM via bash script:
openvim/scripts$ ./install-openvim.sh –noclone
• Our extensions are in the “unikernel” branch:
openvim/scripts$ git checkout unikernel
unikernel/scripts$ ./unikernels_patch_vim_db.sh –u vim –p vimpw install
After updating the database, you can start OpenVIM as usual.
3009/02/2018 CNIT
31. Repository structure
• Unikernel folder contains some tool ad example that is
useful to start work with our patch.
• Descriptors contains some preconfigured ClickOS images
and the descriptors to use as an example to start working
with the Unikernels.
• Docs contains documentation
• Scripts folder contains a bash scripts that updates
OpenVIM database to support the new fields for
unikernel operations and a script for a quick example to
start work with ClickOS.
3109/02/2018 CNIT
32. Conclusions – Feedbacks
• We have designed and implemented a solution for the combined
orchestration of regular VMs and Unikernels
• OpenVIM implementation has been extended. We can propose two
patches:
– 1. Extension to support Xen and Unikernels
– 2. Extension for multi OvS bridges networking model
32
33. Thank you. Questions?
Contacts
Stefano Salsano
University of Rome Tor Vergata / CNIT
stefano.salsano@uniroma2.it
These tools are available on github (Apache 2.0 license)
https://github.com/superfluidity/RDCL3D
https://github.com/superfluidity/openvim4unikernels
https://github.com/netgroup/vim-tuning-and-eval-tools
http://superfluidity.eu/
The work presented here only covers a subset of the work performed in the project
33
34. References
• SUPERFLUIDITY project Home Page http://superfluidity.eu/
• G. Bianchi, et al. “Superfluidity: a flexible functional architecture for 5G networks”, Transactions on
Emerging Telecommunications Technologies 27, no. 9, Sep 2016
• P. L. Ventre, C. Pisa, S. Salsano, G. Siracusano, F. Schmidt, P. Lungaroni,
N. Blefari-Melazzi, “Performance Evaluation and Tuning of Virtual Infrastructure Managers for
(Micro) Virtual Network Functions”,
IEEE NFV-SDN Conference, Palo Alto, USA, 7-9 November 2016
http://netgroup.uniroma2.it/Stefano_Salsano/papers/salsano-ieee-nfv-sdn-2016-vim-performance-for-unikernels.pdf
• S. Salsano, F. Lombardo, C. Pisa, P. Greto, N. Blefari-Melazzi,
“RDCL 3D, a Model Agnostic Web Framework for the Design and Composition of NFV Services”,
submitted paper, https://arxiv.org/abs/1702.08242
34
35. References – Speed up of Virtualization Platforms / Guests
• Light VM project http://cnp.neclab.eu/projects/lightvm/
• F. Manco, C. Lupu, F. Schmidt, J. Mendes, Simon Kuenzer, S. Sati, K. Yasukata, C. Raiciu, F. Huici,
“My VM is Lighter (and Safer) than your Container”, SOSP 2017
• J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco, F. Huici, “ClickOS and the art
of network function virtualization”, NSDI 2014, 11th USENIX Conference on Networked
Systems Design and Implementation, 2014.
• F. Manco, J. Martins, K. Yasukata, J. Mendes, S. Kuenzer, F. Huici,
“The Case for the Superfluid Cloud”, 7th USENIX Workshop on Hot Topics in Cloud Computing
(HotCloud 15), 2015
35
36. References – Unikraft project
• http://cnp.neclab.eu/projects/unikraft/
• https://www.xenproject.org/developers/teams/unikraft.html
The fundamental drawback of unikernels is that they require that applications be manually
ported to the underlying minimalistic OS (e.g. having to port nginx, snort, mysql or memcached to
MiniOS or OSv); this requires both expert work and often considerable amount of time. In
essence, we need to pick between either high performance with unikernels, or no porting effort
but decreased performance and decreased efficiency with standard OS/VM images. The goal of
this proposal is to change this status quo by providing a highly configurable unikernel code base;
we call this base Unikraft.
36
38. Outline
• Superfluidity project goals and approach
• Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers)
• Unikernel orchestration over OpenStack, OpenVIM and Nomad –
Performance evaluation
• Extending ETSI NFV Release 2 models (NFV-IFA 011&014) and OpenVIM to
support Unikernels orchestration – Live demo
• Details of OpenVIM extensions for Unikernels support
38
39. Unikernels: a tool for superfluid virtualization
Containers
e.g. Docker
• Lightweight (not enough?)
• Poor isolation
39
Hypervisors (traditional VMs)
e.g. XEN, KVM, wmware…
• Strong isolation
• Heavyweight
Unikernels
Specialized VMs (e.g. MiniOS, ClickOS…)
• Strong isolation
• Very Lightweight
• Very good security properties
They break the “myth” of VMs being heavy weight…
40. What is a Unikernel?
• Specialized VM: single application +
minimalistic OS
• Single address space,
co-operative scheduler so low
overheads
• Unikernel virtualization platforms
extend existing hypervisors (e.g. XEN)
driver1
driver2
app1
(e.g., Linux, FreeBSD)
KERNELSPACEUSERSPACE
app2
appNdriverN
Vdriver1
vdriver2
app
SINGLEADDRESS
SPACE
40
General purpose OS Unikernel
a minimalistic OS
(e.g., MiniOS, Osv)
41. ClickOS Unikernel
• ClickOS Unikernel combines:
– Click modular router
• a software architecture to build flexible and configurable routers
– MiniOS
• a minimalistic Unikernel OS available with the Xen sources
• ClickOS VMs
– Are small: ~6MB
– Boot quickly: ~ few ms
– Add little delay: ~45µs
– Support ~10Gb/s throughput for almost all packet sizes
09/02/2018 CNIT 41
42. Unikernels (ClickOS) memory footprint and boot time
VM configuration: MiniOS, 1 VCPU, 8MB RAM, 1 VIF
• 4 ms
• 87.77 ms
42
Boot time, state of the art results
Recent results from Superfluidity,
by redesigning the XEN toolstack
Memory footprint
• Hello world guest VM : 296 KB
• Ponger (ping responder) guest VM : ~700KB
43. Unikernels (ClickOS) memory footprint and boot time
VM configuration: MiniOS, 1 VCPU, 8MB RAM, 1 VIF
43
Boot time, state of the art results
Memory footprint
• Hello world guest VM : 296 KB
• Ponger (ping responder) guest VM : ~700KB
Recent results from Superfluidity,
by redesigning the XEN toolstack
• 4 ms
• 87.77 ms
44. VM instantiation and boot time
typical performance (no Unikernels)
44
Orchestrator
request
VIM
operations
Virtualization
Platform
Guest OS (VM)
Boot time
1-2 s
5-10 s
~1 s
45. VM instantiation and boot time
typical performance (no Unikernels)
45
Orchestrator
request
VIM
operations
Virtualization
Platform
Guest OS (VM)
Boot time
1-2 s
~1 ms
~1 ms
XEN Hypervisor
Enhancements
Unikernels
Unikernels and Hypervisor can provide
low instantiation times for “Micro-VNF”
46. VM instantiation and boot time
typical performance (no Unikernels)
46
Orchestrator
request
VIM
operations
Virtualization
Platform
Guest OS (VM)
Boot time
1-2 s
~1 ms
~1 ms
XEN Hypervisor
Enhancements
Unikernels
Can we improve VIM
performances?
Unikernels and Hypervisor can provide
low instantiation times for “Micro-VNF”
47. Outline
• Superfluidity project goals and approach
• Unikernels and their orchestration using VIMs (Virtual Infrastructure Managers)
• Unikernels orchestration over OpenStack, OpenVIM and Nomad –
Performance evaluation
• Extending ETSI NFV Release 2 models (IFA011, IFA014) and OpenVIM to support
Unikernels orchestration – Live demo
• Details of OpenVIM extensions for Unikernels support (proposal for a patch…)
48
48. Performance analysis and Tuning of
Virtual Infrastructure Managers (VIMs) for Unikernel VNFs
• We considered 3 VIMs (OpenStack, Nomad, OpenVIM)
49
- General model of the VNF instantiation process, mapping of the
operations of the 3 VIMs in the general model
- (Quick & dirty) modifications to VIMs to instantiate Micro-VNFs
based on ClickOS Unikernel
- Performance Evaluation
49. Virtual Infrastructure Managers (VIMs)
We considered three VIMs :
• OpenStack Nova
– OpenStack is composed by subprojects
– Nova: orchestration and management of computing resources ---> VIM
– 1 Nova node (scheduling) + several compute nodes (which interact with the hypervisor)
– Not tied to a specific virtualization technology
• Nomad by HashiCorp
– Minimalistic cluster manager and job scheduler
– Nomad server (scheduling) + Nomad clients (interact with the hypervisor)
– Not tied to a specific virtualization technology
• OpenVIM
– NFV specific VIM, originally developed by the OpenMANO open source project, now
maintained in the context of ETSI OSM 50
50. Results – ClickOS instantiation times
(OpenStack, Nomad, OpenVIM)
51
OpenStack Nova
Nomad
seconds
seconds
OpenVIM
seconds
51. The SUPERFLUIDITY project has received funding from the European Union’s Horizon
2020 research and innovation programme under grant agreement No.671566
(Research and Innovation Action).
The information given is the author’s view and does not necessarily represent the view
of the European Commission (EC). No liability is accepted for any use that may be
made of the information contained.
53