Romana, the open source project by Pani Networks, brings stunning simplicity to the usually so complex networking in OpenStack and Kubernetes. Using only native L3 routing and no overlays, along with automated distributed application of network policies and security rules, it provides operators with easy to understand and manage networking, while allowing network hardware to operate at its best and with full efficiency. These slides were used during the OpenStack meetup in Auckland in May 2016, hosted by Catalyst IT.

1. Simplifying the network stack with
Romana
Pani Networks
OpenStack Meetup, Auckland, May 2016

2. romana.io Simplifying the network stack with Romana @romanaproject
Agenda
● “Cloud native”, why does it matter?
● A better network for cloud native architectures
● Demos

3. romana.io Simplifying the network stack with Romana @romanaproject
About us
● Team background:
– Data center networks
– Low-level traffic management
● Created L2 overlay network startup
– Bought by Cisco
● OpenStack networking
● There's got to be a better way
– Time is right

4. What is 'cloud native'?

5. romana.io Simplifying the network stack with Romana @romanaproject
The past: Enterprise networking
● Full control
● Applications need L2 and L3
– May need hard-wired IP addresses
– Broadcasts
● Servers are pets, not cattle: “Careful!”
– VM migration
● Complex!

6. romana.io Simplifying the network stack with Romana @romanaproject
Cloud native applications
● Automate all the things!
– Infrastructure as code
– Cattle, not pets: “Meh... just kill it.”
– Workloads come and go quickly
– Build for resiliance
● IP is all you need
– No hardcoded IP addresses, discovery
– No special network requirements
– Basic IP connectivity

7. The problem

8. romana.io Simplifying the network stack with Romana @romanaproject
We have a mismatch
● Building cloud native applications…
● … on top of enterprise networking
– SDN controllers use overlay L2 domains
– VLAN, VXLAN, OVS, etc.
● Complexity and brittleness
– Lose benefits of simplicity
– Lose performance (encap, blinded hardware)
– Difficult to maintain and trouble shoot

9. romana.io Simplifying the network stack with Romana @romanaproject
The price you pay: Complexity
VXLAN Decap
VXLAN Decap
VXLAN Encap
VXLAN Encap
2 Top of Rack Round
Trips
East/West Traffic
Per Instance Security

10. romana.io Simplifying the network stack with Romana @romanaproject
The price you pay: Performance
Router
Endpoint A Endpoint B
Router
L2 overlay A
L2 overlay B
VRouter

11. romana.io Simplifying the network stack with Romana @romanaproject
Why do we do this to ourselves?
● We don't need any L2 features
● Except maybe traffic segmentation
– Multi tenancy
– Tiers and policies

12. The solution

13. romana.io Simplifying the network stack with Romana @romanaproject
Networking the way it was intended
● Use native L3 capabilities
● No overlays
● De-emphasize IP address ranges
● Still provide segmentation, multi tenancy
● Simple, clear and scalable network setup

14. romana.io Simplifying the network stack with Romana @romanaproject
Truly cloud native networking
● Project Romana
● Open source
● Apache 2.0 license
● Mostly written in Go
● Kubernetes and OpenStack

15. romana.io Simplifying the network stack with Romana @romanaproject
Truly cloud native networking
● Use only IP routing
– No overlays
– All workload addresses are 'real'
– Simplicity!
● Use smart addressing
– Encode tenant or segment in IP address
– Assign “virtual” addresses with host prefixes
– Massive (!) collapse of route table
● Routes are static
– No route updates, no broadcasts for new endpoint

16. romana.io Simplifying the network stack with Romana @romanaproject
Romana Architecture
● On each host: Agent
– Configures routes
– Connects endpoint interfaces
– Sets policy implementations
●
Controller: Cooperating microservices
– Each service with RESTful interface
– Specialized for different tasks
● Environment: Different integration points
– APIs, drivers for various parts of OpenStack or
Kubernetes

17. romana.io Simplifying the network stack with Romana @romanaproject
Romana Architecture
Host A Host B Host C
Agent Agent Agent
Tenant
Topology
IPAM
Root
Environment (OpenStack or Kubernetes)
Policy

18. Beautifully simple networking

19. romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:
192.168.8.11
Host B
eth0:
192.168.8.22
Host C
eth0:
192.168.8.33

20. romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:
192.168.8.11
romana-gw:
10.0.0.1/16
Host B
eth0:
192.168.8.22
romana-gw:
10.1.0.1/16
Host C
eth0:
192.168.8.33
romana-gw:
10.2.0.1/16

21. romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:
192.168.8.11
romana-gw:
10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Host B
eth0:
192.168.8.22
romana-gw:
10.1.0.1/16
10.1.3.52
10.1.9.2
Host C
eth0:
192.168.8.33
romana-gw:
10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6

22. romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:
192.168.8.11
romana-gw:
10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Host B
eth0:
192.168.8.22
romana-gw:
10.1.0.1/16
10.1.3.52
10.1.9.2
Host C
eth0:
192.168.8.33
romana-gw:
10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6

23. romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:
192.168.8.11
romana-gw:
10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Host B
eth0:
192.168.8.22
romana-gw:
10.1.0.1/16
10.1.3.52
10.1.9.2
Host C
eth0:
192.168.8.33
romana-gw:
10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6

24. romana.io Simplifying the network stack with Romana @romanaproject
Routing and route aggregation
Host A
eth0:
192.168.8.11
romana-gw:
10.0.0.1/16
10.0.0.5
10.0.1.7
10.0.1.19
10.0.5.3
Routes:
10.1/16 → 192.168.8.22
10.2/16 → 192.168.8.33
Host B
eth0:
192.168.8.22
romana-gw:
10.1.0.1/16
10.1.3.52
10.1.9.2
Routes:
10.0/16 → 192.168.8.11
10.2/16 → 192.168.8.33
Host C
eth0:
192.168.8.33
romana-gw:
10.2.0.1/16
10.2.0.16
10.2.3.81
10.2.4.6
Routes:
10.0/16 → 192.168.8.11
10.1/16 → 192.168.8.22

25. romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
Rack A Rack B

26. romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10

27. romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
Host A2 Routes
0.0.0.0 192.168.1.200→
10.68/14 192.168.1.1→
10.76/14 192.168.1.3→
10.80/14 192.168.1.4→

28. romana.io Simplifying the network stack with Romana @romanaproject
Larger network: L2 under ToR
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
ToR A Routes
10.128/10 192.168.2.200→
10.68/14 192.168.1.1→
10.72/14 192.168.1.2→
10.76/14 192.168.1.3→
10.80/14 192.168.1.4→
Host A2 Routes
0.0.0.0 192.168.1.200→
10.68/14 192.168.1.1→
10.76/14 192.168.1.3→
10.80/14 192.168.1.4→

29. romana.io Simplifying the network stack with Romana @romanaproject
Larger network: Full L3
Host B1
Host B2
Host B3
Host B4
Host A1
ToR A ToR B
spine network
192.168.1.200 192.168.2.200
192.168.1.1
Host A2
192.168.1.2
Host A3
192.168.1.3
Host A4
192.168.1.4
10.68/14
10.72/14
10.76/14
10.80/14
192.168.2.1
192.168.2.2
192.168.2.3
192.168.2.4
10.132/14
10.136/14
10.140/14
10.144/14
Rack A Rack B
10.64/10 10.128/10
ToR A Routes
10.128/10 192.168.2.200→
10.68/14 192.168.1.1→
10.72/14 192.168.1.2→
10.76/14 192.168.1.3→
10.80/14 192.168.1.4→
Host Routes
0.0.0.0 192.168.1.200→

30. Scalable distributed firewall
and
traffic policies

31. romana.io Simplifying the network stack with Romana @romanaproject
Romana: Traffic segmentation
● Tenant traffic separated:
– Tenants don't get whole CIDR prefix or L2 domain
– But fully isolated from other tenants' traffic
● Tenants can define segments:
– Like tiers, provide isolation and policies
● Use segment and tenant bits in IP addresses:
– Apply policies (iptables) based on that
– Segments can stretch across hosts

32. romana.io Simplifying the network stack with Romana @romanaproject
Semantic and topological addressing
3
1
3
0
2
9
2
8
2
7
2
6
2
5
2
4
2
3
2
2
2
1
2
0
1
9
1
8
1
7
1
6
1
5
1
4
1
3
1
2
1
1
1
0
9 8 7 6 5 4 3 2 1 0
0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1
10
Network prefix bits
The network prefix.
In this example, we
are using the 10/8
address space.
6
Host ID Segment ID
We currently
store tenant ID in
upper bits of
segment ID.
4 67
Endpoint ID
Widths are configurable, don't have to use byte boundaries.

33. romana.io Simplifying the network stack with Romana @romanaproject
Semantic and topological addressing
3
1
3
0
2
9
2
8
2
7
2
6
2
5
2
4
2
3
2
2
2
1
2
0
1
9
1
8
1
7
1
6
1
5
1
4
1
3
1
2
1
1
1
0
9 8 7 6 5 4 3 2 1 0
0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1
10
Network prefix bits
The network prefix.
In this example, we
are using the 10/8
address space.
6
Host ID Segment ID
We currently
store tenant ID in
upper bits of
segment ID.
4 67
Endpoint ID
Widths are configurable, don't have to use byte boundaries.
Encode the
tenant ID

34. romana.io Simplifying the network stack with Romana @romanaproject
Host BHost A
Allowing traffic within tenant
10.0.0.5 10.1.0.12
iptables:
check src/dst addrs
“tenant/segment bits
must match”
Src: 10.0.0.5
Dst: 10.1.0.12
Same
tenant/segment bits

35. romana.io Simplifying the network stack with Romana @romanaproject
Host BHost A
Isolating tenant traffic: Default
10.0.0.5 10.1.128.9
iptables:
check src/dst addrs
“tenant/segment bits
must match”
Src: 10.0.0.5
Dst: 10.1.128.9
Different
tenant/segment bits
Different
tenant

36. romana.io Simplifying the network stack with Romana @romanaproject
Host BHost A
Apply network policy between
segments (full isolation as default)
10.0.0.5 10.1.1.9
iptables:
Does policy chain
exist?
Otherwise: DROP
Src: 10.0.0.5
Dst: 10.1.1.9
Same tenant,
different segment
policy-chain:
From segment 0?
Protocol TCP?
To port 80?

37. Demo 1:
Kubernetes + Romana cluster
on top of Catalyst OpenStack cloud

38. romana.io Simplifying the network stack with Romana @romanaproject
Baking layered cakes
● Kubernetes on OpenStack? Why?
– On demand clusters
– Full tenant isolation
● Not all workloads fit into containers
– Seamless connection between pods and VMs
● Really nice with fully routed networking
– No double encapsulation
– Logical, efficient packet forwarding

39. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview

40. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Jump host with
public IP address

41. romana.io Simplifying the network stack with Romana @romanaproject

42. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo

43. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Install OpenStack
command line tools

44. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
$ neutron port-update
e925b70e-031e-4ef7-a27c-583b4b775290
--allowed-address-pairs type=dict list=true
mac_address=fa:16:3e:e1:df:59,ip_address=10.0.0.0/8

45. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
$ git clone https://github.com/romana/romana
$ cd romana/romana-install
$ ./romana-setup -p static -i my-inventory -s kubernetes install

46. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Romana
installer

47. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Kubernetes + Romana
Romana cluster
address range:
10/8

48. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - Overview
bar-1 bar-2foo
Kubernetes + Romana
Pods
with containers.
Pods have Romana
IP addresses.

49. romana.io Simplifying the network stack with Romana @romanaproject
Demo 1 - What you will see
● Creation of pods
● Network configuration
● Application of network policies

50. Demo 2:
Mixing containers with legacy workloads

51. romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - Overview
bar-1 bar-2foo
Kubernetes + Romana

52. romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - Overview
bar-1 bar-2foo
Kubernetes + Romana
vm-workload
Legacy application
in VM

53. romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - Overview
bar-1 bar-2foo
Kubernetes + Romana
vm-workload
Direct connection:
- No gateway
- No encap/decap
- No NAT

54. romana.io Simplifying the network stack with Romana @romanaproject
Demo 2 - What you will see
● Creation of pods
● Contact pod from VM
● See the packet route

55. Demo 3:
Romana + Kubernetes cluster
on top of Romana + OpenStack cluster

56. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
HW1 HW2 HW3 HW4

57. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
HW1 HW2 HW3 HW4
$ ./romana-setup -p static -i hw-inventory -s devstack install

58. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
HW1 HW2 HW3 HW4
OpenStack + Romana
Romana cluster 1
address range:
10/8

59. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
OpenStack VMs
VMs have
IP addresses
of
Romana cluster 1

60. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
$ ./romana-setup -p static -i vm-inventory -s kubernetes install

61. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3
Kubernetes + Romana
VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
Romana cluster 2
address range:
172.16/12

62. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - Overview
VM2 VM3
Kubernetes + Romana
VM1
HW1 HW2 HW3 HW4
OpenStack + Romana
Pods
with containers.
Pods have
IP addresses
of
Romana cluster 2

63. romana.io Simplifying the network stack with Romana @romanaproject
OpenStack + Romana
Kubernetes + Romana
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4

64. romana.io Simplifying the network stack with Romana @romanaproject
OpenStack + Romana
Kubernetes + Romana
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
Remember this one?
2 Top of Rack
Round Trips
East/West
Traffic
Per Instance
Security
Without pure L3 network
layered clusters
would be even more
complex.

65. romana.io Simplifying the network stack with Romana @romanaproject
OpenStack + Romana
Kubernetes + Romana
Demo 3 - Overview
VM2 VM3VM1
HW1 HW2 HW3 HW4
But with Romana, networking
even in layered clusters becomes
really easy...

66. romana.io Simplifying the network stack with Romana @romanaproject
Demo 3 - What you will see
● Creation of pods
● Pods and VMs with fully routable addresses
● Ease of use showcase: Trouble shooting

67. romana.io Simplifying the network stack with Romana @romanaproject
Conclusion
● Cloud native architectures simplify things
● Need cloud native networking to enjoy benefits
● Romana:
– Cloud native without compromises
– Native network performance
– Mostly static config: Solid network
– Very easy to work with and understand
● Easy to try:
– Simple installers for Kubernetes and OpenStack

68. romana.io Simplifying the network stack with Romana @romanaproject
Thank you!
● Romana Links
– http://romana.io - Project home
– http://romana.io/blog - Blog
– https://github.com/romana/romana - Sources
● Contact
– @romanaproject - Twitter
– info@romana.io - Email
– https://romana.slack.com/ - Slack channel