Slides from the OpenStack Ops Meetup on June 7, 2016.

1. Romana Project
Network and Security Automation
romana.ioJune 2016
OpenStack
Operators Meetup
June 7, 2016

2. romana.io
New Networks, New Problems, New Solutions
• Legacy Apps/Enterprise Private
Cloud
• LAN Emulation to support vMotion
• Automated data center
infrastructure provisioning
• Cloud Native Apps
• Seamless public/private cloud
deployment and orchestration
• Docker and Container networking
• Endpoint explosion and compressed
lifecycle
• Whitebox and GIFEE Networks
• Enterprise SDN
• VMware/NSX
• Cisco ACI
• Others…
• Cloud Native Networks
• Network automation for
rapid provisioning
• Security automation
• Multi-cloud
romana.io
June 2016 Slide 1

3. Cloud Native vs. Enterprise Networks
• Amazon AWS Style v. Enterprise Apps
• Service orientation (Cattle) v. Endpoint orientation (Pets)
• Network requirements
• Reachable IP addresses v. Auto discovered MAC (ARP on VLANs)
• Service orientation further decouples apps from infrastructure
• No VM migration
• No IP Failover
• Good News: Cloud Native apps don’t need layer 2 networks
• Avoiding Layer 2 networks eliminates a lot of SDN complexity
• Bad News: Layer 2 networks provided a convenient way to isolate apps
• Even a small number of VLANs were difficult to automate
Bottom Line: Need a new way to isolate networks
romana.ioJune 2016 Slide 2

4. Romana Network and Security Automation
• Layer 3 based isolation and tenancy model
• Topology-aware addressing
• Embed tenant and segment IDs in IP addresses
• Requires nothing more than standard L3 routing
• Hierarchical design simplifies scalable deployment
• No virtual network required
• Native performance and visibility
• Eliminates overlays
• Routes map to services 1:1
• Simplifies composition, security and control
• Tightly integrated into Cloud Management/Orchestration IPAM
romana.ioJune 2016 Slide 3

5. SDN Complexity melts away
• No VLANs, VXLANs, VTEP/VNID, OpenFlow, OVS/OVN/OVSDB
• Route aggregation simplifies network
• Static routing eliminates need for route distribution (BGP, XMPP, KVS)
• Reduces the number of firewall rules (i.e. network v. endpoint)
• Simplifies Operations
• Existing tools, techniques and diagnostics all just work
• Existing security, policy and control systems all work
• Firewalls, IDS, LB, etc., etc., etc.
June 2016 romana.io Slide 4

6. North/South Traffic
• Neutron Network node
routes traffic between
segments
• Network node
performs all
L3 functions
• East/West traffic
encapsulated, but is direct
to destination host
romana.io
VXLAN Decap
VXLAN Decap
VXLAN Encap
VXLAN Encap
2 Top of Rack
Round Trips
East/West
Traffic
Per Instance
Security
June 2016 Slide 5

7. North/South Traffic
• Latency dramatically
reduced
• No Network node
• No encap
• Identical path for
East/West traffic
romana.io
Eliminated
Bypassed
Bypassed
Romana
Router
Romana
Router
1 Top of Rack
Round Trip
Per Network
Security
June 2016 Slide 6

8. Network Latency
• North/South Latency reduced 50%-85%
• 10% improvement for East/West traffic between hosts (no encap)
• No performance penalty for local on-host East/West traffic
romana.io
North/South
(Routed)
East/West
(Switched)
Time (ms) Local Remote Local Remote
Native OpenStack 1.51* 1.51 0.24 0.85
Romana Networks 0.24 0.77 0.24** 0.77**
Relative Performance Local Remote Local Remote
Native OpenStack 100% 100% 100% 100%
Romana Networks 16% 51% 100% 90%
* All N/S OpenStack traffic
goes off host
** All Romana traffic is
routed
June 2016 Slide 7

9. How does it work?
• Assign CIDR length for host (node), tenant and segment
• Example: host 16, tenant 20, segment 24
• On every host, each tenant gets a real physical CIDR
• Tenant can further sub-net for their own private segments
• Assign IP addresses that maintain reachability
• Apply layer 3 firewall rules for network isolation
• Configure next hop gateway for service composition
June 2016 romana.io Slide 8

10. Example
June 2016 romana.io Slide 9
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Field
Capacity 0 0 0 0 1 0 1 0
Example: Bits Length Purpose
10/8 Network 8 10/8 Network
Hosts 8 Up to 255 Hosts
Tenants 4 Up to 255 Tenants
Segments 4 Up to 16 Segments per Tenant
Endpoints 8 Up to 16 Endpoints per Segment
Host 1 ID CIDR or IP Host 2 ID CIDR or IP Host 3 ID CIDR or IP
Physical Addr 192.168.0.10 Physical Addr 192.168.0.11 Physical Addr 192.168.0.12
Host 1 10.1/16 Host 2 10.2/16 Host 3 10.3/16
Tenant 0 10.1.0/20 Tenant 0 10.2.0/20 Tenant 0 10.3.0/20
Segment 1 10.1.1/24 Segment 1 10.2.1/24 Segment 2 10.3.2/24
VM 1 22 VM 1 22 VM 1 22
VM 2 33 VM 2 33 VM 2 33
Tenant 1 10.1.16/20 Tenant 1 10.2.16/20 Tenant 1 10.3.16/20
Segment 1 10.1.17/24 Segment 2 10.2.18/24 Segment 1 10.3.17/24
VM 3 44 VM 3 44 VM 3 44
VM 4 55 VM 4 55 VM 4 55
Endpoint ID
Up to 255 Hosts Up to 255 Tenants 255 Endpoints for each Tenant
20 17-20
10/8 Net Mask Host ID Bits (8) Tenant/Segment ID Bits (8)
Location
8 1-8
16 9-16
24 21-24
32 25-32
10.1.1.22
10.1.17.55 10.2.18.55 10.3.17.55
10.3.2.22
10.1.1.33 10.2.1.33 10.3.2.33
10.1.17.44 10.2.18.44 10.3.17.44
10.2.1.22

11. Physical Deployment
June 2016 romana.io Slide 10
192.168.0.10 192.168.0.11 192.168.0.12
Host 1
VM 1: 10.1.1.22
G/W: 10.1.0.1/16
VM 2: 10.1.1.33
VM 3: 10.1.17.44
VM 4: 10.1.17.55
10.2/16 -> 192.168.0.11
10.3/16 -> 192.168.0.12
Host 2
VM 1: 10.2.1.22
G/W: 10.2.0.1/16
VM 2: 10.2.1.33
VM 3: 10.2.18.44
VM 4: 10.2.18.55
10.1/16 -> 192.168.0.10
10.3/16 -> 192.168.0.12
Host 3
VM 1: 10.3.2.22
G/W: 10.3.0.1/16
VM 2: 10.3.2.33
VM 3: 10.3.17.44
VM 4: 10.3.17.55
10.1/16 -> 192.168.0.10
10.2/16 -> 192.168.0.11

12. ECMP
BGP/OSPF Area
Leaf 1
Every host gets /16 network, announces to Leaf
Leaf aggregates 64 /16 networks, announces /10 to Spine
Spine contains only four /10 networks
0.0.0.0 via Spine 1
10.0.0.0/16 via Port 1
10.1.0.0/16 via Port 2
10.2.0.0/16 via Port 3
10.3.0.0/16 via Port 4
…
10.63.0.0/16 via Port 64
Spine 1
0.0.0.0 via Internet
10.0.0.0/10 via Leaf 1
10.64.0.0/10 via Leaf 2
10.128.0.0/10 via Leaf 3
10.192.0.0/10 via Leaf 4
Spine 2
Leaf 2 Leaf 3 Leaf 4
0.0.0.0 via Internet
10.0.0.0/10 via Leaf 1
10.64.0.0/10 via Leaf 2
10.128.0.0/10 via Leaf 3
10.192.0.0/10 via Leaf 4
10.2/16 RIP to Leaf for distribution
0.0.0.0 via Leaf 1, Port 8
Port 8
Host 221
10.194.3.71
0.0.0.0 via Leaf 4, Port 3
Port 3
Host 8
10.2.16.34
0.0.0.0 via Spine 1
10.192.0.0/16 via Port 1
10.193.0.0/16 via Port 2
10.194.0.0/16 via Port 3
10.195.0.0/16 via Port 4
…
10.255.0.0/16 via Port 64
romana.ioJune 2016 Slide 11
Endpoints on Host 8 must get address within 10.2.0.0/16
Endpoints on Host 221 must get address within 10.194.0.0/16
Announce route to ToR

13. Leaf 1
Spine 1 Spine 2
Leaf 2 Leaf 3 Leaf 4
10.2/16 RIP to Leaf for distribution
172.16.1.25 host route
0.0.0.0 via Leaf 1, Port 8
Host
10.194.3.71
0.0.0.0 via Leaf 4, Port 3Host 8
10.2.16.34
Edge/
NAT
Host routes to external service endpoints
June 2016 romana.io Slide 12
SLB
VM
SLB get FIP as VIP
FIP 172.16.1.25

14. Security
Policy
Neutron Node
OpenStack Deployment
May 2016 romana.io
IPAM
Routes
Tenant
DB
Topology
Policy
Slide 13
Neutron
ML2IPAM
Compute Node n
VM
iptables
VM
Nova
Agent

15. Network/Security Policy
NetPolicy.json
{
"Name": "policy2",
"PolicyID": "CF2D2BE2-4553-4C28-BD02-140CF83617A2", # unique identifier across tenants, auto generated for POST.
"AppliedTo": [ # can attach multiple tenants to which the policy can be applied to.
{
"Tenant":"tenant2",
"Segment": "Segment1",
“HostCIDR": “10.23.0.0/0", # Apply policy to entire host
},
],
"Tags": [], # meta data attached to policies for various external environments like openstack/kubernetes
"Direction" : "Ingress", # can be Egress or Ingress.
"Peers": [
{
"CidrBlock": "0.0.0.0/0", # IP from L3 header
},
],
"Rules": [{
"Protocol": "ICMP",
"IcmpTypeCode": [0,8],
"IsStateful": true,
},],
"Description": "hello there, security policies are fun!",
}
June 2016 romana.io Slide 14

16. Scalable Deployments
• Need more IP addresses
• Large OpenStack environments
• Container endpoint explosion
• Separate Romana deployment for each OpenStack cluster
• Clusters interact via service endpoints
• Explicitly manage overlapping IPs
• Use datacenter FIPs
• Support Overlapping in Romana IPAM
• Advantage of consistent policy across environment
• IPv6
June 2016 romana.io Slide 15

17. Cluster 2Cluster 1
Romana 1: 10/8
Shared Block: 10.0.1/24
Local FIPs: 10.0.1.128/25
Remote FIPs: 10.0.1.0/25
Edge
Large Scale Deployments
June 2016 romana.io Slide 16
Romana 2: 10/8
Shared Block: 10.0.1/24
Local FIPs: 10.0.1.0/25
Remote FIPs: 10.0.1.128/25
Alternatively use FIPs from
DC addresses
Shared 172.16.1/24
FIPs

18. Security
Policy
k8s Master
Kubernetes Deployment
May 2016 romana.io
IPAM
Routes
Tenant
DB
Topology
Policy
Slide 17
Minion
Pod
iptables
Pod
Agent
Controllers
Scheduler
API
etcd
Pod/Service
Definition
CNI
Listener

19. Nested Container Networking
June 2016 romana.io Slide 18
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Field
Capacity 0 0 0 0 1 0 1 0
Example: Bits Length Purpose
10.0 Network 8 Full Network (10/8)
Hosts 8 Up to 255 Hosts
Tenants 4 Up to 16 Tenants
Segments 4 Up to 16 Segments per Tenant
Endpoints 8 Up to 255 Endpoints per Segment
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Field Host ID Bits (4)
Capacity 1 0 1 0 1 1 0 0 0 0 0 1 Up to 16 Hosts
Example: Bits Length Purpose
172.16 Network 12 Full Network (172.16/12)
Hosts 4 Up to 16 Hosts
Tenants 4 Up to 16 Tenants
Segments 4 Up to 16 Segments per Tenant
Endpoints 8 Up to 255 Endpoints per Segment
Endpoint ID
Up to 255 Hosts Up to 255 Tenant/Segments 255 Endpoints
Tenant and Segment ID Bits (8) Endpoint ID
Up to 255 Tenant/Segments 255 Endpoints
Location
12 1-12
16
20 17-20
10/8 Net Mask Host ID Bits (8) Tenant and Segment ID Bits (8)
Location
8 1-8
16 9-16
24 21-24
32 25-32
13-16
20 17-20
24 21-24
32 25-32
172.16/12 Net Mask

20. Nested Containers
June 2016 romana.io
192.168.0.10 192.168.0.11 192.168.0.12
Slide 19
Host 1
VM 1: 10.1.1.22
G/W: 10.1.0.1/16
10.2/16 -> 192.168.0.11
10.3/16 -> 192.168.0.12
172.17/16-> 192.168.0.11
172.18/16 -> 192.168.0.12
Pod 172.16.1.8
Pod 172.16.2.9
GW 172.16.0.1/16
172.17/16 -> 10.2.0.1
172.18/16 -> 10.3.0.1
Host 2
VM 1: 10.2.1.22
G/W: 10.2.0.1/16
Pod 172.17.6.8
Pod 172.17.2.11
GW 172.17.0.1/16
172.18/16 -> 10.3.0.1
172.16.16 -> 10.1.0.1
Host 3
VM 1: 10.3.1.22
G/W: 10.3.0.1/16
Pod 172.18.3.8
Pod 172.18.4.9
GW 172.18.0.1/16
172.16/16 -> 10.1.0.1
172.17/16 -> 10.2.0.1
10.1/16 -> 192.168.0.10
10.3/16 -> 192.168.0.12
172.16/16 -> 192.168.0.10
172.18/16 -> 192.168.0.12
10.1/16 -> 192.168.0.10
10.2/16 -> 192.168.0.11
172.16/16 -> 192.168.0.10
172.17/16-> 192.168.0.11

21. Ubernetes
June 2016 romana.io
192.168.0.10 192.168.0.11 192.168.0.12
Slide 20
Host 1
VM 1: 10.1.1.22
G/W: 10.1.0.1/16
10.2/16 -> 192.168.0.11
10.3/16 -> 192.168.0.12
172.17/16-> 192.168.0.11
172.18/16 -> 192.168.0.12
Pod 172.16.1.8
Pod 172.16.2.9
GW 172.16.0.1/16
172.17/16 -> 10.2.0.1
172.18/16 -> 10.3.0.1
Host 2
VM 1: 10.2.1.22
G/W: 10.2.0.1/16
Pod 172.17.6.8
Pod 172.17.2.11
GW 172.17.0.1/16
172.18/16 -> 10.3.0.1
172.16.16 -> 10.1.0.1
Host 3
VM 1: 10.3.1.22
G/W: 10.3.0.1/16
Pod 172.18.3.8
Pod 172.18.4.9
GW 172.18.0.1/16
172.16/16 -> 10.1.0.1
172.17/16 -> 10.2.0.1
10.1/16 -> 192.168.0.10
10.3/16 -> 192.168.0.12
172.16/16 -> 192.168.0.10
172.18/16 -> 192.168.0.12
10.1/16 -> 192.168.0.10
10.2/16 -> 192.168.0.11
172.16/16 -> 192.168.0.10
172.17/16-> 192.168.0.11
WAN

22. Networks Define Services
• Tenant ID + Segment ID become a Network ID
• Natural fit for micro- and shared platform
services
• Route control to/from micro services enable
transparent service insertion/chainingand policy
enforcement
• Local/remote/hybrid cloud deployments
romana.io
IP
Int
IP
Int
IP
Int
IP
Int
L/B
Microservice
Endpoint
F/W
Shared Services
June 2016 Slide 21

23. Romana Project
• Cloud Native network and security automation
• All details available at romana.io
• Open source
• Apache 2.0
• Written in Go
• www.github.com/romana
• OpenStack and Kubernetes integration
• Release v0.9 available now
romana.ioJune 2016 Slide 22

24. Demo
• OpenStack on four physical machines
• Launch VMs on private 10/8 network
• Kubernetes running on VMs
• Kubernetes Network 172.16/12
• Container Network Interface (CNI) configuration of pods
• Romana IPAM allocates IPs for VMs and pods
• Chosen specially to maintain static routes and CIDRs to each host
and VM
• All IPs reachable by construction
June 2016 romana.io Slide 23