OpenStack deployments for public or private clouds require overlay networking. Due to the scale and rate of change of virtual resources, it isn't practical to rely on traditional network constructs and isolation mechanims. Today's deployments require performance, resilience, and high availability to be considered truly production-ready. In this session, we deep dive into the MidoNet architecture, and process of sending a data packet across an OpenStack environment through a network overlay. A distributed architecture implements logical constructs that are used to build networks without a single point of failure, all while adding network functionality in a highly-scalable manner. Network functions are applied in a single virtual hop. By applying network services right at the ingress host, the network is free from unnecessary clogging and bottlenecks by avoiding additional hops. Packets reach their destination more efficiently with the single virtual hop. After this session, the audience will understand how distributed architectures allow efficient networking with routing decisions and network services applied at the edge. Also, the audience will understand how it is easier to scale clouds when the network intelligence is distributed.

1. Cloud Networking
OpenStack Day Korea
February 5th, 2015

2. Agenda
 What is Driving Network Change
 Cloud Network Requirements
 Why Not Traditional Networking
 Network Virtualization Overlays
 Neutron?
 MidoNet
1

3. Forces are Reshaping Networking…
Big Web Cloud
Computing
Big
Data
Customer Focus – $ / Node & Port
Azure
Mobile
2

4. IoT and Big
Data
Networking is Experiencing Rapid Change
Services and applications are
moving to the Cloud; workloads
are moving to a virtualization
environment; DevOps
networking adoption
Hardware is commoditized; many
players delivering high-throughput
switching at extremely low prices
Open Source and Service Orientation supports flexibility,
innovation, vendor agnostic design, self-service, shorter
development times and faster time to market
Cloud
Computing
White-box
Hardware
IoT and Big Data impact networks requiring
distributed datacenters and agility to enable
real-time event responses
Open
Source and
Service
Orientation

5. 4
Cloud Networking Requirements

6. Network Virtualization Requirements
•Speed of Provisioning
•Scale
•Multi-tenancy
•Performance
•Elasticity
•Simplicity of Deployment
•Security

7. Requirements for NV
6
Requirements
6
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual
Router (L3)
Tenant A
Virtual Router
Tenant B
Virtual Router
VM6
Virtual L2
Switch B1
Virtual L2
Switch A1
Virtual L2
Switch A2
TenantB office
Tenant B
VPN Router
Office
Network

8. Requirements for NV
7
Requirements
7
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual
Router (L3)
Tenant A
Virtual Router
Tenant B
Virtual Router
VM6
Virtual L2
Switch B1
Virtual L2
Switch A1
Virtual L2
Switch A2
TenantB office
Tenant B
VPN Router
Office
Network
Isolated tenant
networks
(virtual data center)

9. Requirements for NV
8
Requirements
8
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual
Router (L3)
Tenant A
Virtual Router
Tenant B
Virtual Router
VM6
Virtual L2
Switch B1
Virtual L2
Switch A1
Virtual L2
Switch A2
TenantB office
Tenant B
VPN Router
Office
Network
L3 Isolation
(similar to VPC and VRF)

10. Requirements for NV
9
Requirements
9
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual
Router (L3)
Tenant A
Virtual Router
Tenant B
Virtual Router
VM6
Virtual L2
Switch B1
Virtual L2
Switch A1
Virtual L2
Switch A2
TenantB office
Tenant B
VPN Router
Office
Network
Fault-tolerant devices and links
Redundant, optimized, and
fault tolerant paths to
to/from external networks
(e.g. via eBGP)

11. Requirements for NV
10
10
Tenant/Project A
Network A1
VM1 VM3
Network A2
VM5
Tenant/Project B
Network B1
VM2 VM4
uplink
Provider Virtual
Router (L3)
Tenant A
Virtual Router
Tenant B
Virtual Router
VM6
Virtual L2
Switch B1
Virtual L2
Switch A1
Virtual L2
Switch A2
TenantB office
Tenant B
VPN Router
Office
Network
Fault-tolerant devices and links
Fault tolerant
devices and links

12. Requirements for NV
11
Device-agnostic networking services:
• Load Balancing
• Firewalls
• Stateful NAT
• VPN
Networks and services must be fault
tolerant and scalable

13. Requirements for NV
12
Single pane of glass to manage it all.

14. Bonus Requirements for NV
13
Integration with cloud or
virtualization management
systems.
Optimize network by exploiting
management configuration.
Single virtual hop for networking
services
Fully distributed control plane
(ARP, DHCP, ICMP)

15. Checklist for Network Virtualization
14
 Multi-tenancy
 Scalable, fault-tolerant devices
(or device-agnostic network
services).
 L2 isolation
 L3 routing isolation
• VPC
• Like VRF (virtual routing
and fwd-ing)
 Scalable gateways
 Scalable control plane
• ARP, DHCP, ICMP
 Floating/Elastic Ips
 Stateful NAT
• Port masquerading
• DNAT
 ACLs
 Stateful (L4) Firewalls
• Security Groups
 Load Balancing with health checks
 Single Pane of Glass (API, CLI, GUI)
 Integration with management platforms
• OpenStack, CloudStack
• vSphere, RHEV, System Center
 Decoupled from Physical Network

16. Why Traditional Networking Doesn’t Work
•For example
•VLANs for L2 isolation
•VRFs for L3 isolation
•Not Designed For Speedy Provisioning
•Not Designed For Scale
•Consider virtual endpoints
•Not Designed For Multi-tenancy
•Services are not elastic
15

17. 16
Network Virtualization Overlays

18. 17
Encapsulation and Tunneling
Provides isolation

19. 18
Stateless core. Stateful edge.

20. Clos Fabric
19
From Cumulus Networks

21. 20
Network processing at the edge
Decoupled from the physical network

22. 21
Virtual network changes don’t affect
the physical network

23. 22
Single virtual hop network services
avoid “traffic trombones”

24. 23
Centralized state and control for
maximum agility

25. 24
Scalable, fault tolerant gateways to
external networks

26. Using NV Overlays for Cloud Network
25
 Multi-tenancy
 Scalable, fault-tolerant devices
(or device-agnostic network
services).
 L2 isolation
 L3 routing isolation
• VPC
• Like VRF (virtual routing
and fwd-ing)
 Scalable Gateways
 Scalable control plane
• ARP, DHCP, ICMP
 Floating/Elastic IPs
 Stateful NAT
• Port masquerading
• DNAT
 ACLs
 Stateful (L4) Firewalls
• Security Groups
 Load Balancing with health checks
 Single Pane of Glass (API, CLI, GUI)
 Integration with management platforms
• OpenStack, CloudStack
• vSphere, RHEV, Docker
 Decoupled from Physical Network

27. Network Virtualization Overlays Today
26

28. 27
Can’t I just use Neutron?

29. Neutron
•Default Implementation Is Not Scalable
•L4 services (NAT) are still bottlenecks
•Using namespaces
•Agents have serious fault tolerance issues
•DHCP, MetaData, DNS
•Fundamentally hard to fix
28

30. 29
MidoNet

31. 30
MidoNet Network Virtualization Platform
Logical L2 Switching - L2 isolation and path optimization with distributed
virtual switching
Interconnect with VLAN enabled network via L2 Gateway
Logical L3 Routing – L3 isolation and routing between virtual networks
No need to exit the software container - no hardware required
Distributed Firewall – Provides ACLs, high performance kernel integrated
firewall via a flexible rule chain system
Logical Layer 4 Load Balancer – Provides application load balancing in
software form - no need for hardware based firewalls
VxLAN/GRE – Provides VxLAN and GRE tunneling
Provides L2 connectivity across L3 transport. This is useful when L2 fabric
doesn’t reach all the way from the racks hosting the VMs to the physical L2
segment of interest.
MidoNet/Neutron API– Alignment with OpenStack Neutron’s API for
integration into compatible cloud management software
v
Any Application
MidoNet Network Virtualization Platform
Any Network Hardware
OpenStack/Cloud Management System
Distributed
Firewall
Layer 4
Load Balancer
VxLAN/GRE
Any Hypervisor
Logical L2 Logical L3 NAT
MidoNe
t/
Neutron
API
NAT – Provides Dynamic NAT, Port masquerading

32. MidoNet
31
Logical Topology
MidoNet Solution
1
Private IP
Network
MN
MN
MN
Internet
BGP
Multi
Homing
Physical Topology
MN
VM
VM
MN
VM
VM
MN
VM
VM
BGP
To ISP3
BGP
To ISP2
BGP
To ISP1
vPort
Provider
Virtual
Router
Tenant A
Virtual
Router
Tenant B
Virtual
Router
Virtual
Switch A1
Virtual
Switch A2
Virtual
Switch B1
vPort
vPort
vPort
vPort
vPort
Network State Database
MN MN MN
Tunnel

33. Architecture Overview

34. 33
MidoNet Flow Processing

35. Flow Processing at the Edge
•Ingress Simulation
•State Propagation
•Tunneling
•Egress
34

36. 35
MidoNet
Gateway
YourExisting
Infrastructure
Provider
Router
Tenant
Router
Tenant
Network
192.168.5.2 192.168.5.3
Subnet
192.168.5.0/24
Address: 192.168.5.1
Allow incoming tcp/22
NAT 192.168.5.2 <->
112.140.32.94
VM to VM Communication

37. MidoNet
GatewayYourExisting
Infrastructure
Now MidoNet can create a VXLAN tunnel between the
required nodes, and send the packet on its way
36
VXLAN Tunnel

38. 37
Under the Hood

39. Distributed StateOn-demand
state
propagation
Virtual Networking at the Edge
Leverage ZK
RPC over TCP

40. Distributed State
- VM sends first packet
- Kernel flow miss occurs; queues packet for
processing via Netlink
- MidoNet receives Netlink message for processing
Virtual Networking at the Edge
user space
kernel space

41. Distributed State
Virtual Networking at the Edge
user space
kernel space
MidoNet agent may query the
NSDB; then
- Locally processes packet
(virtual layer simulation)
- Installs local flow (drop/mod/fwd)

42. Virtual Networking at the Edge
user space
kernel space
Possible actions on flow table entry match:
- Set src/dst MAC to routerMAC/dstMAC
- Modify TTL
- Encapsulation with GRE or VXLAN + IP.
Key or ID tells dest host the destination vPort.

43. Virtual Networking at the Edge
Packet is delivered with overlay networking.
Destination host owns vport, identified by the
GRE key or VxLAN VNI.

44. Control Protocol Handling
•Agent traps ARP, DHCP, MetaData
•Locally Reply
•DNS coming soon
43

45. 44
Bridge physical and virtual networks
more efficiently

46. MidoNet VTEP Gateway
45

47. MidoNet VTEP Gateway
46

48. 47
Break through performance barriers
of software networking

49. 40Gb VxLAN Offloading: virtualized environments require high
throughput infrastructure
• Integration with Mellanox provides 40 Gbps
saturation
• VxLAN offloading improves CPU utilization levels
• Scale with performance through HW interconnect
• Increase throughput with offloading where no
offloading would otherwise have flat results
• High bandwidth can now be achieved in software
Performance

50. OpenStack Integration
5
Easy integration with OpenStack:
MidoNet provides a plugin for Neutron.
MidoNet Plugin

51. Open Source
•MidoNet was Open Sourced in November 2014
•www.midonet.org
•www.github.com/midonet/
•OpenStack and Docker need a high quality Open
Source NVO solution!
50

52. 51
What’s Next?

53. Network Operating System
•Linux is everywhere
•ONIE & Cumulus Linux
•We can run our software on it!
•Fabric Monitoring and Control
•Resource Monitoring
•Traffic Engineering
•ECMP enhancement
52

54. 53
Get more out of the physical network.

55. Cannot ignore the physical network
54
Dynamic changes to logical
network are not dependent on the
physical network configuration.
Sharing state to and from the
physical network can be
supplementary.
- Monitoring
- Traffic Engineering

56. 55
Get more intelligence out of your network

57. Big Data
56
NOS centralizes information on
your network
We can start taking advantage of
this information
- Security
- Compliance
- Optimizing Networks

58. 57
It’s Open Source
http://www.midonet.org
Check out our blog:
http://blog.midonet.org
Follow us on Twitter:
@midonet

59. 58
Thank You

60. 59
Distributed Flow State

61. Distributed Flow-State
60
• MidoNet’s distributed architecture enables stateful
network functions at the edge
• Given the forward and return flows could have several
ingress and egress nodes, “interested sets” get hints
• Advantages include:
• Lower latency to process flows
• Independence from a centralized transaction, like a
database query

62. Distributed Flow-State
61
• For a new ingress flow, perform
flow computation when flow
state is created and store locally
• Prior to packet forwarding, the
ingress node determines the
interested set and then pushes
the flow state

63. Distributed Flow-State
62
• Flow state is leveraged by flow computation and tunnel
encapsulation
• Flow states are pushed between agents using Tunnel packets with
special tunnel key values indicating “flow state”

64. Distributed Flow-State
63
• “Fire and forget” flow state propagation allows the “interested set”
nodes to be informed without packet delay
• Asymmetrical data flow paths are easily handled given the flow
state is propagated to the “interested set” of nodes

65. Stateful port groups
64
• Create port-group for the stateful ingress port group
midonet-cli> port-group create name SPG stateful true
• Add the ports to be load balanced e.g. all uplinks on Provider Router
midonet> port-group pgroup0 add member port router0:port0
midonet> port-group pgroup0 add member port router0:port1