These are the slides to the deep dive follow on from the incredible data center design show we did with Luikas. We have him back on to cover: Introduction to Overlays, the VXLAN Evolution Beyond Just Encapsulations, Optimized Networking with VXLAN EVPN, Use Cases for VXLAN, Understanding Scalable Layer 2 Domains, Integrated Route and Bridge, Multitenancy Uses.
Go back and watch the video we did at techwisetv.com for even more.
And if you are really serious...check out what Lukas has done with Cisco Press: http://www.ciscopress.com/store/cisco-programmable-fabric-using-vxlan-with-bgp-evpn-9780134272290
4. Overlay Based Data Center Fabrics
Desirable Attributes:
• Mobility
• Segmentation
• Scale
• Automated & Programmable
• Abstracted consumption models
• Full Cross Sectional Bandwidth
• Layer-2 + Layer-3 Connectivity
• Physical + Virtual
RR RR
5. Overlay Based Data Center: Edge Devices
Network Overlays Hybrid OverlaysHost Overlays
• Virtual end-points only
• Single admin domain
• VXLAN, NVGRE, STT
• Physical and Virtual
• Resiliency + Scale
• X-Organizations/Federation
• Open Standards
• Router/Switch end-points
• Protocols for Resiliency/Loops
• Traditional VPNs
• VXLAN, OTV, VPLS, LISP, FP
V
V
V
V
V
V
FYI
6. Overlay Services
• Layer 2
• Layer 3
• Layer 2 and Layer 3
Tunnel Encapsulation
Underlay Transport
Network
Control Plane
• Peer Discovery mechanism
• Route Learning and Distribution
– Local Learning
– Remote Learning
Data Plane
• Overlay Layer 2/Layer 3 Unicast traffic
• Overlay Broadcast, Unknown Unicast,
Multicast traffic (BUM traffic) forwarding
– Ingress Replication
– Multicast
Understanding Overlay Technologies
11. • Standards based Overlay (VXLAN) with Standards
based Control-Plane (BGP)
• Layer-2 MAC and Layer-3 IP information distribution by
Control-Plane (BGP)
• Forwarding decision based on Control-Plane
(minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized
Forwarding in the Overlay
• Multi-Tenancy At Scale
What is VXLAN with BGP EVPN?
12. Control-
Plane
EVPN MP-BGP - RFC 7432
(draft-ietf-l2vpn-evpn)
Data-
Plane
Multi-Protocol Label Switching
(MPLS)
draft-ietf-l2vpn-evpn
Provider Backbone Bridges
(PBB)
draft-ietf-l2vpn-pbb-evpn
Network Virtualization Overlay
(NVO)
draft-ietf-bess-evpn-overlay
EVPN over NVO Tunnels (e.g. VXLAN) for Data Center
Fabric encapsulations
Provides Layer 2 and Layer 3 Overlays over simple IP
Networks
Ethernet VPN – EVPN
13. ID Title Category
RFC 7348
Virtual eXtensible Local Area Network Data Plane
RFC 7432
BGP MPLS based Ethernet VPNs EVPN Control Plane
draft-ietf-bess-evpn-overlay
A Network Virtualization Overlay Solution using EVPN EVPN Control Plane
draft-ietf-bess-evpn-inter-subnet-forwarding
Integrated Routing and Bridging in EVPN EVPN Control Plane
draft-ietf-bess-evpn-prefix-advertisement
IP Prefix Advertisement in E-VPN EVPN Control Plane
Draft-tissa-nvo3-oam-fm
NVO3 Fault Management Mgmt Plane (OAM)
IETF RFC & Drafts – Implemented by Cisco
FYI
14. VXLAN Evolution with BGP EVPN
Protocol Learning
• Uses MP-BGP with EVPN
Address family
• Workload MAC and IP
Addresses learnt by VXLAN
Edge Devices (NVEs)
• Advertises Layer-2 and Layer-
3 Address-to-VTEP
Association
• Flood Prevention
• Optimized ARP forwarding
IP Services
• VXLAN Routing
• Distributed Anycast Gateway
Multi-Tenancy
• Route Reflector for Scale
External Connectivity
• VXLAN Hardware Gateway
Redundancy (VPC)
• Integrated physical and
virtual Overlays (Hybrid
Overlays)
• Inter-Pod Connectivity
• VXLAN Gateway to other
Encaps/Networks
Multicast Independent*
• Overlay Control-Plane
provides dynamic VTEP
discovery
• Head-End Replication
enables Unicast-only mode
(aka ingress Replication)
*Multicast Independence requires the usage of the Overlay Control-Plane or static configuration
15. Overlay with Optimized Routing
Spine
RR RR
V
V
V
V
V
V
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
RR
Route-Reflectors deployed
for scaling purposes (iBGP)
BGP Adjacencies
Border
16. Overlay with Optimized Routing
Spine
RR RR
V
V
V
V
V
V
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
RR
Route-Reflectors deployed
for scaling purposes (iBGP)
BGP Adjacencies
Border
Scalable Multi-Tenancy with Multiprotocol BGP
EVPN Address-Family: Host MAC+IP, internal/external IP Subnets
BGP enhanced for Fast Convergence at Large Scale
Extensions for Fast and Seamless Host Mobility
Distributed Gateway with Traffic Flow Symmetry
ARP Suppression
17. Distributed IP Anycast Gateway
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
route
18. Distributed IP Anycast Gateway
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
route
Any Subnet Routed Anywhere – Any VTEP can serve any Subnet
Integrated Route & Bridge (IRB) - Route whenever you can, Bridge when needed
No Hairpinning – Optimized East/West and North/South Routing
Seamless Mobility - All Leaf share same Gateway MAC
Reduced Failure Domain – Layer-2/Layer-3 Boundary at Leaf
Optimal Scalability – Route Distributed & closest to the Host
19. Multi-Destination Traffic
Ingress Replication
Unicast based Replication
• Source VTEP sends unicast copy to every
Destination VTEP
• Requires only Unicast
• Simples way of Traffic Replication
Multicast
Multicast based Replication
• Source VTEP sends single copy, Multicast
replicates it as needed
• Leverages Multicast Routing (PIM)
• Most optimal way of Traffic Replication
Spine
RR RR
V
V
V
V
V
V
Spine
RR RR
V
V
V
V
V
V
21. • A mode of operation, where multiple independent instances (tenant)
operate in a shared environment.
• Each instance (i.e. VRF/VLAN) is logically isolated, but physically
integrated.
What is Multi-Tenancy
22. Multi-Tenancy at Layer-2
• Per-Switch VLAN-to-VNI
mapping
• Per-Port VLAN Significance
Multi-Tenancy at Layer-3
• VRF-to-VNI mapping
• MP-BGP for scaling with VPNs
Where can we apply Multi-Tenancy
23. Layer-2 Multi-Tenancy
Spine
RR RR
V
V
V
V
V
V
VLAN 100
VLAN 100
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
bridge
25. Layer-2 Multi-Tenancy – Bridge Domains
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Leaf
VV
VLAN 100 VLAN 100
VXLAN Overlay (VNI 30001)
Bridge Domain
The Bridge Domain is the Layer-2 Segment from Host to Host
In VXLAN, the Bridge Domain consists of three Components
1) The Ethernet Segment (VLAN), between Host and Switch
2) The Hardware Resources (Bridge Domain) within the Switch
3) The VXLAN Segment (VNI) between Switch and Switch
32. Leaf#1
bridge-domain 100
member vni 30001
encapsulation profile vni VLAN100-30001
dot1q 100 vni 30001
encapsulation profile vni VLAN200-30001
dot1q 200 vni 30001
CLI Modes - Bridge-Domain based (per-Port)
interface Ethernet 1/8
no switchport
service instance 1 vni
encapsulation profile VLAN100-30001 default
interface Ethernet 1/9
no switchport
service instance 1 vni
encapsulation profile VLAN200-30001 default
FYI
33. Layer-3 Multi-Tenancy
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
VRF-A (VNI 50001)
VRF-B (VNI 50002)
SVI 100, Gateway IP: 192.168.1.1 (VRF-A)
SVI 200, Gateway IP: 10.10.10.1 (VRF-B)
SVI 300, Gateway IP: 172.16.1.1 (VRF-B)
Host1
IP: 192.168.1.11 (VRF-A)
VLAN 100
Host3
IP: 172.16.1.33 (VRF-B)
VLAN 300
Host2
IP: 10.10.10.22 (VRF-B)
VLAN 200
SVI 300
route
route
34. Layer-3 Multi-Tenancy – VRF-VNI or L3VNI
Host1
IP: 192.168.1.11 (VRF-A)
VLAN 100
Host3
IP: 172.16.1.33 (VRF-B)
VLAN 300
Leaf
VV
SVI 100
V
Host2
IP: 10.10.10.22 (VRF-B)
VLAN 200
SVI 200 SVI 300
VRF-A
(VNI 50001)
VRF-B
(VNI 50002)
Routing
Domain
VRF-B
Routing
Domain
VRF-A
35. Layer-3 Multi-Tenancy – VRF-VNI or L3VNI
Host1
IP: 192.168.1.11 (VRF-A)
VLAN 100
Host3
IP: 172.16.1.33 (VRF-B)
VLAN 300
Leaf
VV
VLAN 100
V
Host2
IP: 10.10.10.22 (VRF-B)
VLAN 200
SVI 200 SVI 300
VRF-A
(VNI 50001)
VRF-B
(VNI 50002)
Routing
Domain
VRF-B
Routing
Domain
VRF-A
The Routing Domain is the VRF owning multiple Subnets across multiple Switches
In VXLAN EVPN, the Routing Domain consists of three Components
1) The Routing Domains (VRF), local to the Switch
2) The Routing Domain (L3VNI) between the Switches
3) Multi-Protocol BGP with EVPN Address-Family
45. Hardware and Software Support
Spine
RR RR
V
V
V
V
V
V
Nexus 9300 / Nexus 9500 – Leaf, Spine (RR), Border [Shipping]
Nexus 7000/7700 with F3 I/O Modules – Spine (RR), Layer-3 & LISP Border [Shipping]
Nexus 7000/7700 with F3 I/O Modules – Leaf, MPLS Border [Roadmap]
Nexus 5600 – Leaf, Spine (RR), Border [Roadmap]
ASR 9000 – Border [Roadmap]
FYI
46. • VXLAN EVPN provides a scalable Multi-Tenancy architecture
• Integrated Route and Bridging provides routing granularity from
Subnet down to Host level
• Optimized forwarding based on VXLAN Encapsulation with BGP
EVPN reachability protocol
• Routing when you can, Bridging when you need
• Moving away from Flood&Learn to a more scalable Solution
• ACI provides same functions plus Automation plus, plus, plus
Summary