Discover how how the standards-based Cisco Programmable Fabric with open APIs enables the Cisco Virtual Topology System (VTS) to automate overlay network provisioning with a highly scalable solution that truly integrates physical and virtual networks. Learn how the Cisco VTS dramatically simplifies operations, reducing provisioning time from weeks to mere seconds. Find out how this open standards-based BGP EVPN control plane increases VXLAN scalability, extending workload placement and mobility range.

1. Automate Programmable Fabric
in Seconds with an Open,
Standards-Based Solution
Lukas Krattiger @CCIE21921
October, 2015

2. • Introduction
• Network Overlay Fabrics, Management and Operations
• VXLAN with BGP EVPN
• Cisco’s Virtual Topology System (VTS)
• Demo
Agenda

3. Introduction

4. Cloud
• Elastic resource allocation
• Self Service consumption
• XaaS
Big Data
• Increase East-West Traffic
• Application driven
Networking
Mobility
• Increased number of
Smart & Mobile End-Points
• Any content Anywhere
Social Media
• Application and Storage
Scale
• Performance
• Programmable
Infrastructure
• Open API
• Services Orchestration
• Workload Placement and
Migration
• Physical & Virtual
Integration
• Simplified Management
• Profile-based Provisioning
ManageabilityAgilityProgrammability
• Massive Scale (Compute,
Tenants, Services)
• Scalable Architecture
Scalability
Data Center Trends

5. Programmable NetworkProgrammable FabricApplication Centric
Infrastructure
DB DB
Web Web App Web App
VxLAN-BGP EVPN
standard-based
3rd party controller support
Modern NX-OS with enhanced
NX-APIs
Automation Ecosystem
(Puppet, Chef, Ansible etc.)
Common NX-API
across N2K-N9K
Turnkey integrated solution with
security, centralized management,
compliance and scale
Automated application centric-policy
model with embedded security
Broad and deep ecosystem
Cisco SDN: Providing Choice in Automation and Programmability
Mass Market
(commercial, enterprises, public sector)
Service Providers Mega Scale Datacenters
VTS for software overlay
provisioning and management
across N2K-N9K

6. MAN/WA
N
FabricPath
/BGP
MAN/WA
N
VXLAN
/EVPN
STP
VPC
MAN/WA
N
FabricPath
VXLAN
ACI Fabric
Application Policy
Infrastructure
Controller
APIC
Application Centric Infrastructure
Data Center “Fabric” Journey

7. App-Based Automation
Automated L4-7 Stitching
Turnkey network automation
Application Centric Infrastructure (ACI)
APIC

8. Programmable Fabric
Discover Fabric Topology
Image & Config Repository
Monitor Fabric
Common Point of Access
Open APIs
FEX Switch# show vlan
Vlan
--------------------
VTS
NX-API Rest API
Image and Configuration
Management
POAP
Simplified Management for Ease of Operations
Open BGP EVPN with VXLAN Fabric
Physical
V
M
OS
V
M
OS
Virtual DCI/WAN
Infrastructure Domain
Administrators
vCenter
DevOps Operations /
Programmability
& Automation
Automation and Programmability

9. Programmable Fabric
Network Control
Infrastructure
• Integration with Orchestrators and
Hypervisor Managers
• Automation of Network Provisioning
• Programmable Network Fabric
• Physical and Virtual Switching
• Network Virtualization

10. Network Overlay Fabrics,
Management and
Operations

11. Programmable Fabric
Infrastructure
• Programmable Network Fabric
• Physical and Virtual Switching
• Network Virtualization

12. Overlay Based Data Center Fabrics
Desirable Attributes:
• Mobility
• Segmentation
• Scale
• Automated & Programmable
• Abstracted consumption models
• Full Cross Sectional Bandwidth
• Layer-2 + Layer-3 Connectivity
• Physical + Virtual
RR RR

13. Overlay Based Data Center: Edge Devices
Network Overlays Hybrid OverlaysHost Overlays
• Virtual end-points only
• Single admin domain
• VXLAN, NVGRE, STT
• Physical and Virtual
• Resiliency + Scale
• X-Organizations/Federation
• Open Standards
• Router/Switch end-points
• Protocols for Resiliency/Loops
• Traditional VPNs
• VXLAN, OTV, VPLS, LISP, FP
V
V
V
V
V
V

14. Overlay Services
• Layer 2
• Layer 3
• Layer 2 and Layer 3
Tunnel Encapsulation
Underlay Transport
Network
Control Plane
• Peer Discovery mechanism
• Route Learning and Distribution
– Local Learning
– Remote Learning
Data Plane
• Overlay Layer 2/Layer 3 Unicast traffic
• Overlay Broadcast, Unknown Unicast,
Multicast traffic (BUM traffic) forwarding
– Ingress Replication
– Multicast
Understanding Overlay Technologies

15. Local LAN
Segment
Physical
Host
Local LAN
Segment
Physical
Host
Virtual Hosts
Local LAN
Segment
Virtual Switch
Edge Device
Edge Device
Edge Device
IP Interface
IP Fabric Overlay Taxonomy (1)

16. Local LAN
Segment
Physical
Host
Local LAN
Segment
Physical
Host
Virtual Hosts
Local LAN
Segment
Virtual Switch
VTEP
VTEP
VTEP
VTEP – VXLAN Tunnel End-Point
VNI/VNID – VXLAN Network Identifier
VV
V
Encapsulation
IP Fabric Overlay Taxonomy (2)

17. Programmable Fabric
Network Control
• Integration with Orchestrators and
Hypervisor Managers
• Automation of Network Provisioning

18. Fabric Management & Operations
Troubleshooting
Day-2:
Visibility,
Configuration
increments,
compare changes.
Day- 1:
Configuration and
Configuration
Management
Automated
Configuration
Compute
Integration
Day-0:
Configuration
(POAP)
Underlay Network
Provisioning and
Management
Element
management:
Hardware
Management,
Health Status, and
Inventory

19. VXLAN with BGP EVPN

20. Optimized Networks with VXLAN
2
Overlay
(VXLAN)Integrated
Route/Bridge
Underlay
BGP
(EVPN)

21. • Standards based Overlay (VXLAN) with Standards
based Control-Plane (BGP)
• Layer-2 MAC and Layer-3 IP information distribution by
Control-Plane (BGP)
• Forwarding decision based on Control-Plane
(minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized
Forwarding in the Overlay
• Multi-Tenancy At Scale
What is VXLAN with BGP EVPN?

22. Control-
Plane
EVPN MP-BGP - RFC 7432
(draft-ietf-l2vpn-evpn)
Data-
Plane
Multi-Protocol Label Switching
(MPLS)
draft-ietf-l2vpn-evpn
Provider Backbone Bridges
(PBB)
draft-ietf-l2vpn-pbb-evpn
Network Virtualization Overlay
(NVO)
draft-ietf-bess-evpn-overlay
 EVPN over NVO Tunnels (e.g. VXLAN) for Data Center
Fabric encapsulations
 Provides Layer 2 and Layer 3 Overlays over simple IP
Networks
Ethernet VPN – EVPN

23. ID Title Category
RFC 7348
Virtual eXtensible Local Area Network Data Plane
RFC 7432
BGP MPLS based Ethernet VPNs EVPN Control Plane
draft-ietf-bess-evpn-overlay
A Network Virtualization Overlay Solution using EVPN EVPN Control Plane
draft-ietf-bess-evpn-inter-subnet-forwarding
Integrated Routing and Bridging in EVPN EVPN Control Plane
draft-ietf-bess-evpn-prefix-advertisement
IP Prefix Advertisement in E-VPN EVPN Control Plane
Draft-tissa-nvo3-oam-fm
NVO3 Fault Management Mgmt Plane (OAM)
IETF RFC & Drafts – Implemented by Cisco

24. VXLAN Evolution with BGP EVPN
Protocol Learning
• Uses MP-BGP with EVPN
Address family
• Workload MAC and IP
Addresses learnt by VXLAN
Edge Devices (NVEs)
• Advertises Layer-2 and Layer-
3 Address-to-VTEP
Association
• Flood Prevention
• Optimized ARP forwarding
IP Services
• VXLAN Routing
• Distributed Anycast Gateway
Multi-Tenancy
• Route Reflector for Scale
External Connectivity
• VXLAN Hardware Gateway
Redundancy (VPC)
• Integrated physical and
virtual Overlays (Hybrid
Overlays)
• Inter-Pod Connectivity
• VXLAN Gateway to other
Encaps/Networks
Multicast Independent*
• Overlay Control-Plane
provides dynamic VTEP
discovery
• Head-End Replication
enables Unicast-only mode
(aka ingress Replication)
*Multicast Independence requires the usage of the Overlay Control-Plane or static configuration

25. Overlay with Optimized Routing
Spine
RR RR
V
V
V
V
V
V
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
RR
Route-Reflectors deployed
for scaling purposes
iBGP Adjacencies
Border

26. Overlay with Optimized Routing
Spine
RR RR
V
V
V
V
V
V
EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes
RR
Route-Reflectors deployed
for scaling purposes
iBGP Adjacencies
Border
Scalable Multi-Tenancy with Multiprotocol BGP
EVPN Address-Family: Host MAC+IP, internal/external IP Subnets
BGP enhanced for Fast Convergence at Large Scale
Extensions for Fast and Seamless Host Mobility
Distributed Gateway with Traffic Flow Symmetry
ARP Suppression

27. Gateway Functions in VXLAN
Centralized Gateway
• Extra Bridging hop before and after Routing
• Centralized Gateway (Aggregation) for Routing
• Large amounts of state => convergence issues
• Scale problem for large Layer-2 domains
• Works with VXLAN Flood & Learn
Distributed Gateway
• Route or Bridge at Leaf
• Distributed Gateway (Anycast) for Routing
• Disaggregate state by scale out
• Optimal Scalability
• Used with VXLAN/EVPN!
V
V
V
V
V
V
V
V
Layer-3 Boundary
Layer-3 Boundary

28. Distributed IP Anycast Gateway
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
route

29. Distributed IP Anycast Gateway
Spine
RR RR
V
V
V
V
V
V
SVI 200
SVI 100
SVI 100
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA:AA:AA:AA:AA:AA
IP: 192.168.1.11
VLAN 100
VXLAN VNI 30001
Host3
MAC: CC:CC:CC:CC:CC:CC
IP: 192.168.1.33
VLAN 100
VXLAN VNI 30001
Host2
MAC: BB:BB:BB:BB:BB:BB
IP: 10.10.10.22
VLAN 200
VXLAN VNI 30002
bridge
route
Any Subnet Routed Anywhere – Any VTEP can serve any Subnet
Integrated Route & Bridge (IRB) - Route whenever you can, Bridge when needed
No Hairpinning – Optimized East/West and North/South Routing
Seamless Mobility - All Leaf share same Gateway MAC
Reduced Failure Domain – Layer-2/Layer-3 Boundary at Leaf
Optimal Scalability – Route Distributed & closest to the Host

30. Multi-Destination Traffic
Ingress Replication
Unicast based Replication
• Source VTEP sends unicast copy to every
Destination VTEP
• Requires only Unicast
• Simples way of Traffic Replication
Multicast
Multicast based Replication
• Source VTEP sends single copy, Multicast
replicates it as needed
• Leverages Multicast Routing (PIM)
• Most optimal way of Traffic Replication
Spine
RR RR
V
V
V
V
V
V
Spine
RR RR
V
V
V
V
V
V

31. Hardware and Software Support
Spine
RR RR
V
V
V
V
V
V
Nexus 9300 / Nexus 9500 – Leaf, Spine (RR), Border [Shipping]
Nexus 7000/7700 with F3 I/O Modules – Spine (RR), Layer-3 & LISP Border [Shipping]
Nexus 7000/7700 with F3 I/O Modules – Leaf, MPLS Border [Roadmap]
Nexus 5600 – Leaf, Spine (RR), Border [Roadmap]
ASR 9000 – Border [Roadmap]

32. Cisco Virtual Topology
System (VTS)

33. Cisco VTS: Cisco SDN Across Nexus
Portfolio
VTS
vCenter
REST API
GUI
Nexus Portfolio
Nexus 2k – 9k
Programmable Fabric
Scalable Multi-Tenancy
• MP-BGP EVPN control plane
• Physical and Virtual overlay support
• High performance virtual forwarding
Automated Provisioning
• Group Based Policy model
• Overlay Provisioning
• Service Chaining
Open, Standards Based
• Rest based Northbound APIs
• Multi-protocol support (EVPN, VXLAN)
• Multi-Hypervisor
Overlay Management
• Automatic Topology Discovery
• Resources Management
• Overlay monitoring and troubleshooting

34. VTS: Network Topology & Host Discovery
Virtual Switch Virtual Switch
• Device Inventory Collection
• Topology Discovery through LLDP
• Device Configuration Collection
• Server Host Discovery through LLDP
• Import Server to TOR Connection
• Virtual Machine Discovery through
VMM integration
Virtual Topology System
(VTS)VTS

35. VTS: Resource Pool Management
Virtual Switch Virtual Switch
Fabric Global Resources
• VXLAN VNID
• Multicast Address
Leaf Local Resources
• VLAN ID’s
Virtual Topology System
(VTS)VTS
• VLAN IDs on different Leaf switches can differ
• Automated VLANs to VNIs to ensure expected
connectivity
• Manual Resources Management leads to
"errors" and "mis-configurations”
• Automatic Resource Allocation and De-
Allocation on Provisioning

36. VTS Architecture
Spine
Border
vCenter GUI
3rd Party VM
Manager
Cisco NSO
REST API
V
V
V
V
V
V
RR RR
MP-BGP
BGP EVPN
Virtual Topology System
(VTS)
DCI
RR
• Service Provider Oriented Architecture

37. VTS Architecture
Spine
Border
vCenter GUI
3rd Party VM
Manager
Cisco NSO
REST API
V
V
V
V
V
V
RR RR
Virtual Topology System
(VTS)
DCI
RR
VXLAN
bridge/route
• Service Provider Oriented Architecture

38. Cisco NFVI Platform
Virtualized
Compute
Virtualized
Storage
Virtualized
Network
Compute Storage Network
Openstack
SDN
Controller (Optional)
InfraMonitoring&
Assurance
SinglePaneof
GlassMgmt
Cisco UCS
DAS on UCS or
NAS / SAN Nexus + ASR
Cisco VTF/OVS
vMS
Orchestrator
SP Mobility
Media
Cloud
xAAS
EMS EMS EMS
VF VF VF
EMS
VF
VTS
http://blogs.cisco.com/sp/validating-cisco-network-function-virtualization-infrastructure-nfvi

39. Overlay Provisioning:
Use Case
Border
vCenter GUI
3rd Party VM
Manager
Cisco NSO
REST API
V
V
V
V
V
V
Virtual Topology System
(VTS)
• Layer-2 / Layer-3 VXLAN Configuration
using MP-BGP EVPN control-plane
• Allocate and Manage resources
• Support for Physical and Virtual End-
Hosts
• End-to-End Automation
• Openstack and vCenter integration

40. Overlay Provisioning:
Use Case
Border
vCenter GUI
3rd Party VM
Manager
Cisco NSO
REST API
V
V
V
V
V
V
Virtual Topology System
(VTS)
DCI
• Virtual VTEP integration using VTF
• Integration with DCI

41. Multi Data Center NFVI:
Use Case Cisco NSO
REST API
Cisco VTS
(H/A)
Spine
Border
V
V
V
RR
DCI
DCI
Data Center 1
Cisco
VTS
Availability Zone 1 (DC2)
Cisco
VTS
Availability Zone 2 (DC3)
VXLAN
bridge/route

42. Demo

43. Demo: VTS based provisioning with vCenter
Spine
RR RR
V
V
V
V
V
V
SVI
SVI
SVI
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA
IP: 192.168.1.11
Host3
MAC: CC
IP: 192.168.1.33
Host2
MAC: BB
IP: 10.10.10.22
REST API
VTS
• Service Provider Oriented Operational Model
VTS plugin for
vCenter

44. • Service Provider Oriented Operational Model
Demo: VTS based provisioning with vCenter
Spine
RR RR
V
V
V
V
V
V
SVI
SVI
SVI
SVI 100, Gateway IP: 192.168.1.1
SVI 200, Gateway IP: 10.10.10.1
Host1
MAC: AA
IP: 192.168.1.11
Host3
MAC: CC
IP: 192.168.1.33
Host2
MAC: BB
IP: 10.10.10.22
VTS plugin for
vCenter
REST API
VTS
Create Tenant
Create Network (Layer-2 Segment)
Create Router (VRF and Gateway)
Create Tenant
Create Network and derive VNID automatic from Pool
Create Router (VRF and Gateway)
VLAN is assigned for each Leaf Attach VM to Network
Layer-2 and Layer-3 VXLAN Services is Configured across the Managed Leafs

45. Programmable Network
DB DB
Web Web App Web App
VxLAN with BGP EVPN
standard-based
3rd party controller support
Modern NX-OS with enhanced
NX-APIs
Automation Ecosystem
(Puppet, Chef, Ansible etc.)
Common NX-API
across N2K-N9K
Turnkey integrated solution with
security, centralized management,
compliance and scale
Automated application centric-policy
model with embedded security
Broad and deep ecosystem
VTS for software overlay
provisioning and management
across N2K-N9K
Application Centric Infrastructure Programmable Fabric Programmable Network
Cisco SDN: Providing Choice in Automation
and Programmability

46. • Cisco Application Centric Infrastructure
• http://www.cisco.com/go/ACI
• Cisco VTS
• http://www.cisco.com/go/VTS
• Cisco Nexus 9000 Series Switches - VXLAN Network with MP-BGP EVPN Control
Plane
• http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-
c07-734107.html
• Cisco Nexus 7000: VXLAN BGP EVPN
• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/nx-
os/vxlan/configuration/guide/b_NX-
OS_VXLAN_Configuration_Guide/configuring_vxlan_bgp_evpn.html
Additional Resources