ACI MultiFabric 소개 자료입니다. MultiSite/MultiPoD 의 기술소개와 특장점에 대한 소개입니다.

1. Date : Mar. 2018
Cisco Systems Korea
DC TSA / (whchoi@cisco.com)

2. Cisco ACI Fabric/Policy Domain의 진화
ACI Single Pod Geo Stretch Single Pod
ACI MultiPod ACI MultiSite
ACI 1.0 ACI 1.1
ACI 2.0 ACI 3.0

3. ACI Fabric 구성을 이해하기 위한 용어 정리








4. ACI Fabric 구성을 이해하기 위한 용어 정리
Region / Availability Zone의 이해 - AWS 예
Region



AZ(Avaiability Zone)




5. ACI MultiPod

6. ACI MultiPod Overview
Pod 1 Pod n
MultiPod Fabric
APIC CLUSTER
IPN
VXLAN
Availability Zone





7. ACI MultiPod – Maintenance & Config Zone 지원
Pod 1 Pod n
MultiPod Fabric
APIC CLUSTER
IPN
VXLAN


Config Zone 1 Config Zone 1

8. ACI MultiPod 지원 Topology
10/40/100G Link
APIC CLUSTER APIC CLUSTER
10/40/100G Link
IPN
10/40/100G Link
APIC CLUSTER APIC CLUSTER
10/40/100G Link

9. ACI MultiPod 지원 범위 및 내용
 Nexus 9000 LEAF/SPINE
 ACI LEAF Node (MultiPod )
 3Node APIC Cluster 80 LEAF
 5Node APIC Cluster 300 LEAF
 7Node APIC Cluster 400 LEAF
 Pod 200 LEAF
 Pode 6 SPINE Switch
 Pod
 2.0(1)/2.0(2) – 4 Pod / 2.1(1) - 6 Pod
 2.2(2e) – 10 Pod / 3.0(1) – 12 Pod

10. ACI MultiPod – IPN 스위치 요구 사항
 IPN
 MultiCast PIM BiDir : Layer 2 BUM Traffic
 OSPF : SPINE OSPF
 MTU : VXLAN JumboFrame
 DHCP Relay : TEP address
 IPN Nexus 9000/3000 1 (PIM BiDir )
 IPN SPINE 10G (QSA Adapter )
 Pod SPINE
 PoD to PoD IPN

11. ACI MultiPod – APIC Cluster 구성 고려 사항
Pod1 Pod2
IPN
A B C A B C A B C


X
X
Pod1 Pod2
IPN
A B A B B C


X
A C C
X

12. ACI MultiPod 동작 방식 – 각 PoD AutoProvisioning 동작
Pod 1 Pod 2
MultiPod Fabric
APIC CLUSTER
IPN
1
1. Pod-1 LEAF Node APIC
2. Pod-1 LEAF/SPINE Discovery Provisioning
2
3. Pod-1 SPINE – IPN BGP EVPN
3
4. Pod-2 SPINE – IPN DHCP Reqeust
4
5. Pod-1 APIC DHCP Reply
5
6. Pod-2 SPINE Provisioning
6
7. Pod-2 LEAF/SPINE Discovery Provisioning
8
8. Pod-2 APIC Cluster Join
7

13. ACI MultiPod 동작 방식 – VTEP IP Address 배포 방식
Pod 1 Pod 2
MultiPod Fabric
APIC CLUSTER
IPN
PoD TEP IP Address Pool
- IPN OSPF Routing Summary route advertise.
- IS-IS Convergence Local Pod .
IS-IS to OSPF IS-IS to OSPF
TEP Pool
10.0.0.0/16
TEP Pool
10.1.0.0/16
IPN Routing Table
10.0.0.0/16
10.1.0.0/16
IP Prefix Next-Hop
10.1.0.0/16 Pod1-SPINE
SPINE Pod Routing IS-IS
- Pod VTEP Pod VTEP
.

14. ACI MultiPod 동작 방식 – Pod 간 MP-BGP EVPN 제어부
Pod 1 Pod 2
MultiPod Fabric
APIC CLUSTER
IPN
MP-BGP-EVPN EP(EndPoint), MCAT Group
- Remote Pod Proxy VTEP
- Pod AS .
Pod SPINE iBGP EVPN
- Pod Remote Pod MP iBGP EVPN
Full Mesh .
- PoD 1 BGP RR Spine .
Proxy-1 Proxy-2
EP1 LEAF1
EP2 LEAF3
EP3 Proxy2
EP4 Proxy2
EP1 Proxy1
EP2 Proxy1
EP3 LEAF4
EP4 LEAF6
BGP AS Number
COOP COOP

15. ACI MultiPod 동작 방식 – Pod 간 Unicast DataPlane 1
Pod 1 Pod 2
MultiPod Fabric
APIC CLUSTER
IPN
EP1 LEAF1
* Proxy1
1
1. EP1 Remote Pod EP3
2. LST(Local Station Table) EP3 Unknown ,
Local Proxy SPINE VTEP Encap
EP1 LEAF1
EP3 Proxy2
2
3
3. SPINE Encap Remote Proxy-2 SPINE
VTEP
(Policy, Network VNID, Class-ID )
EP3 LEAF-5
EP1 Proxy1
4. Remote Pod SPINE Encap Remote Pod
LEAF
EP3 LEAF-5
EP1 Proxy2
EP1 Pod-1 LEAF1
5. EP1 , Policy
.
6. Policy Allow , EP3 .
4
5
6

16. ACI MultiPod 동작 방식 – Pod 간 Unicast DataPlane 2
Pod 1 Pod 2
MultiPod Fabric
APIC CLUSTER
IPN
EP1 LEAF1
EP2 Pod-2 LEAF5
* Proxy1
9
7. EP3 Remote EP1
8. LEAF Ingress Enforce Policy , Allow
Remote Leaf1
9. Remote Pod EP3
EP3 LEAF-5
EP1 Proxy2
EP1 Pod-1 LEAF1
8
7
10
10. EP1 EP3
11. EP1, EP3 Pod-1 LEAF1 Pod-2 LEAF5
Encap , Ingress LEAF
.

17. ACI MultiPod 동작 방식 – Pod 간 Multicast DataPlane
Pod 1 Pod 2
MultiPod Fabric
APIC CLUSTER
IPN
1. BD(Bridge Domain) EP1 BUM
2. BD MultiCast Group Multicast
Destination Tree Local Pod Traffic Flood
3. Designated SPINE MCAST Group Traffic IPN
4. IPN Traffic MCAST Group Join
PoD
1
2
3
4
5
5. BUM Traffic Multicast Group Flood
6
6. BUM Traffic Join EP3

18. ACI MultiSite

19. ACI MultiSite Overview
MultiSite
ISN
VXLAN
Availability Zone A




Availability Zone B

20. ACI MultiSite Overview – 정책 구현 방식
MultiSite
ISN
VXLAN
Availability Zone A




Availability Zone B
Availability Zone BAvailability Zone A

21. ACI MultiSite Overview – Network and Identity Extended
MultiSite
APIC CLUSTER
ISN
VXLAN
Availability Zone A Availability Zone B

22. ACI MultiSite Overview – Name Space Normalization
MultiSite
ISN
Availability Zone A



Availability Zone B
SPINE Translation Table

23. ACI MultiSite 지원 SPINE Switch
MultiSite
ISN
VXLAN
Availability Zone A



Availability Zone B

24. ACI MultiSite – ISN (Inter Site Network) 스위치 요구 사항
MultiSite
ISN
VXLAN
Availability Zone A



Availability Zone B

25. ACI MultiSite Networking Option – BD 동작
 BD Subnet
 Layer 3 Intra-VRF Inter-VRF
(VRF / Tenant )

 IP Mobility (Cold VM Migration)
 No Layer2 BUM Flooding across
Site
 Site
 Site Layer2 Domain (Live
VM Migration , Application
Clustering)
 Layer 2 BUM flooding across Site

26. ACI MultiSite 디자인
 Multiple DC Site
 ACI 3.2 (2018 )
 QSA 10G
ISN
 Hybrid
 IPN
 3.1 OSPF IPN
 Hybrid (2018 )

27. ACI MultiSite 확장성 – ACI 3.1 기준

28. ACI MultiSite Orchestrator 소개
ISN













29. ACI MultiSite 동작 방식 – Control Plane & Data Plane Peer
MultiSite
ISN
VXLAN






30. ACI MultiPod 동작 방식 – Site간 TEP 정보 교환 방식
MultiSite Fabric
IPN
 SPINE ISN OSPF Peering
 Routing Site SPINE ETEP
 Local TEP Pool Advertise .
 Site TEP Pool .
IS-IS to OSPF IS-IS to OSPF
TEP Pool
10.0.0.0/16
TEP Pool
10.1.0.0/16
IP Prefix Next-Hop
DP E-TEP 2 Site2-S3,S4
IP Prefix Next-Hop
DP E-TEP 1 Site1-S1,S2

31. ACI MultiPod 동작 방식 – MP BGP EVPN Control Plane
MultiSite Fabric
IPN





COOP COOP
EP1 LEAF1
EP2 LEAF3
EP3 DP-ETEP-2
EP4 DP-ETEP-2
EP1 DP-ETEP-1
EP2 DP-ETEP-1
EP3 LEAF1
EP4 LEAF3
Site A BGP AS Number Site B BGP AS Number

32. ACI MultiSite 동작 방식 – Site 간 Unicast DataPlane 1
MultiSite Fabric
IPN
EP1 LEAF1
* Proxy1
1
1. EP1 Remote Pod EP3
2. LST(Local Station Table) EP3 Unknown ,
Local Proxy SPINE VTEP Encap
EP1 LEAF1
EP3 DP ETEP 2
2
3
3. SPINE Encap Remote DP ETEP2
(Policy, EP1 Class-ID )
EP3 LEAF-1
EP1 DP ETEP1
4. Site2 SPINE VNID Class ID translate Local
Site LEAF
EP3 LEAF-5
* Proxy2
EP1 DP ETEP 1
5. EP1
6. Policy Allow , EP3 .
4
5
6

33. ACI MultiSite 동작 방식 – Site 간 Unicast DataPlane 2
MultiPod Fabric
IPN
EP1 LEAF1
EP3 DP ETEP2
* Proxy1
10
7. EP3 Remote EP1
8. LEAF Ingress Enforce Policy , Allow
DP ETEP1
9. Site 2 SPINE DP-ETEP2 Rewrite ( VNID, ClassID )
EP3 LEAF-1
* Proxy2
EP1 DP ETEP 1
8
7
12
10. Site 1 SPINE VNID, ClassID Local Site
Translate Local LEAF1
11. EP3 LEAF1 LST
EP1 LEAF1
EP3 DP ETEP 2
11
9
12. EP3
13. EP1, EP3 Site-1 LEAF1 Site-2 LEAF1
Encap .
MultiSite Fabric

34. ACI MultiSite 동작 방식 – Site 간 Multicast DataPlane
MultiSite Fabric
IPN
1
1. . BD(Bridge Domain) EP1 BUM
2. BD MultiCast Group Multicast
Destination Tree Local Pod Traffic Flood
2
3
3. SPINE Group IP BUM Traffic
(Multicast HER ETEP B )
4. Site2 SPINE VNID Class ID translate FTAG
Tree
EP3 LEAF-5
* Proxy2
EP1 DP ETEP 1
5. BUM Traffic Group Flooding
6. MCAST Group Join EP3 .
4
5
6

35. 감사합니다.