EYWA is Elastic Load-balancing & high-availablitY Wired network Architecture, Next generation network architecture for IaaS.

1. Limitation
of Cloud Networking
2014-02-04
Jung jung-in (call518@gmail.com)
Jeong wook-jae (wjjung11@gmail.com)

2. SPOF & Bottleneck
in External Traffic

3. SPOF & Bottleneck

4. SPOF & Bottleneck
External
Traffic..
How?
Internet
?

5. SPOF & Bottleneck
One Virtual Router
per Tenant
Internet
Internal Traffic
External Traffic (NAT, LB)

6. SPOF & Bottleneck
Internet
One Virtual Router
per Tenant
Strange.....
Internal Traffic
External Traffic (NAT, LB)

7. SPOF & Bottleneck
Internet
When VM is
increase...
Internal Traffic
External Traffic (NAT, LB)

8. SPOF & Bottleneck
Internet
When VM is
increase...
Internal Traffic
External Traffic (NAT, LB)

9. SPOF & Bottleneck
Internet
When VM is
increase...
Crash
SPOF & Bottleneck
Internal Traffic
External Traffic (NAT, LB)

10. SPOF & Bottleneck
Internet
Also Inbound...
Crash
SPOF & Bottleneck
Internal Traffic
External Traffic (NAT, LB)

11. SPOF & Bottleneck
Internet
Both In/Out...
Crash
SPOF & Bottleneck
Internal Traffic
External Traffic (NAT, LB)

12. There is no clear solution...
(in OpenStack, CloudStack..., Etc...)

13. in OpenStack

14. in OpenStack (Single Host)
 nova-network Basic (Multiple NIC)
Controller
nova-network
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

15. in OpenStack (Single Host)
 nova-network Basic (Multiple NIC)
Controller
nova-network
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

16. in OpenStack (Single Host)
 nova-network Basic (Multiple NIC)
Controller
nova-network
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

17. in OpenStack (Single Host)
 nova-network Basic (Multiple NIC)
Controller
nova-network
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

18. in OpenStack (Single Host)
 nova-network Basic (Multiple NIC)
Controller
nova-network
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

19. in OpenStack (Single Host)
 nova-network Basic (Multiple NIC)
Controller
nova-network
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Compute
nova-compute
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
SPOF &
Bottleneck

20. in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute
Host)
Compute
nova-compute
Compute
nova-compute
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)

21. Compute
nova-compute
Compute
nova-compute
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute
Host)
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
10.0.0.3
10.0.0.11
(GW: 10.0.0.3)
10.0.0.4
10.0.0.12
(GW: 10.0.0.4)

22. Compute
nova-compute
Compute
nova-compute
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
10.0.0.3
10.0.0.11
(GW: 10.0.0.3)
10.0.0.4
10.0.0.12
(GW: 10.0.0.4)
in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute
Host)
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

23. Compute
nova-compute
Compute
nova-compute
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
10.0.0.3
10.0.0.11
(GW: 10.0.0.3)
10.0.0.4
10.0.0.12
(GW: 10.0.0.4)
in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute
Host)
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
but,....

24. Compute
nova-compute
Compute
nova-compute
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
10.0.0.3
10.0.0.11
(GW: 10.0.0.3)
10.0.0.4
10.0.0.12
(GW: 10.0.0.4)
in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute Host)
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

25. Compute
nova-compute
Compute
nova-compute
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
10.0.0.3 10.0.0.11
(GW: 10.0.0.3)
10.0.0.4
10.0.0.12
(GW: 10.0.0.4)
in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute
Host)
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)

26. Compute
nova-compute
Compute
nova-compute
nova-network nova-network
br100
(eth0)
br100
(eth0)
Virtual Network (e.g. 10.0.0.0/8)
10.0.0.3 10.0.0.11
(GW: 10.0.0.3)
10.0.0.4
10.0.0.12
(GW: 10.0.0.4)
in OpenStack (Multiple Host)
 Alternative: Multiple nova-network (per Compute
Host)
eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
!!............

27. in CloudStack
 Using RVM(Router Virtual Machine)
 batter than OpenStack... but,… SPOF & Bottleneck is still….

28. Host Host Host
vnbr1002
(eth0)
vnbr1002
(eth0)
vnbr1002
(eth0)
in CloudStack
vnbr1001
(eth0)
vnbr1001
(eth0)
vnbr1001
(eth0)
Tenat-A Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
Tenat-B Virtual Network (e.g. 10.0.0.0/8)
Tenant-A's VR
(10.0.0.1)
Tenant-B's VR
(10.0.0.1)
A B BA
Tenant B
Tenant A

29. Host Host Host
vnbr1002
(eth0)
vnbr1002
(eth0)
vnbr1002
(eth0)
in CloudStack
vnbr1001
(eth0)
vnbr1001
(eth0)
vnbr1001
(eth0)
Tenat-A Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
Tenat-B Virtual Network (e.g. 10.0.0.0/8)
Tenant-A's VR
(10.0.0.1)
Tenant-B's VR
(10.0.0.1)
A B BA
Tenant B
Tenant A

30. Host Host Host
vnbr1002
(eth0)
vnbr1002
(eth0)
vnbr1002
(eth0)
in CloudStack
vnbr1001
(eth0)
vnbr1001
(eth0)
vnbr1001
(eth0)
Tenat-A Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
Tenat-B Virtual Network (e.g. 10.0.0.0/8)
Tenant-A's VR
(10.0.0.1)
Tenant-B's VR
(10.0.0.1)
A B BA
Tenant B
Tenant A

31. Host Host Host
vnbr1002
(eth0)
vnbr1002
(eth0)
vnbr1002
(eth0)
in CloudStack
vnbr1001
(eth0)
vnbr1001
(eth0)
vnbr1001
(eth0)
Tenat-A Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
Tenat-B Virtual Network (e.g. 10.0.0.0/8)
Tenant-A's VR
(10.0.0.1)
Tenant-B's VR
(10.0.0.1)
A B BA
Tenant B
Tenant A

32. Host Host Host
vnbr1002
(eth0)
vnbr1002
(eth0)
vnbr1002
(eth0)
in CloudStack
vnbr1001
(eth0)
vnbr1001
(eth0)
vnbr1001
(eth0)
Tenat-A Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
Tenat-B Virtual Network (e.g. 10.0.0.0/8)
Tenant-A's VR
(10.0.0.1)
Tenant-B's VR
(10.0.0.1)
A B BA
Tenant B
Tenant A

33. Host Host Host
vnbr1002
(eth0)
vnbr1002
(eth0)
vnbr1002
(eth0)
in CloudStack
vnbr1001
(eth0)
vnbr1001
(eth0)
vnbr1001
(eth0)
Tenat-A Virtual Network (e.g. 10.0.0.0/8)
eth1 eth1 eth1
Physical Network (e.g. 222.122.156.0/24)
Tenat-B Virtual Network (e.g. 10.0.0.0/8)
Tenant-A's VR
(10.0.0.1)
Tenant-B's VR
(10.0.0.1)
A B BA
One VR per Tenant..
But, Still SPOF & Bottleneck
Tenant B
Tenant A

34. to solve…

35. Solution of VR's SPOF & Bottleneck
Host Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA A AB
Tenant B
Tenant A

36. Solution of VR's SPOF & Bottleneck
Host Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA A AB
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
Tenant B
Tenant A

37. Solution of VR's SPOF & Bottleneck
Host Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA A AB
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1 GW: 10.0.0.1 GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A

38. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA A AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1 GW: 10.0.0.1 GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A

39. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA
A
AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1
GW: 10.0.0.1
GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A

40. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA
A
AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1
GW: 10.0.0.1
GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A

41. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA
A
AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1
GW: 10.0.0.1
GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A
X

42. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA
A
AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1
GW: 10.0.0.1
GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A

43. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA
A
AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1
GW: 10.0.0.1
GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A

44. Host
Solution of VR's SPOF & Bottleneck
Host Host
br-B br-B br-Bbr-A br-A br-A
Private-NET A
Physical Network (e.g. 222.122.156.0/24)
Private-NET B
A B BA
A
AB
Selective
10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1 10.0.0.1
GW: 10.0.0.1 GW: 10.0.0.1
GW: 10.0.0.1
GW: 10.0.0.1GW: 10.0.0.1 GW: 10.0.0.1
Tenant B
Tenant A
EYWA

45. EYWA(Elastic load-balancing & high-availabilitY Wired network Architecture)
GltLab https://gitlab.com/call518/eywa-on-opennebula/wikis/home
PoC Demo http://goo.gl/wybYwU
2014-02-04
Jung jung-in (call518@gmail.com)
Jeong wook-jae (wjjung11@gmail.com)

46.  Two Physical Host: CA1, CA2
 HA/LB
 Out-bound
 In-bound
 Network Env.
 Public-IP Range : 172.21.18.0/24
 Private-IP Range : 10.0.0.0/24

47. Out-bound

48. CA1 CA2
S/W (External)
Internet
S/W (Internal)

49. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VR1 (910)
172.21.18.57
10.0.0.1

50. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VR1 (910)
172.21.18.57
10.0.0.1
VM1
(911)
10.0.0.24

51. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VR1 (910)
172.21.18.57
10.0.0.1
VM1
(911)
10.0.0.24

52. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VR1 (910)
172.21.18.57
10.0.0.1
VM1
(911)
VM2
(912)
10.0.0.24 10.0.0.25

53. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VM1
(911)
VM2
(912)
VR1 (910)
172.21.18.57
10.0.0.1
10.0.0.24 10.0.0.25

54. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VM1
(911)
VM2
(912)
VR2 (913)
172.21.18.58
10.0.0.1
VR1 (910)
172.21.18.57
10.0.0.1
10.0.0.24 10.0.0.25

55. CA1 CA2
S/W (External)
google.com
S/W (Internal)
VM2
(912)
VR2 (913)
172.21.18.58
10.0.0.1
VR1 (910)
172.21.18.57
10.0.0.1
VM1
(911)
10.0.0.24 10.0.0.25

56. CA1 CA2
S/W (External)
google.com
S/W (Internal)
VM2
(912)
VR1 (910)
172.21.18.57
10.0.0.1
VM1
(911)
VR2 (913)
172.21.18.58
10.0.0.1Crashed
10.0.0.24 10.0.0.25

57. CA1 CA2
S/W (External)
google.com
S/W (Internal)
VM1
(912)
VM2
(913)
VR1 (911)
172.21.18.57
10.0.0.1
10.0.0.24 10.0.0.25

58. CA1 CA2
S/W (External)
google.com
S/W (Internal)
VM1
(911)
VM2
(912)
VR1 (910)
172.21.18.57
10.0.0.1
VR3 (914)
172.21.18.58
10.0.0.1
Added
10.0.0.24 10.0.0.25

59. CA1 CA2
S/W (External)
google.com
S/W (Internal)
VM1
(911)
VM2
(912)
VR3 (914)
172.21.18.58
10.0.0.1
VR1 (910)
172.21.18.57
10.0.0.1
10.0.0.24 10.0.0.25
Added

60. Load-Balancing
(In-bound)

61. CA1 CA2
S/W (External)
Internet
S/W (Internal)
VM2
(912)
VR3 (914)
172.21.18.58:8080
VR1 (910)
172.21.18.57:8080
VM1
(911)
10.0.0.25:8010.0.0.24:80

62. CA1 CA2
S/W (External)
S/W (Internal)
Internet
VM1
(911)
VM2
(912)
10.0.0.25:8010.0.0.24:80
VR1 (910)
172.21.18.57:8080
VR3 (914)
172.21.18.58:8080

63. CA1 CA2
S/W (External)
S/W (Internal)
Internet
VM1
(911)
VM2
(912)
10.0.0.25:8010.0.0.24:80
VR3 (914)
172.21.18.58:8080
VR1 (910)
172.21.18.57:8080

64. CA1 CA2
S/W (External)
S/W (Internal)
Internet
VM1
(911)
VM2
(912)
DNS-RoundRobin
10.0.0.25:8010.0.0.24:80
VR3 (914)
172.21.18.58:8080
VR1 (910)
172.21.18.57:8080

65. CA1 CA2
S/W (External)
S/W (Internal)
Internet
VM1
(911)
VM2
(912)
DNS-RoundRobin
VR3 (914)
172.21.18.58:8080
Crashed
10.0.0.25:8010.0.0.24:80
VR1 (910)
172.21.18.57:8080

66. CA1 CA2
S/W (External)
S/W (Internal)
Internet
VM1
(911)
VM2
(912)
DNS-RoundRobin
10.0.0.25:8010.0.0.24:80
VR1 (910)
172.21.18.57:8080

67. Architecture
Overview

68. VSe (Common)
VR-A-1
111.2.3.11
10.0.0.1/8
VSi
A
vport-A
VM
VSe (Common) VSe (Common)
Switch (Public Net.)
Switch (Private Net.)
VR-A-2
111.2.3.12
10.0.0.1/8
VR-B-1
111.2.3.21
10.0.0.1/8
VSi
A
vport-A
VM
VSi
B
vport-B
VM
vtep-A vtep-A vtep-B
vnet0
peth0
vnet0
peth0
vnet0
peth0
eth0
eth1
eth0
eth1
eth0
eth1
vnet1
vnet2
peth1 peth1
VSi
A
vport-A
VM
vtep-A
Tenant A
Tenant B
VR-B-2
VSi
B
vport-B
VM
eth0
eth1
111.2.3.22
10.0.0.1/8
ControllerController ControllerController
Orphan Normal
Normal Normal
eth0eth0
vnet1
eth0
vnet3
eth0
vnet1
eth0
vnet2
peth1
vtep-B

69. END