How Neutron Builds Network Topology For Your Multi Tier Application.

1. How Neutron Builds Network Topology For Your
Multi-tier Application
Sadique Puthen
Principal Technical Support Engineer, Red Hat
20 May 2015, Openstack Summit, Vancouver

2. Agenda
❖Neutron Overview
➢Native and External components
➢Network Namespace
❖Application Network Topology.
➢How to build application network topology.
❖Correlate application topology with Neutron
➢What happens inside neutron when we build it?
➢Explore with Native HA available in neutron.
❖Why am I doing this?

3. Neutron Overview and Components

4. ❖Neutron is a project with in Openstack
that delivers Networking-as-a-Service.
➢Create/manage/remove Networks, Routers,
Load Balancers, Firewalls, VPNs, Layer 2
flow, etc on demand.
➢Involves multiple neutron native
components.
➢Depends highly on external components.
Neutron Overview

5. ❖Native Components Involves:
➢Neutron API
➢Neutron DHCP Agent
➢Neutron Layer 3 Agent
➢Neutron LbaaS Agent
➢Neutron FWaaS Agent
➢Neutron VPNaaS Agent
➢Metadata Agent.
➢Neutron Layer 2 Agent.
◆eg. Neutron Openvswitch Agent.
◆Other third party Agents.
Neutron Components
Database
Messagequeue
Neutron API
Servers
lbaas Agents dhcp Agents
l3 Agents
Metadata Agents
layer 2 agents
fwaas Agents
vpn Agents

6. ❖External Components.
➢Network Namespace
➢Openvswitch
➢Dnsmasq
➢Libreswan/Openswan
➢Keepalived
➢Haproxy or a third party HA daemon
➢Iptables firewall
◆DNAT/SNAT
➢Networking userspace components.
➢kernel
Neutron External Components
Layer 2
Agent
Metadata
Agent
VPN Agent
L3 AgentDHCP Agent
fwaas Agent API Server
Openvswitch
dnsmasq
openswan
iptables
haproxy
kernel
networking
tools
keepalived
DNAT/SNAT
LBaaS Agent
Neutron

7. ❖A namespace allows isolation of a
group of resources to its own
space.
➢These resources in the group run
with the illusion that they are the only
processes on the system.
➢Multiple such groups can run on a
system without knowing each other.
■ This facilitates using same parameters.
Namespace and Network Namespace
namespace1 namespace2 namespace3
httpd 192.168.1.1
iptables routing
Port 80
httpd 192.168.1.1
iptables routing
Port 80
httpd 192.168.1.1
iptables routing
Port 80interface interface interface
Operating System/ Kernel
➢Grouping network related resources in such a way forms network namespaces.
■ Neutron uses this to support overlapping ip addressing

8. Neutron Physical Topology
❖Neutron components are usually
spread into three types of physical
systems.
➢Controllers.
■ Usually more than one node is deployed for
high availability.
■ Runs neutron API server.
➢Network Nodes.
■ Usually more than one node is deployed for
high availability.
■ Runs l3, dhcp, lbaas, vpn and layer 2 agents
➢Compute nodes run layer 2 agent.
dhcp-agent
l3-agent
lbaas-agent
vpn-agent
layer 2 agen
metadata-agent
neutron-serverneutron-server
Controller Nodes
Network Nodes
Compute Nodes
Layer 2 agents Layer 2 agents
dhcp-agent
l3-agent
lbaas-agent
vpn-agent
layer 2 agen
metadata-agent

9. Application Network Topology

10. Multi-Tier Application Network Topology
Instance
Instance
Instance
Instance
Instance
Instance
Instance
Instance
DatabaseNetwork
WebserverNetwork
Floating
IP
Instance
Instance
Instance
Instance
ApplicationServer
Network
RemoteNetwork
Tenant X
Openstack Site B Openstack Site ALoadBalancer
Internet
192.168.1.0/24 192.168.2.0/24 192.168.3.0/24192.168.4.0/24

11. Example multi-tier application
❖ On Site A
➢Application that we are going to build involves
■ A web tier with a load balancer in the front.
■ An application tier.
■ A database tier.
➢Each tier is isolated to its own private network.
■ Connected together using separate virtual routers.
■ Webserver network is also connected to internet.
■ There is a firewall between each network.
TenantA has two sites siteA and siteB.
DatabaseNetwork
WebserverNetwork
Floating IP
ApplicationServer
Network
LoadBalancer
192.168.1.0/24 192.168.2.0/24 192.168.3.0/24
Instance
Instance
Instance
Instance
Instance
Instance Instance
Instance
Instance
Internet
➢HTTP requests from end user is listening on a floating ip that forwards requests to
the private VIP for the load balancer.

12. Example multi-tier application
❖On Site B
➢ Tenant has one private network connected to
Internet.
➢Web server private network and this private
network need to be connected together using a
VPN for secure data transfer and enable direct
network communication.
TenantA has two sites siteA and siteB.
Instance
Instance
Instance
Remote
Network
Internet
192.168.4.0/24

13. How To Build It?
What Happens Inside Neutron?

14. How to build it? Neutron Topology.
❖There are multiple ways to build it.
➢Use horizon dashboard.
➢Use CLI.
➢Via API.
➢Use a heat template to automate them.
br-int
br-tun
br-int
br-tun
br-ex
br-ex
Intenet
❖Neutron topology that I am going to
correlate with has two network nodes.
➢Each neutron agent runs with HA using HA
tool (Not discussed here).
➢Tenant virtual routers and dhcp processes
for each network run with native HA
available in neutron.

15. Create Networks
❖On Site A, create three networks and
spawn instances to them.
➢Specify name, subnet, gateway, allocation
pool, etc during creation.
1
2 3

16. Create Networks - Spawn instances to networks
❖Create Networks and spawn instances to it.
➢neutron-api service accepts the request and
moves to AMQP for further processing.
➢For dhcp-agent.
■ create qdhcp-xx namespace.
■ Create a port in the namespace, tag it using
internal vlan id and attach to br-int.
■ spawns dnsmasq process to serve the network
bound to the port.
➢On Network node.
■ Add required ovs flows to map internal vlan id
for each network to tunnel id and vice-versa.
■ Tag associated ports for each networks using its
internal vlan id.
Webservernetwork
Applicationservernetwork
Databsenetwork
qdhcp-web
dns
masq
qdhcp-appqdhcp-db
dns
masq
dns
masq
br-intbr-tun
br-intbr-tun
qdhcp-webqdhcp-appqdhcp-db
dns
masq
dns
masq
dns
masq
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
5
6
7
6
7
5

17. Spawn Instances to Networks - Compute node
❖Associate each private network with an
internal ovs vlan id.
➢This vlan id is translated to a gre/vxlan tunnel id or
external vlan id before transporting the packet
using a physical shared device and vice-versa.
➢For each instance nic:
■ Create a linux bridge.
● Connect instance nic to it. Security group rules are
applied here.
● Connect it to br-int using patch-peer.
● When the packet reaches patch-peer end on br-int,
tag the packet using internal vlan id for that network.
■ The packet is then moved to br-tun where a flow in ovs
removes vlan id and adds tunnel id associated with the
network and moved to the physical network devices.
■ Return packet does this in the reverse order.
65

18. Create Routers, Connect Network Together
❖Create routers.
➢Attach ports from subnets to respective
routers.
1 2

19. Create Routers, Connect networks together
❖Create routers to connect networks together.
➢neutron-api service accepts the request and moves to
AMQP for further processing.
➢ Each l3 agent,
■ creates qrouter-xx namespace for each router.
■ Create an port for VRRP traffic and assign it ip from pre-
created VRRP network.
■ Spawn keepalived process bound to the namespace to
facilitate HA of the router.
■ Creates two or more ports in the namespace, tag it using
internal vlan id and attach to br-int
■ Keepalived elects one of the namespace as master and
binds default gateway of the associated networks to it.
■ kernel ip forwarding is enabled between the ports and
routing between them starts working.
br-intbr-tun
br-intbr-tun
Instanc
e
Webservernetwork
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Instanc
e
Applicationservernetwork
Databsenetwork
qrouter-app
qrouter-app
qrouter-db
Keepa
lived
Keepa
lived
Keepa
lived
Keepa
lived
qr-xxqr-xxtapx
qrouter-db
qr-xxtapxqr-xx
qr-xxqr-xxtapxqr-xxqr-xxtapx
5
6
8
7
6
8
5
6
7
6
8
8

20. Create Load Balancer For Webservers
❖Create Load Balancer.
➢Create health monitor, pool, add
members (instances) to pools, create
and associate VIP and with pool to
create a load balancer.
1 2
3 4

21. Create Load Balancer for Webservers
❖Place load balancer for web server network.
➢neutron-api service accepts the request and moves
to AMQP for further processing.
➢On lbaas agent.
■ Create qlbaas-xx namespace for each load
balancer.
■ Create a port in the namespace, tag it using
internal vlan id for the network, bind VIP to the
port and attach to br-int.
■ Spawn haproxy process on the port to listen for
the service on VIP and load balance among the
back end instances.
br-intbr-tun
br-intbr-tun
Webservernetwork
Applicationservernetwork
Databsenetwork
qlbaas-web
LoadBalancer
tapxx
Instance
Instance
Instance Instance
Instance
Instance Instance
Instance
Instance
hapro
xy
8

22. External network and floating ip
❖Associate floating ip with VIP
➢Create a router, Set external network as
gateway, Attach webserver network to
it.
➢Allocate a floating ip and associate with
VIP1 2 3

23. External network and floating ip
❖Floating ips and routing to external network.
➢On l3_agent
■ Create qrouter-xx namespace for the external
router.
■ Creates two ports in the namespace (qr-xx and
qg-xx), tag it using internal vlan id, bind default
gateway of the tenant network on qr-xx and an ip
from external network on qg-xxx.
■ Binds floating ips as aliases on qg-xx port and
adds appropriate DNAT and SNAT rules to nat
table to forward them to the correct private ip of
the instance.
● In our case, DNAT/SNAT is done to the VIP of the load
balancer.
Webservernetwork
Applicationservernetwork
Databsenetwork
LoadBalancer
Intenet
Instance
Instance
Instance Instance
Instance
Instance Instance
Instance
Instance
RemoteNetwork
Instance
Instance
Instance
br-int
br-tun
br-intbr-tun
br-ex
br-ex
Intenet
qrouter-webqlbaas-web
tapxx
floating
IP
keepali
ved
keepali
ved floating
IP
qr-xxqg-xxtapx
qrouter-web
qr-xxtapxqg-xx
5
8
5
7
8
8
7

24. Create firewalls
❖Create firewalls.
➢ Create firewall rules, policies and
then firewalls so they are automatically
associated with routers.
1 2 3

25. Create firewalls
❖Floating ips and routing to external network.
➢neutron-api service accepts the request and moves
to AMQP. fwaas_agent gets the request that need
to be processed from AMQP.
➢On fwaas_agent.
■ Creation of firewall rules and policies has
nothing to do except updating database.
■ Creating firewall translates firewall rules and
policies to corresponding iptables rules and is
applied on all active routers.
■ Running iptables -nL inside namespace will
show us the list of rules.
■ It also adds a REJECT rule at the end to drop
everything not allowed explicitly.
Webservernetwork
Applicationservernetwork
Databsenetwork
LoadBalancer
Intenet
Instance
Instance
Instance Instance
Instance
Instance Instance
Instance
Instance
RemoteNetwork
Instance
Instance
Instance
br-intbr-tun
br-intbr-tun
qrouter-db
qrouter-db
qrouter-appqrouter-web
qrouter-web
iptables
rules applied
here
iptables
rules applied
here
iptables
rules applied
here
iptables
rules applied
here
iptables
rules applied
here
qrouter-app
qr-xxqr-xxqg-xxqr-xx
qr-xxqr-xxqr-xxqr-xxqr-xxqg-xx
qr-xxqr-xx
5
6
5
6
7
7
iptables
rules applied
here
5
6
7
7
6
5

26. Create VPN and Connect remote network.
❖Create VPN.
➢Create IKE policy, IPsec Policy and VPN
Service.
➢Ceate a Site-to-Site connection using above
details and peer router ip and remote private
network.
1 2
43

27. Create VPN and connect remote network
❖Floating ips and routing to external network.
➢vpn_agent when a site-to-site connection is
created:
■ Spawns a libreswan/openvpn process to the
qrouter-xxx namespace using a configuration file
generated with details of tunnel to be created
between web server private network and remote
private network.
■ This process on the source qrouter-xx gateway
captures packets to the remote private network,
encapsulates it, sends it over internet to the
gateway on remote site via a tunnel.
■ On the remote end, a similar process
decapsulates it and routes to the destination
private network.
Webservernetwork
Applicationservernetwork
Databsenetwork
LoadBalancer
Intenet
Instance
Instance
Instance Instance
Instance
Instance Instance
Instance
Instance
RemoteNetwork
Instance
Instance
Instance
br-int
br-tun
br-intbr-tun
br-ex
br-ex
Intenet
qrouter-web
qrouter-web
External
vpn gw ip
keepali
ved
keepali
ved
libreswan/vpn
libreswan/vpn
External
vpn gw ip
qr-xxqg-xxtapx
qr-xxqg-xxtapx
5
6
8
5
6
8

28. Why This Talk?

29. Applicaton topology - Simple to create.
Instance
Instance
Instance
Instance
Instance
Instance
Instance
Instance
DatabaseNetwork
WebserverNetwork
Floating
IP
Instance
Instance
Instance
Instance
ApplicationServer
Network
RemoteNetwork
Tenant X
Openstack Site B Openstack Site ALoadBalancer
Internet

30. Network node 1 Site ANetwork node 1 Network node 2
qrouter-
web
qrouter-appqrouter-dbqrouter-web
qlbaas-web
qdhcp-webqdhcp-appqdhcp-db
br-int
qdhcp-webqdhcp-appqdhcp-db
patch
peer
qrouter-appqrouter-dbqrouter-web
br-tun
br-tun
libreswan/vpn
libreswan/vpn
br-ex
patch
peerbr-ex
patch
peer
dns
masq
dns
masq
dns
masq
hapr
oxy
patch
peer
br-int
dns
masq
qdhcp-remote
patch
peer
patch
peerbr-tun
libreswan/vpnlibreswan/vpn
br-ex
br-tun
br-ex
libreswan/vpnqrouter-remote
patch
peer
patch
peer
qdhcp-remote
Network node 2Site B
Internet
br-int
Corresponding Neutron topology - Simple to create?
qrouter-remote
dns
masq
qr-xxqg-xxtapxqr-xxqg-xxtapx
qr-xxqg-xxtapxqr-xxqg-xxtapxqr-xxqg-xxtapx
qr-xxqg-xxtapx
qr-xxqg-xxtapx
qr-xxqg-xxtapx
dns
masq
dns
masq
dns
masq
6
7
5
6
7
5
4
br-int
5
7
8
5
7
8
7
6
8
7
6
8
5
6
8
5
6
8
8
1
2
3
2
3
1
4

31. Thank You
Questions?
sputhenp@redhat.com
twitter: @sadiquepp