Weitere Ă€hnliche Inhalte Ăhnlich wie OpenStack Neutron IPv6 Lessons (20) Mehr von Akihiro Motoki (18) KĂŒrzlich hochgeladen (20) OpenStack Neutron IPv6 Lessons1. Lessons in IPv6 deployment in OpenStack
environments
May 8, 2017
Akihiro Motoki
OSS Promotion Center, NEC
2. 2 © NEC Corporation 2017
Why IPv6?
IPv4 Address has run out!
Smart City and IoT needs lots
of addresses
All the Industry and Telecom has to prepare
IPv6 Era
3. 3 © NEC Corporation 2017
Agenda
âMany consideration points on IPv6 deployments with OpenStack
â[0] Understanding IPv6 words
â[1] IPv6 address allocation
ïŹHow to manage IPv6 address pools
ïŹGlobal unique addresses for tenants
â[2] IPv6 address configuration
ïŹHow to distribute IPv6 address to clients
â[3] Routing considerations
ïŹHow to reach tenantsâ network?
ïŹHow to prevent unauthorized global addresses?
â[4] VNF choices
ïŹNeutron router?
ïŹ3-rd party VNF router? Firewall?
âThis presentation is based on OpenStack Mitaka release
5. 5 © NEC Corporation 2017
IPv6 address allocation
âWhat we need?
âGUA (Global unique address) are assigned to tenants
âGUA should be assigned to a specified address range
âTenants still need to use their own addresses
ïŹUnless they are not connected to external
ïŹULA (Unique local address)
Neutron Subnet Pool
Legacy way to specify CIDR
(or tenant subnet pool)
6. 6 © NEC Corporation 2017
Neutron âsubnet poolâ
ââSubnet poolâ defines CIDR pool and allocates CIDR for a
subnet from the pool
âWhat we can?
ïŹNo need to care which CIDR is used or not. Subnet pool manages assigned
CIDR.
ïŹOpenStack operators pre-defines a desired IP address range
ïŹIt allows us to assign non-overlapping address ranges across tenants
Subnet Pool
Pool Prefixes
203.0.113.0/24
Prefix Length
default: 26
Max: 25, Min: 28
Subnet 1
203.0.113.0/26
Subnet 2
203.0.113.128/25
Subnet 3
203.0.113.64/27
neutron subnetpool-create
--pool-prefix 203.0.113.0/24 --default-prefixlen 26
--shared --is-default demo-subnetpool
neutron subnet-create --subnetpool demo-subnetpool --ip-version 4 demo-net
neutron subnet-create --use-default-subnetpool --ip-version 4 demo-net
neutron subnetpool-update
--pool-prefix 203.0.113.0/24 --pool-prefix 198.51.100.0/24 demo-subnetpool
7. 7 © NEC Corporation 2017
IPv6 address allocation â admin
âWe use âsubnet poolâ to manage IPv6 address ranges
âCreate a subnet pool for GUA
ïŹopenstack subnet pool create
--pool-prefix 2001:DB8:1234::/48
--default-prefix-length 64 --min-prefix-length 64 --max-prefix-length 64
subnet-pool-ipv6
ïŹSpecify the global address range as a pool prefix
ïŹ/64 is suggested as prefix length as most IPv6 routers assume 64 in RA
âMark it as âsharedâ so that tenants can consume it
ïŹopenstack subnet pool set --share subnet-pool-ipv6
â(optional) Mark it as a default subnet pool
ïŹ openstack subnet pool set --default subnet-pool-ipv6
ïŹTenants do not need to specify the subnet pool name when creating a subnet
8. 8 © NEC Corporation 2017
IPv6 address allocation - tenant
âCheck per-defined subnet pools
ïŹopenstack subnet pool list
âCreate a subnet using the subnet pool
ïŹopenstack subnet create --use-default-subnet-pool --ip-version 6
--ipv6-ra-mode XXX âipv6-address-mode XXX --network net1 subnet1
ïŹYou can specify a subnet pool explicitly but using a default pool would be easier
âQuota on subnet pool
ïŹSubnet pool supports quota
ïŹIn IPv6 case, it is calculated as the number of /64 subnets.
ïŹIf the subnet pool quota is 3, a tenant can allocate 3 /64 subnets.
âTenants still can specify their own address range
ïŹLimited to local use and no internet connection
10. 10 © NEC Corporation 2017
IPv6 address configuration modes
âThree configuration modes:
ïŹ SLLAC, DHCPv6 stateless, DHCPv6 stateful
âSLACC
ïŹIPv6 address of a client is configured based on RA (Router advertisement)
ïŹGateway is also configured.
ïŹOptionally, DNS information(if RFC6106), MTU and so on can be configured.
ïŹOnly /64 prefix is used
âDHCPv6 stateless
ïŹIPv6 address is configured based on RA.
ïŹOther information is retrieved via DHCPv6.
ïŹLooks used most commonly
ïŹOnly way to distribute DNS info before RFC6106
âDHCPv6 stateful
ïŹAll information is configured based on DHCPv6
ïŹThere is information that GW is not configured properly.
11. 11 © NEC Corporation 2017
IPv6 address configuration modes
âThe configuration mode is determined based on RA flag
ïŹM (Managed) => 0 (RA/DHCPv6 stateless), 1 (DHCPv6 stateful)
ïŹO (Other) => 0 (SLAAC), 1 (use DHCPv6)
âNeed to selection address mode, depending on router
implementation used
ïŹNeutron exposes all modes, but it is not necessarily all modes are available...
12. 12 © NEC Corporation 2017
Understanding Neutron IPv6 two modes
âTwo attributes related to IPv6 address configuration:
ïŹipv6_address_mode
ïŹipv6_ra_mode
âThere are constraints between two modes
ïŹOpenStack networking guide
ïŹhttps://docs.openstack.org/ocata/networking-guide/config-ipv6.html
13. 13 © NEC Corporation 2017
Understanding Neutron IPv6 two modes
14. 14 © NEC Corporation 2017
Understanding Neutron IPv6 two modes
A lot of combinations.
What does each mode mean?
15. 15 © NEC Corporation 2017
Understanding IPv6 two mode
âIPv6 address mode
ïŹSpecifies how IPv6 address is generated and assigned
ïŹIPAM is mainly involved in it.
ïŹAlso controls if the reference DHCP implementation serves
ïŹâslaacâ
âą Neutron generates a port address based on MAC address (EUI-64)
ïŹâdhcpv6-statelessâ
âą Neutron generates a port address based on MAC address (EUI-64)
âą Neutron provides DHCP options for port
ïŹâdhcpv6-statefulâ
âą Any address can be configured.
âą Perhaps non-/64 prefix can be used
ïŹNot Specified (N/S)
âą (Backward compatibility)
âą Any static address can be configured.
16. 16 © NEC Corporation 2017
Understanding IPv6 two mode
âIPv6 RA mode
ïŹSpecifies how neutron router sends RA
ââslaacâ
ââdhcpv6-statelessâ
ââdhcpv6-statefulâ
ïŹNeutron setup radvd on a router and provides RA
ïŹRA flags are set accordingly
âNot Specified (N/S)
ïŹNeutron does nothing.
ïŹradvd on a router is not setup
17. 17 © NEC Corporation 2017
Deployment patterns
â(1a) Tenant Router (OpenStack)
ïŹNeutron router; VNF integrated with neutron
â(1b) Tenant Router (VNF)
ïŹVirtual appliance; not integrated with neutron
â(2) Provider Router
VM
Upstream
Router
External
network
(1a) Tenant Router (OpenStack) (1b) Tenant Router (VNF)
VM
Neutron
network
Neutron
router
Upstream
Router
External
network
NW infra
VM
Neutron
network
VNF (vRouter,
vFW)
Upstream
Router
External
network
Managed by
OpenStack
VNF
manager
NW infra NW infra
(2) Provider router
Managed by
OpenStack
Managed by
OpenStack
18. 18 © NEC Corporation 2017
IPv6 two modes: tenant neutron router
âSimplest case!
âTenant router is implemented as neutron router
ïŹL3 agent (reference implementation)
ïŹOther 3rd party L3 plugin (if you use)
âNeutron provides both RA and DHCPv6
ïŹRef implementation: RA = radvd, DCHPv6 = dnsmasq
âIf 3rd party L3 plugin is provided, the behavior needs to be checked
ipv6_address
_mode
ipv6_ra_
mode
Description (L3 agent)
SLAAC SLAAC Radvd runs on netns and provides RA
DHCPv6
stateless
DHCPv6
stateless
Radvd on router netns and dnsmasq
on dhcp netns
DHCPv6
stateful
DHCPv6
stateful
Same as above
VM
Neutron
network
Neutron
router
Upstream
Router
External
network
NW infra
Managed by
OpenStack
19. 19 © NEC Corporation 2017
IPv6 two modes: tenant VNF router
âTenant router is implemented as virtual appliance
âBUT it is not well integrated with neutron (= no L3 plugin)
âSLLAC works well
âDHCPv6-stateless / stateful : neutron DHCP (dnsmasq) serves too
ïŹIf VNF provides DHCPv6, two DHCP servers works as a result
and they may conflicts...
ipv6_address
_mode
ipv6_ra_
mode
Description
SLAAC Off (N/S) VNF router sends RA
DHCPv6
stateless
DHCPv6
stateless
VNF router sends RA
neutron dnsmasq is configured
If VNF provides DHCPv6, the feature
potentially conflictsâŠOff
DHCPv6
stateful
DHCPv6
stateful
VNF router sends RA
neutron dnsmasq is configured
If VNF provides DHCPv6, the feature
potentially conflictâŠOff VM
Neutron
network
VNF (vRouter,
vFW)
Upstream
Router
External
network
Managed by
OpenStack
VNF
manager
NW infra
20. 20 © NEC Corporation 2017
IPv6 two modes: provider router
âRouter on a provider network sends RA.
âFrom the perspective of neutron, same as âtenant VNF routerâ case
âSLLAC works well
âDHCPv6 stateless : DHCPv6 by neutron and upstream router may
potentially conflict
âDHCPv6 stateful : neutron DHCPv6 is the only option?
ïŹ There is no way to share address information between provider router and neutron
VM
Upstream
Router
External
network
NW infra
(2) Provider router
Managed by
OpenStack
ipv6_address
_mode
ipv6_ra_
mode
Description
SLAAC Off (N/S) provider router sends RA
DHCPv6
stateless
DHCPv6
stateless
provider router sends RA
neutron dnsmasq is configured
If provider router provides DHCPv6,
the feature potentially conflictsâŠOff
DHCPv6
stateful
DHCPv6
stateful
provider router sends RA
neutron dnsmasq is configured
If VNF provides DHCPv6, address from
both will conflicts XXXOff
21. 21 © NEC Corporation 2017
In our case..
â(SLAAC)
ïŹNot many VNF supports DNS option in RA message
â(DHCPv6-stateless)
ïŹNeutron DHCPv6-stateless does not provide DNS.. Needs investigation
ïŹSome VNFs support SLAAC only. Cannot configure RA flags.
VNF SLAAC DHCPv6-stateless
Neutron router OK OK
Cisco CSR1000V OK (+DNS) OK (+DNS)
NEC Intersec VM/SG OK -
Paloalto VM-100 OK -
23. 23 © NEC Corporation 2017
Dynamic routing to tenant IPv6 subnets
âHow to reach tenantsâ network?
âTo deliver packets from the Internet to tenantsâ networks, the
upstream router must know routes to tenantsâ networks
âIn IPv6, tenant networks get GUA (global unique address)
dynamically
âRoute tables on the upstream router needs to be updated
âWe need some mechanism to update routes automatically
Self Service
Network 1
2001:0DB8:1:/64
Self Service
Network 2
2001:0DB8:2:/64
Self Service
Network 3
2001:0DB8:3:/64
Router1
Router2
Router3
Upstream
Router
Provider
Network
(router:
external)
Internet
24. 24 © NEC Corporation 2017
Dynamic routing to tenant IPv6 subnets
âDynamic Routing
ïŹBGP
ïŹOSPF
âIPv6 Prefix Delegation
âStatic Route (with some automation mechanism)
ïŹNeed some mechanism to monitor neutron-side changes
25. 25 © NEC Corporation 2017
Dynamic routing : BGP (neutron integration)
âNeutron provides BGP dynamic route advertisement
âCreate a BGP peering between BGP agent (dr-agent) and upstream
Router
âBGP agent advertises routes to tenant networks dynamically
âOnly networks whose address scope is same as that of the external
network are advertised. Networks on different address scope are not
advertised.
Self Service
Network 1
2001:0DB8:1:/64
Self Service
Network 2
2001:0DB8:2:/64
Self Service
Network 3
2001:0DB8:3:/64
Router1
Router2
Router3
Upstream
Router
Provider
Network
(router:
external)
Internet
Address ScopeL3 agent
BGP agent
BGP peering
AS 4321
AS 1234
http://docs.openstack.org/newton/networking-guide/config-bgp-dynamic-routing.html
26. 26 © NEC Corporation 2017
Dynamic routing : OSPF
âOSPF can be used for dynamic routing
âCreate router relationships between tenant and upstream routers
when a tenant router is created
âOnce a relationship is established, a route to a tenant network is
configured to the upstream router
âUseful for small network where BGP is not preferred
âMost VNF router supports OSPF
âNo neutron integration
Self Service
Network 1
2001:0DB8:1:/64
Self Service
Network 2
2001:0DB8:2:/64
Router1
(VNF)
Router2
(VNF)
Upstream
Router
Provider
Network
(router:
external)
Internet
Router relationship
AS 4321
http://docs.openstack.org/newton/networking-guide/config-bgp-dynamic-routing.html
27. 27 © NEC Corporation 2017
IPv6 Prefix Delegation (PD)
âUpstream router is a PD (prefix delegation) server and this manages
IPv6 address ranges to be assigned to OpenStack tenant networks
âPD server assigns CIDR (normally /64 prefix) to PD client
âNeutron router acts as a PD client
âUpstream router sets up a route to PD client when assigning a prefix
ïŹ The upstream router knows an external IP address of Neutron router (PD client)
ïŹ LLA (Link local address) can be used as IP address of PD client
âNeutron integration
Self Service
Network 1
2001:0DB8:1:/64
Self Service
Network 2
2001:0DB8:2:/64
Self Service
Network 3
2001:0DB8:3:/64
Router1
Router2
Router3
Upstream
Router
Provider
Network
(router:
external)
Internet
PD Server
http://docs.openstack.org/newton/networking-guide/config-ipv6.html#prefix-delegation
PD Client
Assign CIDR
28. 28 © NEC Corporation 2017
Comparison
Choices depend on network policy and router types to be used
âDynamic routing (BGP)
ïŹNeutron integration
ïŹDepending on network policy of upstream network
ïŹSome operators does not use BGP inside their network
ïŹOne AS is required for OpenStack deployment
âDynamic routing (OSPF)
ïŹNo neutron integration
ïŹOSPF is used for smaller deployment
ïŹMost VNF router supports OSPF
âPrefix delegation
ïŹNeutron integration
ïŹOnly simple topology is supported.
ïŹCannot handle nested tenant router
ïŹNot a small number of VNF does not support prefix delegation
30. 30 © NEC Corporation 2017
How to prevent unauthorized global addresses?
âNeutron allows tenants to assign arbitrary IP addresses.
âIn IPv6, global unique addresses are assigned to tenants from the
predefined ranges.
âIf some tenant assigns overlapping address ranges with other
tenant, what happens?
âThere needs a way to block unauthorized global unique address
assigned by tenants in their own way.
31. 31 © NEC Corporation 2017
Neutron address scope
âAddress Scope
ïŹConcept to define which IP addresses can directly communicate each other
ïŹSubnet pool is associated to some address Scope
ïŹRouter allows traffic among a same address scope
âą Router identifies an address scope of each router interface
Net1
R
Net2
Net3
Networks on a same
address scope can
communicate
Networks in different
address scope cannot
communicate
Subnet
pool 1
Address
Scope 1
Subnet
pool 2
Address
Scope 2
or CIDR (If CIDR is specified,
âImplicit Address
Scopeâ is used)
32. 32 © NEC Corporation 2017
How to prevent unauthorized global addresses?
âBy using address scope, administrators can only allow
traffic from IP address ranges they authorize.
âAdministrator
ïŹ Create a address scope
ïŹ Create a subnet pool for external communication
ïŹ Associate the subnet pool with the above address scope
ïŹ Create a subnet of external network from the above subnet pool
âTenant user
ïŹ Create a subnet from the shared subnet pool (when communicating externally)
ïŹ Subnet whose CIDR is specified explicitly cannot communicate with the Internet even if
it has the same CIDR.
Ext-Net
R
Net2
Net3
Networks on a same
address scope can
communicate
Networks in different
address scope cannot
communicate
Subnet
pool 1
Address
Scope 1
Subnet
pool 2
Address
Scope 2
(or CIDR)
External Network
33. 33 © NEC Corporation 2017
How to prevent unauthorized global addresses?
âCreate a address scope
ïŹopenstack address scope create --share --ip-version 6 address-scope-ip6
âCreaet a subnet pool for external communication
âAssociate the subnet pool with the above address scope
ïŹopenstack subnet pool create --address-scope address-scope-ip6
--share --pool-prefix 2001:db8:a583::/48 --default-prefix-length 64
subnet-pool-ip6
âCreate a subnet of external network from the above subnet pool
ïŹopenstack subnet create --subnet-pool subnet-pool-ip6
--ip-version 6 --disable-dhcp --network external-network
external-subnet-ip6
âCreate a subnet from the shared subnet pool
ïŹopenstack subnet create --subnet-pool subnet-pool-ip6
--ip-version 6 --ipv6-address-mode slaac --ipv6-ra-mode slaac
--network my-network my-subnet-ip6
35. 35 © NEC Corporation 2017
Additional topics
âFor SLAAC/DHCPv6 stateless, /64 is actually the only unit assigned
to tenant
ïš The required number of /64 prefix will be the number of tenants
âAll IPv6 modes cannot be used necessarily, but neutron exposes all
ïš Some mechanism to expose available IPv6 mode to users
âWindows supports address randomization for EUI-64 address
ïŹIt randomizes the lower 64 bits of addresses
ïŹEphemeral address is also supported
ïŹIt works ..
ïŹbut a generated IPv6 address will be different from Neutron port information
âMost VNF does not provide Neutron L3 plugin
ïš Need a way to retrieve information from neutron
âą IPv6 RA flag (Managed, Other)
âą Address scope
Prefix (64) EUI-64 (64)
randomize
36. 36 © NEC Corporation 2017
Summary
âThis presentation share our experiences on IPv6
âHope it helps introductions of IPv6 deployment
âMost things work well.
âThere are several things remaining to be improved
âLetâs upstream it!
ïŹ Feel free to file a bug to neutron!
âLetâs share our knowledge!
ïŹ OpenStack Networking Guide is a good place
ïŹLatest release : https://docs.openstack.org/ocata/networking-guide/
ïŹDevelopment version : https://docs.openstack.org/draft/networking-guide/