Tổng hợp các điểm mới trong OpenStack Liberty

1. Tokyo Summit Recap
Nova
Liberty known issues
 neutron LBaaS does not work with Heat autoscaling
 keystone v3 has a lot of undocumented config options
 nova conductor scaling is worst than expected.
 Nova switched to use pymysql driver in Liberty
 glance client broken with urllib3
Kilo known issues
 The release notes have quite big holes. Things that was deprecated in Juno and removed in Kilo
are not documented
 Global issues:
 Neutron-client was pinned to a wrong version
 upper constrains requirements are provided! This is the last known requirement version
that works with Kilo.
 what-broke tool that helps finding new pypi releases that might broke our unit test
 Upgrading Cinder is hard because of database changes
Nova major issues:
These issued are listed in: https://review.openstack.org/#/c/240959/7/priorities/mitaka-priorities.rst,cm
1. Cells V2
Use case: Operators want to partition their deployments into cells for scaling, failure domain, etc.
When partitioned, we need a way to route queries to the cell database for an instance.
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/cells-db-connection-
switching.html
 Priorities:
 Cell v2 will become default in Mitaka version.
 Upgrade path from v1 to v2 and multiple cells support in the next OpenStack version .
 The requirement of Cell API is that it should show the instance before it is scheduled to a
specific cell, therefore the information of the instance has to be persisted somewhere. From the
Kilo version, scheduler uses RequestSpec that contains the information of spawning instance so
that RequestSpec must be persisted. Besides, we can persist the BuildSpec instead of
RequestSpec.

2.  Instead of using nova-cell proxy, nova-api directly connect to the cell db and cell queue for each
instance. CellMapping contains the connection information of the cell. The interaction between
DB layer (master cell db) and CellMapping mus be dynamically provided. The situation is the
same with message queue.
 Flavor is a global concept that should be stored in only one database. Since the information of
flavor lives in Cell API so that tables related to flavor should be created in API database.
2. V2.1 API
 Priorities:
 The lack of document of the V2.1 API
 API concept guide
 API code review guideline
 Support improved service catalog
 make project_id optional in every API
 API V2.1 is default in Liberty for all API endpoints
3. Scheduler
 Request spec object implementation is finished in Liberty
 The idea that scheduler might be detached from Nova architechture to be run as a
separate service so that the below aspects are created to support this long run:
 The resource allocation happens on the compute node, in the future, this action will take part in
scheduler too.
 In current versions of OpenStack, the compute decides itself that what resource is free, used,
which host is fit for a specific vm. In the future, those works will belong to scheduler.
 Basically, a flavor is a composition of resource and capability needs. If we need to add new
capability or new resource type, it means we create new flavors therefore the number of flavors
is growing rapidly. Instead of dealing with raw information in flavor when booting a vm, we
decompose flavor to get the specific resources for scheduler interface.
 Initiating resource object will make easier to add new resource (e.g. numa topology) into the
scheduler interface. By decomposing flavor, the new resource will be taken from resource
object.
 It needs a new REST API that can get the capabilities of a cloud (e.g. disk, gpus)

3. 4. OS VIF Lib
https://review.openstack.org/#/c/193668/3/specs/liberty/approved/os-vif-library.rst,unified
- Create a new library (actually an interface) containing all the VIF drivers that will make Neutron and
Nova more easily work together.
- Implement a new VIFConfig abstract base class as a versioned object using oslo.versionedobjects:
 Neutron should return the port as either VIFConfig object instance or the legacy dict format.
 The lib interface contains only an init, plug, unplug method.
 xml generation can be handled by oslo.versionedobject serialization
Nova minor issues
SRIOV
Nova-compute needs to know which PCI devices are allowed to be passed through to the VMs.
Also for SRIOV PCI devices it needs to know to which physical network the VF belongs to
 Link: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking-Liberty
Cinder - Nova
Volume multi-attach :
 Currently, Nova only allows a volume to be attached to one instance and or host at a time. Nova
makes an assumption in a number of places that assumes the limitation of a single volume to a
single instance.
 Use case: Clustered applications with two nodes where one is active and one is passive. Both
require access to the same volume although only one accesses actively. When the active one
goes down, the passive one can take over quickly and has access to the data.
https://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/multi-attach volume.html
Nova - Ironic interactions
 Changes from a single compute model to a multi compute model in order to avoid single point
of failure. A single ironic node will belong to a nova compute node so that migration and
evacuation has no effect
 Nova-compute will report the resource usage of resource providers. But for Ironic, resources are
not elastic, it is static and it may have chunks of resources types (disks, ram, etc.) that map to
the underlying hardware of Ironic nodes. The resource pool concept will solve this problem in
the future.
 Nova scheduler will only schedule based on compute resource count, ironic needs to do the real
scheduling.

4. Nova small issues
 Nova has an instance locked by field but external service wants to lock other things, keypair,
snapshot
 An instance needs to prove others who it is. Metadata is insecure, config drive is not flexible
enough to solve key rotation.
 Live-migrate issues are fixed.
 The third mode of shelf: Previously, shelf has two modes, normal (associated data, resources are
reserved and kept) and offload (resources are released and instance disk is delelted). The
proposal for the third mode is about: Release resource but keep the instance disk.
 Supports shelving and unshelving of instances in Horizon.
https://blueprints.launchpad.net/horizon/+spec/horizon-shelving-command
 Mix and Match resource federation:
https://blueprints.launchpad.net/nova/+spec/mix-and-match-resource-federation
- Multiple landlords cloud model where multiple services providers can cooperate to
stand up in a single cloud environment.
- Use case of Nova and Cinder that means an instance can be booted in an OpenStack
deployment with volume from another OpenStack deployment. In this case, Nova
needs to talk with different cinder, local keystone needs to talk with remote
keystone, uuid token should be globally unique.
 Hierarchical quota: A sub-project shall be able to get quota from the main project (duplicate or
split). Cinder already implemented it so nova needs to adapt.
https://blueprints.launchpad.net/nova/+spec/nested-quota-driver-api
 Nova image signature verification: Glance shall provide a checksum along with the image,
glance message needs to be signed with a CA. Finally, nova needs to check the checksum at
booting instance.
 Nova load balancer: It can help dinamically move VMs between computes.
https://wiki.openstack.org/wiki/Watcher
 Centralized config definitions: Today the nova config option definitions are spread around in
implementation files. Let's move them to a central place.
 SRIOV attach interface: Nova interface-acttach/detach can not support SRIOV neutron port.
https://blueprints.launchpad.net/nova/+spec/sriov-interface-attach-detach
CrossProject:live upgrade (Source:Ericssonsum up after Mitaka submit)
 Dan Smith (nova core) has a comprehensive set of blog psts about openstack upgrade
 There are four pillars of the live upgrade:
 RPC

5.  Understanding multiple version of the same rpc api in the same deployment so
you can upgrade your services separately
 define compatible versions to limit the possible combinations
 client shall be able to send old message version if told to, server shall understand
the old version
 order the services for upgrade (to limit the amount of combinations)
 Data facade:
 define the schema of the data you are passing around
 db schema shall be independent
 this layer can handle version differences
 DB schema:
 Do not allow shrinking db schema changes
 Allow the schema change to happen any time
 Data facade shall handle data migration
 Removing things needs extra care: data facade moves the data, next code revision
removes the field from the db model, then the db schema can be shrinked
 Validation:
 test that old client can talk to the new server
 in nova gate: jenkins starts liberty, do smoke test, populate the system with data,
shut down everything except nova-compute, roll out the new patch under test,
start the services up, run full tempest.
 in nova gate: multi node test is experimental but shall be ready in Mitaka, it will
allow testing even more service separation during upgrade
 Nova supports upgrade between one release because data migration needs to have a well
defined scope
 Always move to the latest stable release before the upgrade (bug fixes for the upgrade code)
 This should be easy as stable releases cannot contain db schema change and rpc version
change
 ironic, cinder, neutron, ceilometer needs to implement this support to have a meaningful stack
upgrade
Oslo.config forconfiguration changes
 oslo.config already supports rereading the config file at SIGHUP however current services are
not prepared for this kind of config change.
 oslo.config shall support marking mutable config variables as changeable and only reload those
that are marked mutable by the service. The config sample generator shall be able to put this
information in the config sample too.
 Services shall start marking the changeable parameters in the code

6. Guru Mediation Report
 http://docs.openstack.org/developer/nova/gmr.html
 https://wiki.openstack.org/wiki/GuruMeditationReport
Off-Topics:
OPS billing
Alternatives:
 ceilometer
 gnochi
 cloud kitty
 ceilometer with monasca backend for low-lever querying
OPS quota
 A more fine-grained quota system that can measure many types of resources like objects,
flavors, availability zones, etc. needs to be built and implemented.
 Nested quota to support sub project shared quota. Cinder implemented it already.
 https://wiki.openstack.org/wiki/CinderBrick
Distributed locking
 Main problem: OpenStack needs a common distribute lock solution. As today the db, and the
message queue is not used for it.
 No single solution is acceptable for the community
 Needs:
 service discover
 locking to have single writers
 leader election (out of scope of distribute lock)
 load sharing between workers (re-queue the workpiece if a worker dies, ensure that
resurrected worker does not resume the workpiece)
 fair locking (starvation)
 Zookeeper is mature but is java based, some operator does not want to use java in their
environment as well as openjdk might not scale and oracle has licensing issues.
 Tooz is a potential candidate as an abstraction layer to provide distributed locking. Zookeeper
will be the default backend but other can be added
 Need make a plan how to move from today's solutions to Tooz

7. NEUTRON
Trunk Port
IT is an important feature for telco applications that have the need of multiple networks. A new spec
was taken about new abstraction of trunk port in Neutron. It is now under the review of API
alternatives:
https://review.openstack.org/#/c/243786/
Prospective Issues:
- SFC (Service Function chaining)
- Kuryr and nested containers in Kuryr.
Other Issues
Performance Testings
https://etherpad.openstack.org/p/mitaka-cross-project-performance-team-kick-off
There are several tools for perf testing: Rally, Gatling, Zipkin, Locust, etc.
Rally nowadays is the most popular tool but there is no standard benchmarking solutions/toolsfor
OpenStack. We have 2 most usecases of testings that are performance testing and scale testing.
Mirantis is using Rally for both of cases. Check the etherpadd for more information.
Nova/Neutron:OS VIF lib
Since there are more new neutron drivers so that new vif types are needed. However, now the vif
drivers are in the nova code tree therefore it is needed to take them out into their own library. See more
details in Nova OS VIF lib.
Neutron: Scalability, Reliability Pain Points
https://etherpad.openstack.org/p/mitaka-neutron-next-ops-painpoints
Neutron: Extending Network Model
https://etherpad.openstack.org/p/mitaka-neutron-next-network-model
The idea is about change the Neutron API from the architect of Net-Subnet to Net-IpNetwork-Subnet.
The use case can be like this: What if people do not care about the network where the vm is attached.
They just care about the ip range in which the vm is assigned from. In the large deployment, we are

8. going to have a lots of networks so that remembering the network where the vm is attached to is
somehow superfluous. That is the reason why the vm should be attached to IpNetwork and then
automatically assigned to a Net.
Cross-Project:Distributed Lock Manager
https://etherpad.openstack.org/p/mitaka-cross-project-dlm
Since OpenStack is a distributed system so that it needs a distributed lock manager (DLM). Each sub-
project of OpenStack has its own locking and its own solution. What if some others projects would
want to have locking but would rather not create their own solutions? Does OpenStack accept a single
DLM for all of sub-projects. If YES, what is the solution. Zookeper, etcd, Consul are all the potential
candidates with their own strengths and weaknesses.
Cross-Project:Dynamic reconfiguration
Before Liberty, oslo.config was created to be capable of re-reading changes of configurations. But up
to now, no any OpenStack services use it. In daemon running, log changes are ignored by SIGHUP but
oslo.config can be responsible for it.
Other interesting projects
 OVN: blog entry on OVN
 Kuryr: Kuryr on github
 Dragonflow: SDN controller in 3 kLOC of Python
 BGPVPN
 Neutron L2 Gateway
.

9. KEYSTONE
Keystone Federation
- Add k2k plugin into keystoneclient: https://review.openstack.org/#/c/207585/
- People are trying to make use of openstack-client to use keystoneV3 since it does not support many
services of OpenStack. A cloud config file should be given for user to be easy of switching among the
cloud deployments.
http://docs.openstack.org/developer/python-openstackclient/configuration.html#configuration-files
- From Kilo, Service Provider idea was initiated and now in K2K it should be limited due to the reasons
of security, performance, etc. in large cloud deployment. The filters should be provided to enable only
the associating service providers with projects where the user is operating.
https://review.openstack.org/#/c/188534/
Token and Tokenless:
- Since Fernet has some problems about performance but it is planned to be the default token type in
Mitaka.
- Fernet token information: http://dolphm.com/openstack-keystone-fernet-tokens/
Cross-project:
Based on the topics of: https://etherpad.openstack.org/p/keystone-mitaka-summit-x-project
- LDAP implementations: People are putting more efforts to make LDAP more mature.
- Custon TTL on tokens for the long term operations: auth_token makes token validation request to
keystone as GET /v3/auth/token?ttl=123456. Keystone receives token validation request, ignores the
expiration in the token, re-calculates the expiration based on actual created_at + ttl, and uses that to
perform validation.

10. Reference Links:
https://github.com/openstack
http://accelazh.github.io/openstack/Openstack-Tokyo-Summit-Notes/
http://blog.openattic.org/posts/conference-report-openstack-summit-2015-tokyo-japan/
http://www.solidfire.com/blog/openstack-summit-tokyo-recap-containers-cloud-native-and-a-dash-of-
cinder/
https://wiki.openstack.org/wiki/Design_Summit/Mitaka/Etherpads