1. Matt Griffin
George Lorch
Tom Diederich
January 15, 2015
Cloud Online Meetup:
High performance MySQL in an
OpenStack cloud

2. www.percona.com
Agenda
● MySQL guests in the cloud
● Benchmarking the MySQL core of OpenStack
● Supporting the OpenStack community
Questions: In ReadyTalk
Slides: bit.ly/percona-jan2015-cloud-online-meetup

3. www.percona.com
About us
Percona delivers high performance MySQL to
our clients and the community.
Open Source Software
● Percona Server
● Percona XtraDB Cluster
(Galera)
● Percona XtraBackup
● Percona Toolkit
100% MySQL-compatible
MySQL Services
● Support
● Consulting
● Managed Services
Community
● Events: Percona Live,
OpenStack Live, Percona
University
● MySQL Performance Blog
● Related communities

4. www.percona.com
Percona & OpenStack
MySQL is the top database
for OpenStack services
MySQL is one of the top
use cases for guests in the
cloud
We have performance
expertise to share with the
developer community and
production users
OpenStack User Survey results, November 2014
http://superuser.openstack.org/articles/openstack-user-
survey-insights-november-2014

5. www.percona.com
MySQL Guests in the Cloud
(Trove DBaaS)

6. www.percona.com
Utility user
Needed a MySQL user to perform specific management
functions
Potential risk in an actual user making changes to this user
Created a special type of MySQL user in Percona Server
designed to
● Do specific tasks
● Be protected/hidden
Currently only used by HP Helion

7. www.percona.com
Options modifiers
Operators can set limits on MySQL configuration changes
to reduce their risk.
e.g., maximum-query_cache_size=4M
Percona extended MySQL’s five included options with:
● minimum
● hidden
● readonly
Enhanced oversight for cloud operators over guest DBs
Available now when using Trove with Percona Server

8. www.percona.com
Enforcing a storage engine
InnoDB is most likely the best choice… but many other
options exist
An operator can now lock down
End result is enhanced consistency and reliability in guest
environments
Available now when using Trove with Percona Server

9. www.percona.com
Security and Capacity
Can disable LOAD DATA INFILE and SELECT INTO
OUTFILE
● Both potentially high risk resources to interact with the
file system
● Percona extended –secure-file-priv option
Added Binary and Slow Log size and rotation
● Set tighter control in Percona Server over how database
logs behave and the disk capacity they consume per
instance
Both available now when using Trove with Percona Server

10. www.percona.com
Percona Server + XtraBackup
Percona Server 5.6 with Percona XtraBackup 2.2.x
enables:
● Changed page tracking
● Reduced locking during backups
Enabled by default when deploying a Trove image
containing Percona Server and Percona XtraBackup

11. www.percona.com
Percona XtraBackup
Additional benefits of Percona XtraBackup
● Reduced disk usage with enhanced compression
● Higher security with AES 64-256 encryption
● Simpler backups administration
Coming soon
● Asymmetric/public key encryption
● Streaming of backups to Swift (currently in Alpha)

12. www.percona.com
Additional Resources
OpenStack Summit Paris
Percona Server features for OpenStack and
Trove Ops
George Lorch, Percona
Vipul Sabhaya, HP

13. www.percona.com
Benchmarking the MySQL Core of
OpenStack

14. www.percona.com
Objectives
Examine the database communication and load in an
OpenStack environment
Identify database-layer bottlenecks at various levels of
concurrency
Understand the query load placed against the database
server during typical requests to Nova and Neutron

15. www.percona.com
Benchmarking Setup
Initial setup: 7 AWS instances
● 1 controller nodes
● 1 compute worker node
● 1 network node
● 3 PXC nodes
● 1 Rally benchmark node
c3.8xlarge instance type except
compute
● 24 cores / 60GB RAM / SSD
ephemeral drives
Updated setup: 16 AWS instances
● 1 controller node (we want more
in the future)
● 10 compute worker nodes
● 1 network + queue node
● 3 PXC nodes
● 1 Rally benchmark node
c3.8xlarge instance type except
compute
● 24 cores / 60GB RAM / SSD
ephemeral drives
c3.large
● 2 cores / 3.75GB RAM / SSD
ephemeral drives

16. www.percona.com
Load Tests
Boot 10,000 servers
● Some increase in response
time
● Some errors (Galera
certification timeouts)
● Database wasn’t the
bottleneck
Then boot/delete 5,000
more servers
● Writing to several nodes
significantly improves
performance
● Some services retry on
certification timeouts (e.g.,
@_retry_on_deadlock in
Nova) but some don’t
● Most operators will probably
not see timeouts in
production

17. www.percona.com
Lessons Learned I
● SELECT … FOR UPDATE can be an issue for Galera
clusters powering OpenStack
● Hidden behind ‘deadlock’ error
● Better performance when combining Network and
Queue node
● Use pt-query-digest (in Percona Toolkit) to analyze
workloads. “Options” very helpful to filter output. Look
for:
● Slowest transactions (default output)
● Large transactions

18. www.percona.com
Lessons Learned II
● Database is not the bottleneck
● Controller node’s API functions are saturated earlier
with maxed out CPU but little stress on the database
● Likely many opportunities for database optimizations for
production clouds given identified query-level
bottlenecks
● Targeted indexes will help
● Continue examining OpenStack’s MySQL workload
across services and with future releases
● Future patches coming

19. www.percona.com
Additional Resources
OpenStack Summit Paris
MySQL and OpenStack Deep Dive
Peter Boros, Percona
Jay Pipes, Mirantis
Ansible playbooks: Set up your own OpenStack benchmarking cluster
on AWS
“OpenStack users shed light on Percona XtraDB Cluster deadlock
issues” by Peter Boros
“Understanding reservations, concurrency, and locking in Nova” by Jay
Pipes

20. www.percona.com
Supporting the OpenStack
Community

21. www.percona.com
Contributions in Progress
● Trove clustering
● HA Guide update
● MySQL Performance Blog: OpenStack advice based on
our MySQL expertise
● Peter and Jay’s Ansible playbooks are evolving

22. www.percona.com
Hands-on tutorials covering
technologies including:
● Trove (DBaaS)
● Barbican (security)
● Ceph (storage)
● Neutron (networking)
Sessions on topics such as:
● Rally benchmarking
● Trove in the real world
● Stories from enterprise users
like Time Warner Cable
● Cloud Foundry PaaS
Early Bird registration ends soon
http://www.percona.
com/live/openstack-live-2015/

23. www.percona.com
Questions
Matt Griffin matt.griffin@percona.com
George Lorch george.lorch@percona.com
Tom Diederich tom.diederich@percona.com
Slides: bit.ly/percona-jan2015-cloud-online-meetup