Weitere ähnliche Inhalte Ähnlich wie MySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell (20) Mehr von Miguel Araújo (13) Kürzlich hochgeladen (20) MySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell1. Copyright © 2019 Oracle and/or its affiliates.
MySQL InnoDB Cluster
Management and troubleshooting
using MySQL Shell
Senior Software Developer
MySQL Middleware and Clients
September 18, 2019
Miguel Araújo
2. The following is intended to outline our general product direction. It is intended for information purposes
only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing decisions. The development,
release, timing, and pricing of any features or functionality described for Oracle’s products may change
and remains at the sole discretion of Oracle Corporation.
Statements in this presentation relating to Oracle’s future plans, expectations, beliefs, intentions and
prospects are “forward-looking statements” and are subject to material risks and uncertainties. A detailed
discussion of these factors and other risks that affect our business is contained in Oracle’s Securities and
Exchange Commission (SEC) filings, including our most recent reports on Form 10-K and Form 10-Q
under the heading “Risk Factors.” These filings are available on the SEC’s website or on Oracle’s website
at http://www.oracle.com/investor. All information in this presentation is current as of September 2019
and Oracle undertakes no duty to update any statement in light of new information or future events.
Safe Harbor
Copyright © 2019 Oracle and/or its affiliates.
4. Copyright © 2019 Oracle and/or its affiliates.
Miguel Araújo
• MySQL Developer since 2011
• AdminAPI Tech Lead @ MySQL Shell team
• MySQL InnoDB Cluster evangelist
• Living in Portugal
• http://about.me/miguel-araujo
• @M1guelAraujo
5. 1.
2.
3.
4.
5.
Copyright © 2019 Oracle and/or its affiliates.
Introduction to InnoDB cluster
Node Provisioning
Monitoring
Maintenance and Troubleshooting
Final thoughts and resources
What’s Ahead
7. Copyright © 2019 Oracle and/or its affiliates.
• MySQL Group Replication
• MySQL Shell
• MySQL Router
InnoDB cluster
“A single product – MySQL – with high
availability and scaling features baked in;
providing an integrated end-to-end solution
that is easy to use.”
8. One Product:
MySQL
1
Copyright © 2019 Oracle and/or its affiliates.
MySQL InnoDB Cluster Goals
• All components
developed and integrated
together
• Full stack testing
9. One Product:
MySQL
1 2
Copyright © 2019 Oracle and/or its affiliates.
MySQL InnoDB Cluster Goals
• All components
developed and integrated
together
• Full stack testing
• One client: MySQL Shell
• Easy packaging
• Integrated Orchestration
• Homogeneous servers
Easy to use!
10. One Product:
MySQL
Flexible and
Modern
1 2 3
Copyright © 2019 Oracle and/or its affiliates.
MySQL InnoDB Cluster Goals
• All components
developed and integrated
together
• Full stack testing
• One client: MySQL Shell
• Easy packaging
• Integrated Orchestration
• Homogeneous servers
Easy to use!
• SQL and noSQL together:
X Protocol
• Protocol Buffers
• Asynchronous API
• Developer friendly
11. !
Copyright © 2019 Oracle and/or its affiliates.
MySQL InnoDB Cluster Goals
Completely Open Source --- GPL
No license required to have High Availability!
12. • Interactive multi-language console interface that supports
development and administration for the MySQL Server:
• Supports JavaScript, Python and SQL
• Interactive and batch operations (scripting)
• Supports Document and Relational models (X and MySQL protocols)
Copyright © 2019 Oracle and/or its affiliates.
MySQL Shell
13. • Exposes full Development and Administration APIs:
• Scriptable “DevOps” APIS
• Unified interface for developers and DBAs
Copyright © 2019 Oracle and/or its affiliates.
MySQL Shell
14. • Creation and Management of InnoDB clusters
• Available in both JavaScript and Python; and command-line
• Simple and straight-forward
• Flexible, powerful and secure!
Copyright © 2019 Oracle and/or its affiliates.
MySQL AdminAPI
15. Copyright © 2019 Oracle and/or its affiliates.
Creating a Cluster
// Configure servers for MySQL InnoDB cluster usage
mysqlsh-js> dba.configureInstance(“clusteradmin@mysql1:3306”)
16. Copyright © 2019 Oracle and/or its affiliates.
Creating a Cluster
// Configure servers for MySQL InnoDB cluster usage
mysqlsh-js> dba.configureInstance(“clusteradmin@mysql1:3306”)
// Connect to target instance
mysqlsh-js> c clusteradmin@mysql1:3306
17. Copyright © 2019 Oracle and/or its affiliates.
Creating a Cluster
// Configure servers for MySQL InnoDB cluster usage
mysqlsh-js> dba.configureInstance(“clusteradmin@mysql1:3306”)
// Connect to target instance
mysqlsh-js> c clusteradmin@mysql1:3306
// Create the cluster
mysqlsh-js> var cluster = dba.createCluster(“CodeOne19”)
18. Copyright © 2019 Oracle and/or its affiliates.
Creating a Cluster
// Configure servers for MySQL InnoDB cluster usage
mysqlsh-js> dba.configureInstance(“clusteradmin@mysql1:3306”)
// Connect to target instance
mysqlsh-js> c clusteradmin@mysql1:3306
// Create the cluster
mysqlsh-js> var cluster = dba.createCluster(“CodeOne19”)
// Add instances to the cluster
mysqlsh-js> cluster.addInstance(“clusteradmin@mysql2:3306”)
19. • Transparent client routing between the application and the
backend servers:
ü Application connection failover
• Native support for InnoDB clusters
• Stateless
Copyright © 2019 Oracle and/or its affiliates.
MySQL Router
21. • Provisioning a server is the process of configuring and getting it
ready to be used in an InnoDB cluster.
• The server needs to have all clusters’ data to be able to join it
• That’s typically done by:
1. Performing a backup of one of the cluster members
2. Restoring the image in the new member that wants to join
Copyright © 2019 Oracle and/or its affiliates.
Provisioning
22. • Distributed recovery is a process that runs when a new member is
added to the group and has missing transactions
• A donor that can provide the missing data is automatically selected
• The catch-up happens transparently and the server is
synchronized with the rest of the cluster
Copyright © 2019 Oracle and/or its affiliates.
Distributed Recovery
23. • Incremental recovery fetches only the missing data based on the
GTIDs into
- The cluster must be able to provide all the missing data
• Relies on the classic Asynchronous Replication
• Also provides group transactional information
Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
24. Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
Some data is inserted in
the DB
25. Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
The data modifications are
stored in the binlog.
And the transactions are
delivered to every cluster
member
26. Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
Group Replication is
eventually synchronous.
Guarantees that
transactions are delivered
to all members in the
same order.
27. Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
The binlogs keep track of
all data modifications.
28. Incremental Recovery to Provision
A new member wants to
join the cluster
Copyright © 2019 Oracle and/or its affiliates.
29. Incremental Recovery to Provision
A new member wants to
join the cluster
The missing data is
fetched and the server
synchronizes with the rest
of the cluster
Copyright © 2019 Oracle and/or its affiliates.
30. Incremental Recovery to Provision
A donor is automatically
selected to fetch the
missing data from, using
plain Asynchronous
Replication
Copyright © 2019 Oracle and/or its affiliates.
31. Incremental Recovery to Provision
A donor is automatically
selected to fetch the
missing data from, using
plain Asynchronous
Replication
Copyright © 2019 Oracle and/or its affiliates.
32. Incremental Recovery to Provision
A donor is automatically
selected to fetch the
missing data from, using
plain Asynchronous
Replication
Copyright © 2019 Oracle and/or its affiliates.
33. Incremental Recovery to Provision
The instance successfully
joins the cluster!
Copyright © 2019 Oracle and/or its affiliates.
34. Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
!May take a considerable amount of time
35. • But what if the cluster members lost all binary logs?
• Or even... Just some data is missing that is not
available on any binary log
• What about space constraints?
Copyright © 2019 Oracle and/or its affiliates.
Incremental Recovery to Provision
?
36. Incremental Recovery to Provision
Some binary logs were
purged.
And the instance has no
data yet.
Copyright © 2019 Oracle and/or its affiliates.
37. Incremental Recovery to Provision
Some binary logs were
purged.
And the instance has no
data yet.
Failed attempt to join the
cluster!
Copyright © 2019 Oracle and/or its affiliates.
38. • Performs physical snapshots of databases
ü Really fast!
• Transfers the snapshots over the network to provision servers
ü Can be used remotely!
• Replaces all the server data
Copyright © 2019 Oracle and/or its affiliates.
MySQL Clone Plugin
!Available since 8.0.17
39. Distributed recovery = initial provisioning + online catch-up
• Seamless and automated solution for provisioning
• No external tools or scripting required
Copyright © 2019 Oracle and/or its affiliates.
Clone to Provision
40. Common issues easily solved with clone:
• Very large datasets
• Incomplete binary log history
• Re-provision diverged instances
Copyright © 2019 Oracle and/or its affiliates.
Clone to Provision
41. Clone to Provision
Some binary logs were
purged.
And the instance has no
data yet.
Copyright © 2019 Oracle and/or its affiliates.
42. Clone to Provision
Some binary logs were
purged.
And the instance has no
data yet.
Clone required !
Copyright © 2019 Oracle and/or its affiliates.
43. Clone to Provision
A donor is automatically
selected.
And a full snapshot of its
data is performed.
Copyright © 2019 Oracle and/or its affiliates.
44. Clone to Provision
The instance successfully
joins the cluster!
Copyright © 2019 Oracle and/or its affiliates.
47. Copyright © 2019 Oracle and/or its affiliates.
• Incremental recovery is possible if:
• At least 1 member can provide the joiner
with the same set of transactions it has
processed.
• The joiner does not have errants
transactions
• Incremental recovery appropriate if:
• No purged transactions
• Joiner does not have an empty GTID set
• GTID support and binary logs were always
enabled
Which recovery shall be used?
48. Copyright © 2019 Oracle and/or its affiliates.
AdminAPI Guidance
Incremental recovery safe and appropriate:
49. Copyright © 2019 Oracle and/or its affiliates.
AdminAPI Guidance
Clone-based recovery required:
50. • Built-in support for full instance provisioning!
• AdminAPI transparently:
ü Setups the cluster to be ready for clone usage (plugin setup, user grants, etc.)
ü Detects every possible scenario and guides the user accordingly to the best provisioning
method
ü Still allowing manual intervention to choose the provisioning method
Copyright © 2019 Oracle and/or its affiliates.
InnoDB cluster 8.0.17+
51. • Binlogs can now be purged frequently!
• But (at least) one cluster member should keep them for 2 days
(minimum) for recovery.
Copyright © 2019 Oracle and/or its affiliates.
Just a tip!
!
53. Copyright © 2019 Oracle and/or its affiliates.
Node Provisioning Monitoring
The AdminAPI was extended to provide monitoring features:
• For any type of recovery method: clone or incremental
• Real-time progress information
• Detailed stage status
• Asynchronous progress information from cluster.status()
54. Copyright © 2019 Oracle and/or its affiliates.
Node Provisioning Monitoring
ü Clone-based recovery real-time progress info
55. Copyright © 2019 Oracle and/or its affiliates.
Node Provisioning Monitoring
ü Clone-based recovery progress from cluster.status()
56. Copyright © 2019 Oracle and/or its affiliates.
Cluster Status
Provides general information
about the cluster and each
member status
mysqlsh-js> cluster.status()
57. Copyright © 2019 Oracle and/or its affiliates.
Extended Status
Provides information of:
• GR Group name
• Group protocol version
• Cluster members UUIDs
• Cluster members roles and states
• Read-only settings
mysqlsh-js>
cluster.status({extended: 1})
58. Copyright © 2019 Oracle and/or its affiliates.
Extended Status
Provides information of:
• Transactions processed by
connection and applier
mysqlsh-js>
cluster.status({extended: 2})
59. Copyright © 2019 Oracle and/or its affiliates.
Extended Status
Provides information of:
• Detailed stats about the replication
machinery of each cluster member
mysqlsh-js>
cluster.status({extended: 3})
60. • Built-in reports
• User-defined reports
Copyright © 2019 Oracle and/or its affiliates.
Reporting Framework
// displays the result one time
mysqlsh-js> show query <query>
// executes repeatedly with screen refresh
mysqlsh-js> watch query <query>
// registers a user-defined-report
mysqlsh-js> shell.registerReport(<UDR>)
61. • Built-in reports
• User-defined reports
Copyright © 2019 Oracle and/or its affiliates.
Reporting Framework
// displays the result one time
mysqlsh-js> show query <query>
// executes repeatedly with screen refresh
mysqlsh-js> watch query <query>
// registers a user-defined-report
mysqlsh-js> shell.registerReport(<UDR>)
! Monitor P_S to check which
secondary instance is slower
63. Copyright © 2019 Oracle and/or its affiliates.
Maintenance
Monitoring a cluster can provide info/hints about:
• Faulty networks
• High latencies
• Server failures
• ...
64. Copyright © 2019 Oracle and/or its affiliates.
Cluster Options
Provides information of:
• The cluster global configuration
options
• Each cluster member settings
mysqlsh-js> cluster.options()
65. expelTimeout
Avoid frequent member expels
due to faulty network
memberWeight
Next primary in line when a
failover happens
exitStateAction
Action when a server is expelled
from the cluster: abort, read-only,
offline-mode
autoRejoinTries
Rejoin attempts after an eviction
Copyright © 2019 Oracle and/or its affiliates.
Cluster options
66. Copyright © 2019 Oracle and/or its affiliates.
Changing Cluster Options
// Change the member-weight value of a member
mysqlsh-js> cluster.setInstanceOption(“mysql2:3307”, “memberWeight”, 75)
67. Copyright © 2019 Oracle and/or its affiliates.
Changing Cluster Options
// Change the member-weight value of a member
mysqlsh-js> cluster.setInstanceOption(“mysql2:3307”, “memberWeight”, 75)
// Change the exit-state-action of a member
mysqlsh-js> cluster.setInstanceOption(“mysql3”, “exitStateAction”, “ABORT_SERVER”)
68. Copyright © 2019 Oracle and/or its affiliates.
Changing Cluster Options
// Change the member-weight value of a member
mysqlsh-js> cluster.setInstanceOption(“mysql2:3307”, “memberWeight”, 75)
// Change the exit-state-action of a member
mysqlsh-js> cluster.setInstanceOption(“mysql3”, “exitStateAction”, “ABORT_SERVER”)
// Change the expel-timeout of the cluster
mysqlsh-js> cluster.setOption(“expelTimeout”, 60)
69. Copyright © 2019 Oracle and/or its affiliates.
Consistency Levels
• InnoDB cluster operates, by default, in single-primary mode:
• Apps and users write to the primary instance
• What guarantees do we have that:
• We always read the most up-to-date data?
• We do not read stale data of an evicted server?
• Dirty reads do not happen?
?
70. Copyright © 2019 Oracle and/or its affiliates.
Consistency Levels
// Create a cluster with “BEFORE” consistency level
// Ensures that reads always read the most up-to-data data
mysqlsh-js> cluster = dba.createCluster(“CodeOne19”, {consistency: “BEFORE”})
71. Copyright © 2019 Oracle and/or its affiliates.
Consistency Levels
// Create a cluster with “BEFORE” consistency level
// Ensures that reads always read the most up-to-data data
mysqlsh-js> cluster = dba.createCluster(“CodeOne19”, {consistency: “BEFORE”})
// Change the consistency level at any time
// AFTER ensures that there are no dirty reads
mysqlsh-js> cluster.setOption(“consistency”, “AFTER”)
72. Copyright © 2019 Oracle and/or its affiliates.
Topology Modes
• The cluster’s primary instance can be changed, immediately:
73. Copyright © 2019 Oracle and/or its affiliates.
Topology Modes
• The topology-mode can be changed live:
// Change the topology-mode to multi-primary
mysqlsh-js> cluster.switchToMultiPrimaryMode()
// Change back to single-primary-mode
mysqlsh-js> cluster.switchToSinglePrimaryMode()
74. Copyright © 2019 Oracle and/or its affiliates.
Security
• Internal recovery accounts have randomly secure generated
passwords
• But security policies may enforce passwords lifetime
ü Support to reset a cluster’s recovery accounts credentials added
// Reset the recovery accounts credentials
mysqlsh-js> cluster.resetRecoveryAccountsPassword()
Available in 8.0.18
75. “Anything that can go wrong will go wrong.”
Copyright © 2019 Oracle and/or its affiliates.
Troubleshooting
76. “Anything that can go wrong will go wrong.”
Copyright © 2019 Oracle and/or its affiliates.
Troubleshooting
• Shell’s interactive behavior provides the users with valuable
information, by default.
• But sometimes more information is needed for troubleshooting...
77. Copyright © 2019 Oracle and/or its affiliates.
Logging
Logging levels:
• 1: none
• 2: internal
• 3: error
• 4: warning
• 5: info
• 6, 7, 8: debug
$ mysqlsh –-log-level=8
78. Copyright © 2019 Oracle and/or its affiliates.
Logging
Verbose output level:
• [1, 4]
• Enables diagnostic
messages output
$ mysqlsh –-verbose=4
79. Copyright © 2019 Oracle and/or its affiliates.
Logging
Log AdminAPI SQL:
• 0: no logging
• 1: log all queries
except SELECT and
SHOW
• 2: log all queries
$ mysqlsh –-dba-log-sql=2
Available in 8.0.18
80. • InnoDB cluster is THE built-in HA solution for MySQL
• MySQL Clone Plugin pushed InnoDB cluster to a whole new
level of usability!
• InnoDB Cluster features built-in support for full instance provisioning
• MySQL Shell is the unified interface for developers and DBAs
and the frontend manager for InnoDB cluster
Final Thoughts
Copyright © 2019 Oracle and/or its affiliates.
81. Resources
Copyright © 2019 Oracle and/or its affiliates.
• https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-
cluster-userguide.html (InnoDB cluster userguide)
https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/ (Shell APIs JS Ref manual)
https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/ (Shell APIs Python Ref manual)
• https://mysqlhighavailability.com (Blogging)
• http://mysqlserverteam.com/ (Blogging)
• https://mysqlcommunity.slack.com (Community Slack)
83. • MySQL Shell: The Best DBA Tool? How to Use MySQL Shell as a
Framework for DBAs [DEV2056]
• What’s New in MySQL Optimizer and Executor? [DEV2077]
• MySQL Connector/J 8.0 Feature Review and How to Upgrade from
Connector/J 5.1 [DEV6221]
• MySQL 8.0 Enterprise Backup [DEV4110]
• Group Replication: How I Stopped Worrying About Consistency and
Focused on Building Apps [DEV3016]
MOAR MySQL
Copyright © 2019 Oracle and/or its affiliates.
84. Session Survey
Help us make the content
even better. Please complete
the session survey in the
Mobile App.
Copyright © 2019 Oracle and/or its affiliates.
85. The preceding is intended to outline our general product direction. It is intended for information purposes
only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing decisions. The development,
release, timing, and pricing of any features or functionality described for Oracle’s products may change
and remains at the sole discretion of Oracle Corporation.
Statements in this presentation relating to Oracle’s future plans, expectations, beliefs, intentions and
prospects are “forward-looking statements” and are subject to material risks and uncertainties. A detailed
discussion of these factors and other risks that affect our business is contained in Oracle’s Securities and
Exchange Commission (SEC) filings, including our most recent reports on Form 10-K and Form 10-Q
under the heading “Risk Factors.” These filings are available on the SEC’s website or on Oracle’s website
at http://www.oracle.com/investor. All information in this presentation is current as of September 2019
and Oracle undertakes no duty to update any statement in light of new information or future events.
Safe Harbor
Copyright © 2019 Oracle and/or its affiliates.