Moodle Moot Spain: Moodle Available and Scalable with MySQL HA - InnoDB Cluster & ReplicaSet

Keith Hollman
Principal Solution Engineer
MySQL
keith.hollman@oracle.com
available
and scalable with
MySQL HA
InnoDB Cluster & ReplicaSet

© 2020 Oracle MySQL
Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
’ .

Introduction
Program agenda
1

High Availability: Terms and Concepts
 Availability
 Outage and downtime
 Mean Time Between Failures (MTBF)
 Mean Time To Recover (MTTR)
 Service Level Agreement (SLA)
 ’ : 5 = 5 mins downtime per year
 Capacity / Performance
 Redundancy
 Fault Tolerance
 Disaster Tolerance/Recovery

High Availability: Considerations
 SLA requirements
What is needed to support your business objectives
 Operational capabilities
Will you have the workforce to implement and maintain it
 Service agility
Can it grow and change as your company does
 Time to market
Can you go to market quickly
 Budgetary constraints
Cost of downtime versus cost of HA implementation
What To
Use?

High Availability: Factors
 Environment
 Redundant servers in different datacenters and geographical areas will protect you
against regional issues—power grid failures, hurricanes, earthquakes, etc.
 Hardware
 Each part of your hardware stack—networking, storage, servers—should be redundant
 Software
 Every layer of the software stack needs to be duplicated and distributed across separate
hardware and environments
 Data
 Data loss and inconsistency/corruption must be prevented by having multiple copies of
each piece of data, with consistency checks and guarantees for each change

High Availability: Design Factors
 Reliability, Availability, and Serviceability (RAS)
 Reliability
 Quality hardware and software components that you can manage and trust
 Availability
 Multiple copies of each datum replicated to separate hardware
 Multiple copies of each piece of the hardware and software stack
 Multiple physical locations
 Separate power grids
 Enough physical distance to survive disasters such as floods, earthquakes, and power grid outages
 Resiliency
 Automation to transparently deal with any potential failure in the infrastructure, hardware, software
 Serviceability
 Ensure that you can monitor, diagnose, maintain, and repair it all

High Availability: Database Needs
 Management infrastructure
 Monitoring of status, health, performance
 Facilities for service changes, service transitions
 Failure detection and handling
 Identify and handle failures
 Elasticity
 Scale up to ensure acceptable performance is always maintained
 Consistency guarantees
 Conflict detection and handling ; data loss protection/prevention
 Online maintenance

High Availability: Common MySQL Solutions
MySQL
Source /
Replica
Replication
Master/Slave
with
Automated
Failover
DRBD
OS or VM
vendor
active/passi
ve solutions
MySQL
Group
Replication
9 9 . 9 9 9 % Continuous
Availability

A study by the Gartner Group projected that
through 2015, 80% of downtime will be due to
people and process issues
10 © 2020 Oracle MySQL
40%
40%
20%
Software/Application Human Error Hardware
* Source: Gartner Group 1998 survey
High Availability: Causes of Downtime

 Calculate a cost per minute of downtime
 Average revenue generated per-minute over a year
 Cost of not meeting any customer SLAs
 Factor in costs that are harder to quantify
1. Revenue
2. Reputation
3. Customer sentiment
4. Stock price
5. ’
6. ’
THIS is why HA
matters!
High Availability: The Business Cost of Downtime

MySQL High Availability Solutions
• MySQL NDB Cluster
– NDB storage engine.
– Memory database.
– Automatic sharding of data.
– SQL Access via MySQL with
cross shard join support.
– Native access via several
I’ .
– Read/write consistency.
– Read/write scalability (2pc).
– ACID and transactions.
• MySQL InnoDB Cluster
– Easy HA built into MySQL
5.7+ for InnoDB.
– MySQL Group Replication,
Shell and Router.
– Write consistency.
– Read Scalability.
– Native CRUD API in
MySQL 8.
– Synchrounous (Paxos).
• MySQL Replication
– Core part of MySQL, used
by almost everyone.
– Can be used by any
storage engine.
– Asynchronous and semi-
sync option.
– Scale out reads.
– ReplicaSet from 8.0.19
with integration to Shell
and Router.

Program agenda
2 InnoDB Cluster

MySQL InnoDB Cluster
 High Availability becomes a core first class feature of MySQL!
 A single product — MySQL — with high availability and scaling
features baked in
 Providing an integrated end-to-end solution that is easy to use
 Group Replication: Automatic membership changes, network
partition handling, consistency...
 Shell to provide a powerful interface that helps in automating
an integrating all components
 Router provide transparent routing between your application
and back-end MySQL Servers
 InnoDB CLONE to automatically provision members, fully
integrated in InnoDB
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
MySQL Group Replication
App Clients

InnoDB Cluster: Application access
Application access the database
through MySQL Router.
Node 1 is Primary (RW).
Primary
Node 1
Router
Secondary
Node 2
Secondary
Node 3
Group Replication

InnoDB Cluster: Failure detection
Crash happens ...
Primary
R/W
Router
Secondary
Node 2
Secondary
Node 3
Crash
Group Replication

InnoDB Cluster: Failover PRIMARY role to Node 2
A new primary is automatically
elected since we still have a
majority of nodes available.
Primary
Node 1
Router
Primary
Node 2
Secondary
Node 3
Group Replication

InnoDB Cluster: Recovery of failed node as Secondary (RO)
Recover and rejoin the
crashed node.
Recovery is automatic in case
of NW outage.
Secondary
Node 1
Router
Primary
Node 2
Secondary
Node 3
Group Replication

MySQL Group Replication: Deploy Modes
• Multi-Primary Mode
– Every server in the group is allowed to execute transactions at any time, even
transactions that change state (RW transactions)
– Risk of more aborted transactions
– Send DDL operations to only one node in the cluster
• Single-Primary Mode (Default)
– The group has a single primary server that is set to read-write mode. All the other
members in the group are set to read-only mode.
– No need to modify application.

MySQL Group Replication: Deploy Modes *
• Multi-Primary Mode
– Every server in the group is allowed to execute transactions at any time, even
transactions that change state (RW transactions)
– Risk of more aborted transactions
– Send DDL operations to only one node in the cluster
• Single-Primary Mode (Default)
– The group has a single primary server that is set to read-write mode. All the other
members in the group are set to read-only mode.
– No need to modify application.

MySQL Group Replication: ” a ” Consistency levels (8.0.14)
• Eventual Consistency (default)
– Transaction does not wait at all.
– Executes on the current snapshot of the data on that member.
• Before Consistency (Synchronize on Reads)
– Transaction waits for all preceding transactions to complete.
– Executes on the most up to date snapshot of the data in the group.
• After Consistency (Synchronize on Writes)
– Transaction waits until all members have executed it.
– Executes on the current snapshot of the data on that member.

Consistency Levels – User Interface
• Consistency System Variable Controls the Behavior: group_replication_consistency.
• Global and Session scope
– Can be set per transaction.
• Values:
– EVENTUAL
– BEFORE_ON_PRIMARY_FAILOVER
– BEFORE
– AFTER
– BEFORE_AND_AFTER

CLONE: Automatic Full Data
Snapshot Provisioning
MySQL
Group Replication
Joining
Member
MySQL Group Replication: CLONE plugin (8.0.17)

MySQL InnoDB Cluster: Getting started
• Docs: https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-userguide.html
• Workshop: https://github.com/wwwted/MySQL-InnoDB-Cluster-3VM-Setup
• Blogs:
– https://lefred.be/content/category/mysql/innodb-cluster/
– https://mysqlhighavailability.com/category/replication/group-replication/

Planned Changes of the Primary
MySQL Router JS> cl.setPrimaryInstance("nodeA:3306")
User tells current primary to give up its role and assign it to another server

Tagging the Metadata
MySQL JS> cl.setInstanceOption(nodeA:3306, "tag:_hidden", true);
Hide instances from client traffic or declare additional information
 Support for the metadata "tags" property:
 _hidden : A boolean that indicates if the node should be hidden
 true makes the node hidden, while false or any other value considers it not hidden
 _disconnect_existing_sessions_when_hidden : A boolean that indicates if the existing connections to the node
should be dropped in case the node is hidden
 false means existing connections to hidden nodes are not disconnected while true or any other value means the
opposite
 Hidden instances only affect user connections; the instance remains available to Router when performing
tasks such as fetching metadata and calculating the quorum
 Also support for user defined tagging
 Tag namespace, where you can store information in the key-value pairs associated with a given node
https://dev.mysql.com/doc/refman/8.0/en/admin-api-tagging.html

Program agenda
3 InnoDB ReplicaSet

MySQL InnoDB ReplicaSet
“A quick & easy way to get
MySQL Replication & MySQL Router
up and running”
30
App Servers with
MySQL Router
MySQL Replication
MySQL Shell
Setup, Manage,
Orchestrate
App Clients

MySQL InnoDB ReplicaSet - Introduction
 An InnoDB ReplicaSet consists of a single source & multiple replicas (primary/secondary)
 Enable you to administer a set of MySQL instances running asynchronous GTID-based replication in a
similar way to InnoDB Cluster
 Enable you to work with MySQL replication without a deep understanding of the underlying concepts
 I ’ quick and easy way to get MySQL Replication and MySQL Router up and running
 You can automatically configure MySQL Router to use your InnoDB ReplicaSet
 Making it well suited to:
 Scaling out reads
 Provide manual failover capabilities in use cases that do not require the high availability offered by InnoDB Cluster

MySQL InnoDB ReplicaSet - Goals
Flexible and Modern
 SQL & NoSQL together
 Protocol Buffers
 Asynchronous API
 Developer friendly
A single product: MySQL
 All components created together
 Tested together
 Packaged together
 Proven & well known async replication
Easy to use
 One client: MySQL Shell
 Easy packaging
 Integrate orchestration
 Homogenous servers
Completely Open Source - GPL
 No license required to have
 High Availability

MySQL InnoDB ReplicaSet – The Big Picture
S
R
R
MySQL Shell and Orchestration Tooling
Simple Mapping,
State and Extra Metadata
Control,
Coordinate,
Provision
Monitoring (MEM)
MySQL Router
MySQL Async Replication
MySQL Router

MySQL InnoDB ReplicaSet - Requirements & Limitations
 Requirements:
 MySQL 8.0.19+
 GTID-based & Row based replication
 Limitations:
 Manual failover -- which is a good thing
 No coordination between servers mandates external arbitration
 However, secondary instances remain available for reads.
 No multi-primary as such topology cannot guarantee data consistency
 No data reconciliation
 No conflict handling
 No protection from partial data loss
 All secondary members replicate from primary
 Single tiered replication support.
 It is recommended that you deploy InnoDB Cluster wherever possible
https://dev.mysql.com/doc/refman/8.0/en/introducing-innodb-replicaset.html

InnoDB ReplicaSets Features
Now
 Automatically provisioning new members:
InnoDB Clone
 fi
& Replication
 Integrated MySQL Router load balancing
 Automatic Router Bootstrapping – no config
 Router is stateless, adapts to topology
changes
 Easy to use manual switchover/failover
Before
 Restore a backup to provision a member
 fi Replication fi
Replication
 Manually fi ,
servers in MySQL Router or alternatives
 Manually or relying on external tools to
make topology changes
 Use additional monitoring tool log in on all
machines to check topology

MySQL ReplicaSet – “Han -On“
Configure MySQL instances for ReplicaSet usage
mysql-js> dba.configureReplicaSetInstance("root@servers:3306")
Create ReplicaSet
mysql-js> c root@server1:3306
mysql-js> rs = dba.createReplicaSet(“ y ep ica e ")
Check Status
mysql-js> rs.status()
Add Instances to ReplicaSet
mysql-js> rs.addInstance("root@server2:3306")
mysql-js> rs.addInstance("root@server3:3306")
Switch-Over
mysql-js> rs.setPrimaryInstance("server3:3306")

Program agenda
2
Replication

MySQL Replication
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
App Clients
MySQL Replication

Database Replication: Multiple copies of data with replication process

MySQL Replication Overview
Sender
thread
Receiver
thread
Applier
Threads
SOURCE REPLICA

MySQL Replication Overview - Background
 Replication
MySQL Replication transfers database changes from one source server to its replicas
 Starts when transactions are prepared
Replication preserves the data changes in a binary log file on the Source just before the transactions are ready to commit
 Transfers database changes in a binary log
The binary log file contains an engine-agnostic representation of the data changes
It is sent to replicas whenever they request it or can be used for backup, point-in-time recovery and integration purposes
 Statement-based vs Row-based binary logs
The binlog can either carry the statements issued by the user or the deterministic row-changes resulting from those
statements
 Are asynchronously applied
The Replicas stream the binary log from the Source to an intermediate file on the Replica, the relay log, which is discarded
once it is no longer needed;
Workloads are applied asynchronously using a best effort approach; it competes for resources with other workloads on the
replicas
 Using single or multi-threaded schedulers
The Replicas scan the relay log and apply the data changes to the database using one of the available appliers: single-
threaded, DATABASE or LOGICAL_CLOCK

Global Transation Identifiers (GTID) Based MySQL Replication
 GTID is simple to track & compare replication across MySQL instances
 Unique identifier for each transaction written to the Binlog
 source_id:transaction_id
 eg : 3E11FA47-71CA-11E1-9E33-C80AA9429562:12
 Automatically identify the most up-to-date Replica for failover
 Simple to determine whether sources and replicas are consistent
 When using GTIDs you can take advantage of:
 Auto-positioning
 Easy fail-over,
 Monitor replicated transactions using Performance Schema tables
Self Healing MySQL Replication
Source
GTID=AAA:123
GTID=AAA:123
GTID=AAA:123
Replica 1
Replica 2
https://dev.mysql.com/doc/refman/8.0/en/replication-gtids.html

Enable GTID- mode
 To enable GTID start the server with:
 gtid_mode = ON
 enforce_gtid_consistency = ON
 server_uuid must exist for GTIDs to function correctly
 auto.cnf file
 GTID can also be enable online
 https://dev.mysql.com/doc/refman/8.0/en/replication-mode-change-online-enable-gtids.html
 Automatically configured with MySQL InnoDB ReplicaSet

MySQL Replication Topologies
Multi-source

Program agenda
5 Enterprise Solutions

MySQL 8: W a ’ n w!
’
 Transactional (InnoDB) Data Dictionary
 Persistent runtime configuration changes (SET PERSIST)
 Auto-increment counter value will now persist across server restarts!
 Indexing of PS tables
 SQL Roles (collection of privileges)
Developers
 Common Table Expressions
 Windows Functions
 Avoid locking with: SELECT ... FOR UPDATE [SKIP LOCKED | NOWAIT]
 More JSON and GIS functionality

Why MySQL Enterprise Edition?
Insure Your Deployments
Get the Best Results
Delight Customers
Improve
Performance &
Scalability
Enhance Agility &
Productivity
Reduce TCO
Mitigate
Risks
Get
Immediate
Help if/when
Needed
Increase
Customer
Satisfaction

Management
Tools
Advanced Features Support
•Scalability
•High Availability
•Security
•Encryption + TDE
•Data Masking
•Firewall
•Monitoring
•Backup
•Development
•Administration
•Migration
• Technical Support
• Consultative Support
• Oracle Certifications
MySQL Enterprise Edition

Standard Enterprise Carrier Grade

Enterprise
Security Architecture
 Workbench
•Model
•Data
•Audit Data
•User Management
  Enterprise Monitor
•Identifies Vulnerabilities
•Security hardening policies
•Monitoring & Alerting
•User Monitoring
•Password Monitoring
•Schema Change Monitoring
•Backup Monitoring
Data Encryption
•TDE
•Encryption
•PKI
 Firewall
 Enterprise Authentication
•SSO - LDAP, AD, PAM
 Network Encryption
 Enterprise Audit
•Powerful Rules Engine
 Audit Vault
 Strong Authentication
 Access Controls
 Assess
 Prevent
 Detect
 Recover
 Enterprise Backup
•Encrypted
 HA
•Innodb Cluster
Thread Pool
•Attack minimization
 Key Vault
•Protect Keys
 Enterprise
Masking & De-Identification
•Masking
•Substitute/Subset
•Random Formatted Data
•Blacklisted Data

MySQL Enterprise Authentication
Integrates MySQL with existing
security infrastructures
• Integrate with Centralized Authentication Infrastructure
– Centralized Account Management
– Password Policy Management
– Groups & Roles
• PAM (Pluggable Authentication Modules)
– Standard interface (Unix, LDAP, Kerberos, others)
– Access native Windows service - Use to Authenticate users using Windows Active
Directory or to a native host

MySQL Enterprise Firewall
• Real Time Protection
– Queries analyzed and matched against White List
• Blocks SQL Injection Attacks
– Positive Security Model
• Block Suspicious Traffic
– Out of Policy Transactions detected & blocked
• Transparent
– No changes to application required
• Learns White List
– Automated creation of approved list of SQL command patterns on a per user basis
MySQL Enterprise Firewall monitoring

MySQL Enterprise TDE (Transparent Data Encryption)
SQL
• New option in CREATE TABLE
ENCRYPTION=“Y”
• New SQL : ALTER INSTANCE ROTATE
INNODB MASTER KEY
Keyring plugins
• Encrypted file on disk
• Hashicorp Vault
• KMIP, Oracle KeyVault, Gemalto
KeySecure or Fornetix
Plugin Infrastructure
• New plugin type : keyring
• Ability to load plugin before InnoDB
initialization : --early-plugin-load
InnoDB
• Support for encrypted tables
• IMPORT/EXPORT of encrypted tables
• Support for master key rotation

MySQL Enterprise Audit
• Out-of-the-box logging of connections, logins, and query
• User defined policies for filtering, and log rotation
• Dynamically enabled, disabled: no server restart
• XML-based or JSON audit stream
• Supports encryption (MySQL 5.7.18+)
• Certified with tools like Oracle Audit Vault
Adds regulatory compliance to MySQL applications
(HIPAA, GDPR, PCI, etc.)

MySQL Enterprise Masking and De-Identification
• Data Masking
– String masking
– Dictionary based replacement
– Specific masking
• SSN
• Payment card : Strict/Relaxed
Adds regulatory compliance to MySQL applications
(HIPAA, GDPR, PCI, etc.)
• Random Data Generators
– Random number within a range
– Email
– Payment card (Luhn check compliant)
– SSN
– Dictionary based generation

MySQL Enterprise Monitor
• Start monitoring MySQL in 10 minutes
• Real-time MySQL performance and
availability monitoring
• Visually find & fix problem queries
• Disk monitoring for capacity planning
• Cloud friendly architecture
– No agents required
• Role based access controls

MySQL Enterprise Monitor: Query Analyzer
• Real-time query performance.
• Visual correlation graphs.
• Find & fix expensive queries.
• Detailed query statistics.
• Query Response Time index (QRTi).
“ of ” (QoS) measurement for
each query QoS measurement for a server,
group, or every instance.
Single metric for query performance.

MySQL Enterprise Plugin for Enterprise Manager
• Availability monitoring
• Performance monitoring
• Configuration monitoring
• All available metrics collected
– Allowing for custom threshold based
incident reports
• MySQL auto-detection

MySQL Enterprise Backup
• Online, non-locking backup and recovery
– Complete MySQL instance backup (data and config)
– Partial backup and restore
• Direct Cloud storage backups
– Oracle Storage Cloud, S3, etc.
• Incremental backups
• Point-in-time recovery
• Advanced compressed and encryption
• Backup to tape (SBT)
• Cross-Platform (Windows, Linux, Unix)

MySQL Enterprise Support
• Largest MySQL engineering and support organization
• Backed by the MySQL developers
• World-class support, in 29 languages
• Hot fixes & maintenance releases
• 24x7x365
• Unlimited incidents
• Consultative support
• Global scale and reach
Get immediate help for any MySQL
issue, plus expert advice

MySQL Enterprise Consultative Support
• Remote troubleshooting
• Replication review
• Partitioning review
• Schema review
• Query review
• Performance tuning
• ...and more
Get immediate help for any MySQL
issue, plus expert advice

Work Directly with The MySQL Team
• A direct relationship with the MySQL team
• The ability to participate in:
– Product roadmaps
– Product betas
– Customer advisory boards
• Work closely with Support Engineers
– Resolve issues faster
– Request bug and feature request escalations

MySQL Enterprise Oracle Certifications
• Oracle Enterprise Manager
• Oracle Linux & Oracle VM
• Oracle Solaris
• Oracle Solaris Clustering
• Oracle Clusterware
• Oracle OpenStack
• My Oracle Support
• Oracle Fusion Middleware
• Oracle GoldenGate
• Oracle Audit Vault
• Oracle Database Firewall
• Oracle Secure Backup
MySQL integrates into your Oracle environment

Time for a quick demo?
Primary
Router
Secondary Secondary
Group Replication
ic3ic2ic1
ap1
Moodle 3.8
O.Linux 7.4
MySQL 8.0.22
MySQL Shell 8.0.22
MySQL Router 8.0.22

InnoDB Cluster
Source Site
Asynchronous
replication
IC
IC
IC
Group Replication
DR Site
GR
GR
:7001
:6446
:6446
:7001
Router:6446,Router:7001 DR-Router:6446,DR-Router:7001
Combine HA and DR?
MySQL InnoDB Cluster with Group Replication DR
Router Router

InnoDB Cluster
Source Site
IC
IC
IC
Group Replication
DR Site
GR
GR
:7001
:6446
:6446
:7001
Router:6446,Router:7001 DR-Router:6446,DR-Router:7001
Combine HA and DR?
MySQL InnoDB Cluster with Group Replication DR
Asynchronous
replication
RouterRouter

References
Moodle with MySQL Database Service: https://lefred.be/content/using-oci-to-install-moodle-with-mysql-8-0/
MySQL InnoDB Cluster Tutorial https://mysqlserverteam.com/mysql-innodb-cluster-8-0-a-hands-on-tutorial
InnoDB Cluster – Demo https://www.slideshare.net/keithhollman5/mysql-innodb-cluster-ha-overview-demo
Engineering blogs https://mysqlhighavailability.com/
MySQL InnoDB Cluster Documentation https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-userguide.html
MySQL InnoDB ReplicaSet Documentation https://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-replicaset.html
MySQL Replication https://dev.mysql.com/doc/refman/8.0/en/replication.html
MySQL Shell https://dev.mysql.com/doc/mysql-shell/8.0/en/
MySQL Router https://dev.mysql.com/doc/mysql-router/8.0/en/
Using MySQL AdminAPI https://dev.mysql.com/doc/refman/8.0/en/admin-api-userguide.html
MySQL 8.0.22 New Features Summary http://dasini.net/blog/2020/11/10/mysql-8-0-22-new-features-summary/

Wrap- p: y …
…
Community and Enterprise, Ent. Server and Cluster CGE
…
Business owners, Architects, Developers, Operations, etc.
…
SQL and NoSQL together, modular or micro services
…
Support 24x7, Business Continuity, Security compliance, etc.

Questions?

¡GRACIAS!
Keith Hollman
Solution Engineering EMEA
MySQL
¡GRACIAS!

Moodle Moot Spain: Moodle Available and Scalable with MySQL HA - InnoDB Cluster & ReplicaSet

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Moodle Moot Spain: Moodle Available and Scalable with MySQL HA - InnoDB Cluster & ReplicaSet

Ähnlich wie Moodle Moot Spain: Moodle Available and Scalable with MySQL HA - InnoDB Cluster & ReplicaSet (20)

Mehr von Keith Hollman

Mehr von Keith Hollman (8)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Moodle Moot Spain: Moodle Available and Scalable with MySQL HA - InnoDB Cluster & ReplicaSet