2. What is replication?
Allows 2 or more databases to maintain state
between them
MySQL 3.23 and up supports asynchronous
replication
One server acts as a master for one or more slaves
Slaves pull data to be replicated from server’s
binary logs and execute the relevant statements
locally
3. 3
MySQL Replication
Why? How?
1. High Availability Snapshots (Backup)
Possibility of fail-over 1. Client program mysqldump
2. Load-balancing/Scale- With log coordinates
out 2. Using backup
Query multiple servers InnoDB, NDB
3. Off-site processing
Don’t disturb master
Binary log
1. Replication
Asynchronous pushing to slave
2.Point-in-time recovery
Roll-forward
4. Replication basics…
One master supports multiple slaves
Each node on the network is assigned a unique identifying number
Each slave attempts to connect to the master and fetches data to be replicated
“Master” keeps logs of all changes – called
“binary logs” or “binlogs”
“Slave” connects to the master through normal
MySQL protocol (TCP port 3306)
Master
Slave
Slave
Slave
Slaves
5. Replication basics…
Clients perform data modification on master
server
INSERT, SELECT, DELETE, LOAD DATA
EXECUTE in MySQL 5.0 and above
Master SlaveClient
DATA DATA
INSERT
INTO …
6. Replication basics…
Immediately following execution of command on master,
the command is written to the local binary log
Additionally, the master records its unique ID (to prevent
endless loops in circular replication scenarios) and the
timestamp for use with statements which use NOW(), etc.
Master SlaveClient
DATA
DATA
INSERT
INTO …
Binary
Log
7. Replication basics…
If the slave is online, the command is transmitted to the slave in
parallel (well, immediately following) to being written in the local
binary log
Otherwise, when the slave next connects it will receive a list of all
pending statements from the master server’s binary log
The slave’s replication IO thread stores the command in the local relay
log
Master SlaveClient
DATA
DATA
INSERT
INTO …
INSERT
INTO …
Relay
Log
Replication
Thread
8. Replication basics…
Once the data is received in the slave’s relay log, the slave
SQL thread executes the command locally, bringing the
slave up-to-date with the master
In MySQL 3.23, the IO and SQL threads were just one
thread. In later versions this was changed to boost
performance
Master SlaveClient
DATA
DATA
INSERT
INTO …
INSERT
INTO …
Relay
Log
Replication
Thread
DATA
9. Replication basics…(contd)
Replication works with all tables types
• Any “critical” reads must be done on the master –
replication is asynchronous, there may be a delay
• Master will rotate binary logs automatically for
every 1G of log records
• You must purge any old, unused, logs yourself
10. Terminology
Synchronous replication Master
• A transaction is not committed until the data MySQL
has been replicated (and applied) Server
• Safer, but slower
• This is available in MySQL Cluster
Replication
Asynchronous replication
• A transaction is replicated after it has been
MySQLcommitted
Server
• Faster, but you can in some cases loose
transactions if master fails Slave
• Easy to set up between MySQL servers
11. Terminology
Master MySQL Server
• Changes data
• Has binlog turned on Master
• Pushes binlog events to slave after slave has requested themMySQL
Server
Slave MySQL Server
• Main control point of replication
Replication• Asks master for replication log
• Gets binlog event from master
MySQL
Binary log Server
• Log of everything executed Slave
• Divided into transactional components
• Used for replication and point-in-time recovery
12. Simple Replication Setup
Modify my.cnf to include a unique server-id for
each node
On master server, ensure that log-bin (binary
logging) is enabled in my.cnf
On slave, configure login credentials on master,
either via my.cnf or CHANGE MASTER TO
statement
Copy initial data snapshot from master to slave
Configure initial binary log position on slave
Start replication with SLAVE START command
13. server-id – The unique server ID of this MySQL
• log-bin – Enable logging of changes to binary
logs
• log-slave-updates – Log updates that arrive on
the slave thread to the binary logs as well
(required if this master is also a slave of another
machine)
• binlog-do-db – Disables logging of any changes,
except to the specified databases
• binlog-ignore-db – Log all changes, as usual,
except for the specified databases
15. ON MASTER SIDE
1 ) grant slave users on the master server :
Each slave must connect to the master using a standard
MySQL user name and password, so there must be a user
account on the master that the slave can use to connect. In this
scenario with existing slave, user for replication already exist on
the master server.
16. Permissions
Slaves need REPLICATION SLAVE permission on
master for basic usage
If LOAD TABLE FROM MASTER or LOAD DATA
FROM MASTER statements are used, slave will also
need SUPER and RELOAD privileges
17. Configuration - grants on master
GRANT REPLICATION SLAVE on
*.* TO ‘rep_user’@’slave-
host’ IDENTIFIED BY ‘this-is-the-password’
18. ON MASTER SIDE
2) Locking databases:
Take a section of mysql and lock the tables with the below
command.
USE exampledb; database name
mysql> FLUSH TABLES WITH READ LOCK;
Do not close the section. Kept it open
19. ON MASTER SIDE
3) Position Noting:
Use the below command and note down the position of master
SHOW MASTER STATUS;
21. ON MASTER SIDE
4) Backup the database:
Backup the databases using the following command
#> mysqldump -u root -p --databases trains >
/tmp/NNNNNN.sql
22. ON MASTER SIDE
5) unlocking databases;
On successfully compellations of backup activity up can unlock
databases on the activity section left open previously
mysql> UNLOCK TABLES;
26. ON SLAVE SIDE
2) Defining Position:
CHANGE MASTER TO MASTER_HOST='IP',
MASTER_USER='slave_user',
MASTER_PASSWORD='<some_password>',
MASTER_LOG_FILE='mysql-bin.NNNNN',
MASTER_LOG_POS=NN;
34. Replication is independent of Storage Engines
You can replicate between any pair of engines
InnoDB to InnoDB
MyISAM to MyISAM
InnoDB to MyISAM
MEMORY to MyISAM
etc...
The binary log is not the InnoDB transaction log (or the
Falcon log, or ...)
38. ... Server 3 is still up, but out of sync
Master/
Master Slave
X
39. Each server has a unique "server_id"
server_id=3server_id=1
Master/
Master Slave
Slave
server_id=2
... and every event in a binary log file contains
the server id number of the server where the
event originated.
41. The ring topology is not a recommended
configuration
Master/
Slave
Master/
Slave
Master/
X
42. Pair of Masters
Master/ Master/
Slave Slave
The pair is a “special case” of the ring topology
used for
high availability.
43. The two most common topologies for
MySQL Replication
Master
Master/ Master/
Slave Slave
Slave
Slave
Slave
44. The "Relay Slave"
The master has to
handle only one
Master TCP connection.
Relay
Slave
log_slave_updates
Slave Slave Slave Slave Slave
45. And now introducing... the blackhole
storage engine
Master
engine = blackhole The relay slave
Relay manages replication
Slave
logs, but not actual
data.
Slave Slave Slave Slave Slave
47. SHOW MASTER STATUS
Used on master
Requires SUPER or REPLICATION CLIENT privileges
Gives log file and position master is writing to Also
shows database filters used
mysql> SHOW MASTER STATUS;
+---------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+---------------+----------+--------------+------------------+
| mysql-bin.003 | 73 | test | manual,mysql |
+---------------+----------+--------------+------------------+
48. SHOW BINARY LOGS
Used on master
Requires SUPER privileges
Will display a list of binary logs on the server
Use it before using PURGE BINARY LOGS
mysql> SHOW BINARY LOGS;
+---------------+-----------+
| Log_name | File_size |
+---------------+-----------+
| binlog.000015 | 724935 |
| binlog.000016 | 733481 |
+---------------+-----------+
49. SHOW BINLOG EVENTS
Used on master
Requires REPLICATION SLAVE privileges
Show events in binary log
Also check mysqlbinlog utility
mysql> SHOW BINLOG EVENTS FROM 390 LIMIT 1G
*** 1. row ***
Log_name: slave-bin.000001
Pos: 390
Event_type: Query
Server_id: 2
End_log_pos: 476
Info: use `test`; create table t1 (a int) 1
row in set (0.00 sec)
50. SHOW SLAVE HOSTS
Used on master
Requires REPLICATION SLAVE privileges Shows list of
slaves currently registered with the master Only slaves
started with report-host option are visible
mysql> SHOW SLAVE HOSTS;
+-----------+-----------+------+-----------+
| Server_id | Host | Port | Master_id |
+-----------+-----------+------+-----------+
| 2 | 127.0.0.1 | 9308 | 1 |
+-----------+-----------+------+-----------+ 1
row in set (0.00 sec)
51. PURGE BINARY LOGS
Used on master
Requires SUPER privileges
Removes log files before a certain log file or date
MASTER can be used in place of BINARY
Alternative is to use variable EXPIRE_LOGS_DAYS
52. SHOW SLAVE STATUS
Used on slave
Requires SUPER or REPLICATION CLIENT privileges
Shows some interesting information:
If the slave threads are running What
position the I/O thread read last What
position the SQL thread executed last
Error message and code, if thread stopped due to an error
53. SHOW SLAVE STATUS (5.1)
mysql> SHOW SLAVE STATUSG
*** 1. row ***
Slave_IO_State: Last_Errno: 0
Master_Host: 127.0.0.1 Last_Error:
Master_User: root Skip_Counter: 0
Master_Port: 10190 Exec_Master_Log_Pos: 0
Connect_Retry: 1 Relay_Log_Space: 102
Master_Log_File: Until_Condition: None
Read_Master_Log_Pos: 4 Until_Log_File:
Relay_Log_File: slave-relay-bin.000001 Until_Log_Pos: 0
Relay_Log_Pos: 4 Master_SSL_Allowed: No
Relay_Master_Log_File: Master_SSL_CA_File:
Slave_IO_Running: No Master_SSL_CA_Path:
Slave_SQL_Running: No Master_SSL_Cert:
Replicate_Do_DB: Master_SSL_Cipher:
Replicate_Ignore_DB: Master_SSL_Key:
Replicate_Do_Table: Seconds_Behind_Master: NULL
Replicate_Ignore_Table: Last_IO_Errno: 0
Replicate_Wild_Do_Table: Last_IO_Error:
Replicate_Wild_Ignore_Table: Last_SQL_Errno: 0
Last_SQL_Error:
1 row in set (0.00 sec)
54. START SLAVE and STOP SLAVE
Used on slave
Used to start or stop the slave threads Defaults to
affecting both I/O and SQL thread ... but individual
threads can be started or stopped START SLAVE
SQL_THREAD
START SLAVE IO_THREAD
55. RESET SLAVE
Used on slave
Removes all info on replication position
Deletes master.info, relay-log.info and all relay logs
Relay logs are unconditionally removed!
... even if they have not been fully applied
56. SET GLOBAL SQL_SLAVE_SKIP_COUNTER
Used on slave
Global server variable
Requires SUPER privileges
Slave SQL thread shall not be running
Slave will skip events when starting
Useful when recovering from slave stops
Might leave master and slave with different data in tables
... so be careful when you use it
65. Use Cases, Part 3 - Multiple Data Centers
secure
San Jose New Yorktunnel
rep
Active
Master
Master
wr wr
wr wr
Slave Slave
rd rd
app app
Slave Slave
( Jeremy Cole - MySQL Users Conf 2006 )
66. After Failover
secure
San Jose New Yorktunnel
rep
Active
Master
Master
wr wr
wr wr
Slave Slave
rd rdapp app
Slave Slave
( Jeremy Cole - MySQL Users Conf 2006 )
67. Internal Threads
Since MySQL 4.0, replication slaves run two threads
IO thread continuously receives updates from master
and writes to local relay log
SQL thread continuously executes statements in relay
log
68. IO thread isolation
Isolating IO thread means that slave won’t have to
wait for long-executing statements to finish executing
before retrieving data from master
Also, slave will continue reading data from master if a
statement creates a data conflict
69. SQL thread isolation
SQL thread isolation allows for replication in an
environment without a continuous link between
slave and masters
If master fails (or slave simply has no access), the
IO thread will try to reconnect endlessly (waiting
60 seconds between attempts)
SQL thread will continue processing relay logs
even while IO thread is unable to connect to
master
70. Master Thread
Additionally, the master server runs the Binlog Dump
thread
This thread is simply dedicated to scanning the binary
logs on the master and sending updates to the
connected slave
If this thread isn’t running, it means that replication
isn’t running – more accurately, that no slaves are
currently connected
71. Status files
2 status files for replication’s use
Their use is to record the state of replication between
server shutdown and startup
master.info records information about the slave’s
master server
relay-log.info records information about the local
relay logs
72. Information in master.info
Master log file
Read master log pos
Master Host
Master User
Password (will not be shown in SHOW SLAVE
STATUS)
Master Port
Connect Retry
In MySQL 4.1+, SSL options are stored if SSL is
used
74. Backup master
Master backups can be accomplished with
mysqldump
Care must be taken to ensure the following 2
special considerations:
1. Consistent snapshot of master date (via lock tables
for MyISAM or single transaction for InnoDB)
2. Recording of binary log information, for use on slaves
(master-data)
75. Backup master files
If a file-system level backup is required, care should be
taken to manually record binary log name and position via
SHOW MASTER STATUS statement.
To ensure consistency between backup and binary log
position, the tables should be locked via FLUSH TABLES
WITH READ LOCK immediately before backup (and
SHOW MASTER STATUS)
LEAVE THE CLIENT CONNECTED!!!
After backup finishes, execute UNLOCK TABLES to release
the read lock
76. Backup slave
Same idea as master file system backup
Instead of recording position, it’s enough to backup
the master.info and relay-log.info files
Instead of acquiring global read lock, it’s enough to
STOP SLAVE before backup and START SLAVE once
backup finishes
77. DO
Make sure server-id is set on all machines
Enable log-bin on your master
If a slave will also be a master, set log-
slaveupdates
Take backups on a slave
Architect your replication setup for growth!
78. DO NOT
Don’t use master-* lines in my.cnf; use CHANGE
MASTER instead
Don’t take your backups on the master