Scaling your web app with MySQL replication

Scaling your web app
with MySQL replication
Giuseppe Maxia
MySQL Community Team Lead

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. 1
Thursday, 30 September 2010 1

Why replication
From single
server to
replication
How to set
replication
Table of Adding a slave
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
2

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
3

database server

a simple web
application
scheme

r/w requests

web server

clients
4

database server

scaling web
requests

r/w requests

web servers

load balancer

clients
5

database load
on a simple
web
application
r
e
write a
d
85% 15%

6

write read
20% 80%
database load on a
successful web
application
7

database server
scaling up
means buying
a bigger

✘ r/w requests
database
server

web servers

load balancer

clients
8

write read
20% 80%

the bigger database
server will eventually
have the same problem
9

read/write
master a web
application
read/only
slaves
scheme with
replication
load balancer
R/W

R/O

web servers

load balancer

clients
10

r
e
write a read
d

85% 15% 100%
read/write
master

read/only
slaves

database load
with
replication
11

r
e
write a read
d

85% 15% 100%
read/write
master

read/only
slaves

scaling
database load
with
replication
12

Replication assessment
without replication with replication

database handling easy harder

performance high lower (binary logs)

Point in Time recovery none easy

failover none possible

write scaling none minimal

backup with downtime without downtime

read scaling none easy

13

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
14

client master
transaction

binary log

reads
slave
IO thread

relay log replication
concepts
SQL thread reads
15

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
16

1 SHUT DOWN THE DATABASE SERVER

Master

17

2 MAKE A BACKUP COPY

Master

18

3 ENABLE THE MASTER

Master

Configuration file
[mysqld]
log-bin=mysql-bin
server-id=1

19

4 RESTART THE MASTER

Master

20

5 CREATE REPLICATION USER

Master

SQL command
GRANT REPLICATION SLAVE ON *.*
to 'slave_user@'10.10.100.%'
IDENTIFIED BY 'slave_pass';

21

6 INSTALL MySQL on the slave

Slave 1

Make sure that:
• You're using the same version of MySQL
• You have the same directory structure
• The server is not started yet

22

7 COPY THE MASTER DATA to the slave

Slave 1

23

8 ENABLE THE SLAVE

Slave 1

Configuration file
[mysqld]
server-id=2
relay-log=mysql-relay
read-only
# optional:
log-bin=mysql-bin 24

9 START THE SLAVE SERVER

Slave 1

25

10 INITIALIZE THE SLAVE

Slave 1

SQL command
SET MASTER TO
MASTER_HOST=master_IP,
MASTER_PORT=3306,
MASTER_USER=slave_user,
MASTER_PASSWORD='slave_pwd';
26

11 START THE SLAVE SERVICE

Slave 1

SQL command
START SLAVE;

27

12 CHECK THE SLAVE

Slave 1

SQL command
SHOW SLAVE STATUS G
...
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
...
28

Troubleshooting

• SHOW SLAVE STATUS says
SLAVE_IO_RUNNING=No

• Make sure that the slave host can connect
to the master

• Make sure that master and slave have
different Server-id

• Check the error log of both master and
slave

29

Testing the slave

• Create a table in the master.
• Make sure that the slave has replicated the
table.

• Insert data in the master
• read that data in the slave

30

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
31

1 NO NEED TO STOP THE MASTER!

Master

32

2 STOP THE SLAVE

Slave 1

SQL command
STOP SLAVE IO_THREAD;
# wait until the SQL_THREAD
# has done everything
STOP SLAVE SQL_THREAD;
# STOP THE SERVER
33

3 MAKE A COPY OF THE DATA DIRECTORY

Slave 1

34

4 RESTART THE SLAVE

Slave 1

35

5 INSTALL MySQL on the new slave

Slave 2

Make sure that:
• You're using the same version of MySQL
• You have the same directory structure
• The server is not started yet

36

6 COPY THE old slave DATA on the slave

Slave 2

37

7 ENABLE THE NEW SLAVE

Slave 2

Configuration file
uni que!
[mysqld]
mus t be
server-id=3
relay-log=mysql-relay
read-only
# optional:
log-bin=mysql-bin 38

8 START THE NEW SLAVE

Slave 2

39

9 CHECK THE SLAVE

Slave 2

SQL command
SHOW SLAVE STATUS G
...
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
...
40

Why it works

• No need to issue a CHANGE MASTER TO
command.

• Because we cloned the old slave
• The new slave gets its parameters from
the .info files in the data directory

41

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
42

From single server application

r/w requests

43

To replication-aware application
read/write
master

read/only
slaves

R/W
load balancer
R/O

44

Single server application
$link = mysql_connect(
$server_IP,
'mysql_user',
'mysql_password'
);
$result = mysql_query(
'INSERT INTO table_name (x) VALUES (1)',
$link
);
$result = mysql_query(
'SELECT * FROM table_name WHERE x=1',
$link
);
45

Making an application aware of
replication
<<database handling>>
db
<<R/W database
handling>> IP
user
db password

IP connect
user
password

connect <<read-only database <<R/W database
handling>> handling>>
read db db
write
IP IP
user user
password password
read connect
read
write

46

Using replication:
the WRONG way
No
Write Yes
statement?

Connect to the Connect to
next available the master
slave

Read from Write to the
slave master

R/W split
by Stop
statement 47

Why statement split is wrong

• Breaks transactions
• High risk of inconsistency
• Loses or corrupts data

48

Using replication:
the RIGHT way
No
Write Yes
function?

Connect to the Connect to
next available the master
slave

Read from Read and write
slave from master

more Yes more
queries? queries?
Yes

R/W split No No

by function Stop
49

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
50

Managing replication

•MONITORING
• … or die
51

Sample get slave
status
monitoring
slave

Running? No

Yes
master
Get master
binlog and
position

Same or later
No
Yes
binlog/position?

check table
contents
alert 52

slave
master
GET table
GET table
GET table
CRC table
GET
CRC
CRC
CRC

same? No

Yes

monitoring
contents Stop

alert 53

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
54

Replacing Slave
a slave crashe
s

No
are there Yes
more slaves?

STOP the
master STOP one
slave

add the ﬁrst add another
slave slave

Stop
55

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
56

Replacing the Master
master crashe
s

Let all slaves
catch up with
execution STOP
replication in
all slaves

FIND the most make it the
up to date slave master

FIND which run missing
connect all
transactions are transactions
slaves to the
missing from to other
new master
other slaves slaves

Stop
57

Why replication
From single
server to
replication
How to set
replication
contents
Using
replication
Replacing a
Managing slave
replication
Replacing the
master
Leveraging
replication
58

read/write load
master balancing

read/only
slaves

R/W
load balancer
R/O

59

master
backup

slaves

STOP SLAVE

remove slave
perform
from load
backup
balancer

attach slave
START Let slave
to load
SLAVE catch up
balancer 60

master
make
summary
tables
slaves

STOP SLAVE

calculate remove slave
summary from load
tables balancer

attach slave
Let slave
START SLAVE to load
catch up
balancer 61

master
Partitions
innodb
for heavy non partitioned
statistics

slave slave

innodb
innodb non partitioned
partitioned by range

slave

MyISAM
partitioned by range 62

Simulating master
multiple innodb
non partitioned
dimensions
slave

slave

innodb
non partitioned
ARCHIVE
(date)
slave
slave
ARCHIVE
ARCHIVE (location)
(product) 63

Semi-synchronous replication

• Available in 5.5 and higher
• Makes sure that at least one slave has copied
the data.

• Increases reliability

64

client
master transaction
with regular
commit replication

execute

slave
returns
binary log
to client

replication
65

client transaction
master
with semi-
commit synchronous
replication

execute

slave
returns
binary log
to client

sends
transaction relay log
to slave

gets
acknowledgement
66

Scaling your web app with MySQL replication

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Scaling your web app with MySQL replication

Ähnlich wie Scaling your web app with MySQL replication (20)

Mehr von Giuseppe Maxia

Mehr von Giuseppe Maxia (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Scaling your web app with MySQL replication