A short introduction to database replication, theory and practice (MySQL, PostgreSQL, CouchDB)

1. Database replication
Vatroslav Mileusnić, univ.bacc.inf.
mileusnic.vatroslav@gmail.com

2. What is replication?
Database replication - creating and maintaining
multiple copies of the same database
Types of replication:
synchronous and asynchronous
master-slave and master-master (multimaster)
...

3. Synchronous Asynchronous
propagates every transaction propagates multiple
transactions
creating connections for
every transaction causes uses less network bandwidth
bigger overhead because and offers better
performance
stops replication if one server
is unavailable high availability of replication
group
possibility of dana collision
possibility of lost transactions

4. Master-slave Multimaster
1 master, other slaves all servers are masters
write on master write on any server,
propagate to slaves propagate to others
divide reading between divide reading between all
slaves servers
configure slave to take powerful and flexible
over as master if the solution, but also very
original master falls out complex

5. Multimaster replication problems
possibility of conflict in asynchronous replication
update conflict
unique conflict
delete conflict
possible solutions:
configure application to write in only one database
configure application to write in another database if the first
one falls out

6. Multimaster replication problems
very short intervals of empting queries
uses a huge number of system resources
starting and stopping the process requires a lot processor
cycles
solution:
don't exaggerate with application requrements
intervals of 1 or more minutes are fine

7. Multimaster replication problems
tool for monitoring replication system
if using synchronous replication, all servers have to be
active and ready
automated tool that periodicaly checks the status of all
servers
if the tool decides a server is unavailable, it should try to
restart it, and if it fails, remove it from replication, so
replication can go on

8. Primjer replikacije
Asynchronous master-slave replication in MySQL
Asynchronous master-slave replication in PostgreSQL
Asynchronous multimaster replication in CouchDB
Displayed solutions are not configured for production
servers, but are good enough for learning and
understanding

9. MySQL
On servers create a database (replicate_me)
On master server change /etc/mysql/my.cnf
#skip-networking
#bind-address = 127.0.0.1
log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db = replicate_me
server-id=1
Save file and restart MySQL

10. MySQL
Enter MySQL and grant rights to the slave user
GRANT REPLICATION SLAVE ON *.* TO
slave_user@192.168.1.101 IDENTIFIED BY 'password';
GRANT RELOAD SLAVE ON *.* TO
slave_user@192.168.1.101 IDENTIFIED BY 'password';
GRANT SUPER SLAVE ON *.* TO
slave_user@192.168.1.101 IDENTIFIED BY 'password';
FLUSH PRIVILEGES;

11. MySQL
Perform commands to display master's status
USE replicate_me;
SHOW MASTER STATUS;
take note of File and Position columns
UNLOCK TABLES;

12. MySQL
On slave server add in /etc/mysql/my.cnf
server-id = 2
master-host = 192.168.1.100
master-user = slave_user
master-password = password
master-connect-retry = 60
replicate-do-db = replicate_me

13. MySQL
On slave server, in MySQL, do
STOP SLAVE;
CHANGE MASTER TO MASTER_HOST = '192.168.1.100'
MASTER_USER='slave_user',
MASTER_PASSWORD='password',
MASTER_LOG_FILE='mysql-bin.000001',
MASTER_LOG_POS=106;
START SLAVE;

14. PostgreSQL
As postgres user initialize cluster on both master and
slave servers
initdb /var/lib/pgsql/data
Allow communication of postgres user
SSH or another way

15. PostgreSQL
On master in /var/lib/pgsql/data/postgresql.conf
wal_level = hot_standby
archive_mode = on
archive_command =
'scp %p postgres@192.168.1.101:/var/lib/pgsql/archive/%f'
Start cluster
pg_ctl -D /var/lib/pgsql/data start

16. PostgreSQL
Perform backup
psql -c "SELECT pg_start_backup('initial backup')"
template1
tar cvf pg_base_backup.tar /var/lib/pgsql/data
psql -c "SELECT pg_stop_backup()" template1
Copy the result file to slave and unpack
mv /var/lib/pgsql/pg_base_backup.tar /
tar xvf pg_base_backup.tar
On slave, delete /data/postmaster.pid

17. PostgreSQL
Master, /var/lib/pgsql/data/postgresql.conf
listen_addresses = '*'
max_wal_senders = 5
wal_keep_segments = 32
Master, /var/lib/pgsql/data/pg_hba.conf
host replication postgres 192.168.1.101/24 trust

18. PostgreSQL
On slave server /var/lib/pgsql/data/postgresql.conf
hot_standby = on
Add recovery.conf
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.100 port=5432
user=postgres'
should contain additional settings
Start servers
pg_ctl -D /var/lib/pgsql/data start

19. CouchDB
Access administration console Futon
<computer's IP address>:5984/_utils
In Configuration set it so it listens to all IP addresses
bind_address = 0.0.0.0
Create databases
In Replicator set
<another computer's IP
address>/database_on_other_computer
turn on Continuous, click Replicate

20. CouchDB
To do multimaster replication, repeat steps on other
computer
<another computer's IP
address>/database_on_first_computer
turn on Continuous
click Replicate

21. Zaključak
Impressions
usual MySQL vs PostgreSQL discussion
realisticaly: we won't choose a system solely based on
whose replication is better
for small projects and home usage, CouchDB
hr.linkedin.com/in/vatroslavmileusnic

22. Questions?