Weitere ähnliche Inhalte Ähnlich wie Set Up & Operate Tungsten Replicator (20) Kürzlich hochgeladen (20) Set Up & Operate Tungsten Replicator2. ©Continuent 2014
Introducing Continuent
2
• The leading provider of clustering and
replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance
scaling and data management for MySQL
• Replication - Flexible, high-performance data
movement
4. ©Continuent 2014
Tungsten Replicator Overview
4
Master
(Transactions + Metadata)
Slave
THL
DBMS
Logs
Replicator
(Transactions + Metadata)
THLReplicator
Download
transactions
via network
Apply using JDBC
5. ©Continuent 2014
Master Replication Service
5
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
MySQL
Master
Transaction
History Log
In-Memory
Queue
Slave
ReplicatorsBinlog
tcp/ip
6. ©Continuent 2014
Slave Replication Service
6
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
Transaction
History Log
In-Memory
Queue
Slave
DBMS
Master
Replicator
tcp/ip
7. ©Continuent 2014
EC2 Installation Using Github project
1.Set up Github vagrant project
git clone https://github.com/continuent/
continuent-vagrant
2.Pick EC2 Vagrantfile and fill in keys
3.Copy examples/MasterSlave/default.pp to
manifests/
4.Run launch.sh
7
(Follow directions in README.md)
8. ©Continuent 2014
Tpm installation overview
8
/opt/continuent/etc/tungsten/tungsten.ini
check prereqs
[copy code]
configure topology
[defaults]!
user=tungsten!
home-directory=/opt/continuent!
datasource-user=tungsten!
datasource-password=secret!
skip-validation-check=MySQLPermissionsCheck!
start-and-report=true!
!
[university]!
master=db1!
slaves=db2,db3!
topology=master-slave!
tpm update
(Installation)
10. ©Continuent 2014
Commands to Check Replication
• trepctl services -- All services
• trepctl status -- Overall pipeline status
• trepctl status -name stages -- Stage summary
• trepctl status -name stores -- Store summary
• trepctl status -name tasks -- Task detail
• trepctl [-service name] heartbeat -- Is it Alive?
10
12. ©Continuent 2014
Reconfiguring a Replicator
• Edit /etc/tungsten/tungsten.ini
• No restart on local host
trepctl offline!
tpm update --no-restart!
trepctl configure!
trepctl online!
!
• With automatic restart
tpm update
12
Don’t change replicator configuration files directly!
14. ©Continuent 2014
Parallel Replication of Sharded Data
14
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Stage
Pipeline
Transaction
History Log
Parallel
Queue
Slave
DBMS
Extract Filter Apply
Extract Filter Apply
Extract Filter Apply
Master
Replicator
tcp/ip
16. ©Continuent 2014
The Scene of the Crime
16
trepctl -host db2 status
Processing status command...
NAME VALUE
---- -----
...
pendingError : Event application failed:
seqno=1957469 fragno=0 message=java.sql.SQLException:
Statement failed on slave but succeeded on master
pendingErrorCode : NONE
pendingErrorEventId : mysql-bin.000008:0000000461736666;0
pendingErrorSeqno : 1957469
pendingExceptionMessage: java.sql.SQLException: Statement
failed on slave but succeeded on master
create table trouble(id int primary
key) /* ___SERVICE___ = [university] */
...
Finished status command...
17. ©Continuent 2014
Correcting problems
1.Look at the transaction (thl list -seqno NNN)
2.Look at the replicator message log
3.Fix the problem and go online
OR
4.Skip over the problem and go online
OR
5.Reprovision the slave
17
19. ©Continuent 2014
Simple Facts about Logs
• Replicator log = THL (Transaction History Log)
• One THL per replication service
• 2x the size of MySQL binlog
• Default retention of 7 days
• Take service offline to move or delete logs
safely
• The thl tool shows you log contents
19
24. ©Continuent 2014
What Backup Tools Does MySQL Have?
24
MySQL
Server
Process
File System
DBMS Files
mysqldump
(dataset < 10Gb)
xtrabackup
(10Gb < dataset < 1Tb
tar/rsync
(dataset < 1Tb)
file system snapshot
(1Tb << dataset)
26. ©Continuent 2014
How Do I Install Fan-In Replication?
26
db1
Replicator
db3
Service db1
db2
Replicator
Service db2
Replicator
Service db1
Service db2
27. ©Continuent 2014
How Do I install Multi-Master?
27
db1 Replicator
Service db1
Service db2
Service db3
db3
Service db1
Service db2
Service db3
db2
Replicator
Service db1
Service db2
Service db3
29. ©Continuent 2014
Tpm install from staging directory
29
db1
db2
db3
Staging copy
of files
check prereqs
copy code
configure topology
tpm configure ...
tpm update
30. ©Continuent 2014
Commands to Perform Staging Install
30
$ wget http://downloads.tungsten-replicator.org/download.php?
file=tungsten-replicator-2.2.1-403.tar.gz
!
$ tar -xvzf tungsten-replicator-2.2.1-403.tar.gz
!
$ ln -s tungsten-replicator-2.2.1-403 tungsten
!
$ tungsten/tools/tpm configure defaults --reset
--user=tungsten --home-directory=/opt/continuent
--datasource-user=tungsten --datasource-password=secret
--skip-validation-check=MySQLPermissionsCheck
--start-and-report=true
!
$ tungsten/tools/tpm configure fanin
--members=db1,db2,db3
--masters=db1,db2
--master-services=db1,db2
--topology=fan-in
!
$ tungsten/tools/tpm update
32. ©Continuent 2014
Tungsten Replicator is 100% Open
Source
32
• Project home:
http://code.google.com/p/tungsten-replicator/
• Log bugs, find builds, post in discussion group
• Documentation:
https://docs.continuent.com
• Install automation:
https://github.com/continuent
!
34. ©Continuent 2014
We Do Clustering Too
34
Tungsten clusters combine off-
the-shelf open source DBMS
servers into data services with:
!
• 24x7 data access
• Scaling of load on replicas
• Simple management commands
!
...without app changes or data
migration
Amazon
US West
apache
/php
GonzoPortal.com
Connector Connector
35. ©Continuent 2014
Conclusion
35
• Tungsten Replicator is an advanced, open source
replicator
• Use tpm to install either from staging or using
batch commands
• Use trepctl to manage replication
• Tungsten Replicator has capabilities far beyond
what we showed today
Use your imagination!
36. ©Continuent 2014
Continuent Web Page:
http://www.continuent.com
!
Tungsten Replicator:
http://code.google.com/p/tungsten-replicator
Our Blogs:
http://scale-out-blog.blogspot.com
http://datacharmer.org/blog
http://www.continuent.com/news/blogs
http://flyingclusters.blogspot.com/
560 S. Winchester Blvd., Suite 500
San Jose, CA 95128
Tel +1 (866) 998-3642
Fax +1 (408) 668-1009
e-mail: sales@continuent.com