Tungsten University: Setup & Operate Tungsten Replicator
•
1 like•1,703 views
Do you have the background necessary to take full advantage of Tungsten Replicator in your environments? Tungsten offers enterprise-quality replication features in an open source package hosted on Google Code. This virtual course will teach you how to set up innovative topologies that solve complex replication problems. We start with single MySQL servers running MySQL replication and show a simple path migration path to Tungsten. Course Topics - Checking host and MySQL prerequisites - Downloading code from http://code.google.com/p/tungsten-replicator/ - Installation using the tungsten-installer utility - Transaction filtering using standard filters as well as customized filters you write yourself - Enabling and managing parallel replication - Configuring multi-master and fan-in using multiple replication services - Backup and restore integration - Troubleshooting replication problems - Logging bugs and participating in the Tungsten Replicator community. Replication is a powerful technology that takes knowledge and planning to use effectively. We give you the background that makes replication easier to set up, and allows you to take full advantage of the Tungsten Replicator benefits. Learn how to configure and use it more effectively for your projects in the cloud as well as on-premise hardware.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Tungsten University: Setup & Operate Tungsten Replicator
Similar to Tungsten University: Setup & Operate Tungsten Replicator (20)
More from Continuent
More from Continuent (20)
Recently uploaded
Recently uploaded (20)
Tungsten University: Setup & Operate Tungsten Replicator
- 1. Tungsten University: Set Up and Operate Tungsten Replication Robert Hodges, CEO ©Continuent 2013
- 2. Introducing Continuent • The leading provider of clustering and replication for open source DBMS • Our Product: Continuent Tungsten • Clustering - Commercial-grade HA, performance scaling and data management for MySQL • Replication - Flexible, high-performance data movement ©Continuent 2013 2
- 4. Tungsten Replicator Overview Master Replicator Download transactions via network DBMS Logs (Transactions + Metadata) Slave Replicator Apply using JDBC ©Continuent 2013 THL THL (Transactions + Metadata) 4
- 5. Tungsten Replication Service Pipeline Stage Extract Filter Master DBMS ©Continuent 2013 Stage Apply Extract Filter Transaction History Log Stage Apply Extract Filter In-Memory Queue 5 Apply Slave DBMS
- 6. Overview of Installation Process 1. Set up hosts 2. Prepare MySQL replicas 3. Download software 4. Install using tpm utility Amazon Setup: https://docs.continuent.com/wiki/display/TEDOC/ Preparing+EC2+Servers ©Continuent 2013 6
- 7. Old Way of Installing Tungsten... • tungsten-installer - Perform initial installation • con!gure-service - Set up individual replication services • update - Change parameters on services after installation The tpm utility replaces all of this! ©Continuent 2013 7
- 8. How tpm Works Staging copy of files db1 db2 check prereqs copy code configure topology ©Continuent 2013 db3 8
- 9. What Is Tungsten Doing? ©Continuent 2013 9
- 10. Commands to Check Replication • • • • • • ©Continuent 2013 trepctl services -- All services trepctl status -- Overall pipeline status trepctl status -name stages -- Stage summary trepctl status -name stores -- Store summary trepctl status -name tasks -- Task detail trepctl [-service name] heartbeat -- Is it Alive? 10
- 11. Changing Replicator Con!guration ©Continuent 2013 11
- 12. Commands to Change Con!guration • • tpm update - Alter con"guration tpm update - Upgrade the replicator to a new version Hint: Don’t change replicator configuration files directly... ©Continuent 2013 12
- 13. Recon!guring a Replicator • No restart on local host trepctl offline tools/tpm update ... --no-restart trepctl configure trepctl online • With automatic restart tools/tpm update ... ©Continuent 2013 13
- 14. Enabling Parallel Replication ©Continuent 2013 14
- 15. Parallel Replication of Sharded Data Pipeline Stage Extract Filter Master DBMS ©Continuent 2013 Stage Apply Extract Filter Transaction History Log Apply Stage Extract Filter Extract Filter Extract Filter Parallel Queue 15 Apply Apply Apply Slave DBMS
- 17. The Scene of the Crime trepctl -host db2 status Processing status command... NAME VALUE -------... pendingError : Event application failed: seqno=1957469 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master pendingErrorCode : NONE pendingErrorEventId : mysql-bin.000008:0000000461736666;0 pendingErrorSeqno : 1957469 pendingExceptionMessage: java.sql.SQLException: Statement failed on slave but succeeded on master create table trouble(id int primary key) /* ___SERVICE___ = [university] */ ... Finished status command... ©Continuent 2013 17
- 18. Correcting problems 1. Look at the transaction (thl list -seqno NNN) 2. Look at the replicator message log 3. Fix the problem and go online OR 4. Skip over the problem and go online OR 5. Reprovision the slave ©Continuent 2013 18
- 20. Simple Facts about Logs • • • • • Replicator log = THL (Transaction History Log) One THL per replication service 2x the size of MySQL binlog Default retention of 7 days Take service o#ine to move or delete logs safely • The thl tool shows you log contents ©Continuent 2013 20
- 22. Resetting and Reinstalling Reset 1. Stop replicator 2. Remove THL "les 3. Remove tungsten_<svc> databases Reinstall 1. All of the above, plus delete /opt/continuent/* 2. Rerun installation ©Continuent 2013 22
- 23. Backup and Restore ©Continuent 2013 23
- 24. What Backup Tools Does MySQL Have? MySQL Server Process mysqldump (dataset < 10Gb) xtrabackup (10Gb < dataset < 1Tb tar/rsync (dataset < 1Tb) DBMS Files file system snapshot (1Tb << dataset) File System ©Continuent 2013 24
- 26. How Do I Install Fan-In Replication? db1 Replicator Service db1 Replicator Service db1 db2 Service db2 Replicator Service db2 ©Continuent 2013 db3 26
- 27. How Do I install Multi-Master? db1 Replicator Service db1 Service db2 db3 Service db3 Service db1 Service db2 db2 Service db3 Service db1 Service db2 Service db3 Replicator ©Continuent 2013 27
- 28. Tpm Understands Topologies! • Use --topology argument! Fan-In --topology=fan-in ©Continuent 2013 All-Masters Star --topology=all-masters --topology=star 28
- 29. For More Help: Tungsten Cookbook • • • • • Installs complex topologies automatically Directions in cookbook/README Parameters in cookbook/USER_VALUES.sh Node names in COMMON_NODES.sh Installation script for each supported topology http://datacharmer.blogspot.com/2013/07/tpm-multimaster-composer.html ©Continuent 2013 29
- 30. Joining the Community ©Continuent 2013 30
- 31. Tungsten Replicator is Open Source • Project home: http://code.google.com/p/tungsten-replicator/ • Log bugs, "nd builds, post in replicator discussion group • Documentation: https://docs.continuent.com/wiki/display/TEDOC/ Tungsten+Documentation+Home ©Continuent 2013 31
- 33. More Webinars and Talks! • How Gittigidiyor (a subsidiary of eBay) Replicates Data in Real Time from MySQL to Oracle (6 November 2013 10am EDT/3pm GMT) • Come see us at Percona Live London! (11-12 November 2013) Send any feedback to: tu@continuent.com ©Continuent 2013 33
- 34. 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://"yingclusters.blogspot.com/ Continuent Web Page: http://www.continuent.com Tungsten Replicator 2.0: http://code.google.com/p/tungsten-replicator ©Continuent 2012.