Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Evolution of MySQL Parallel Replication

1.572 Aufrufe

Veröffentlicht am

MySQL replication has evolved a lot in 5.6 ,5.7 and 8.0. This presentation focus on the changes made in parallel replication. It covers MySQL 8.0. It was presented at Mydbops database meetup on 04-08-2016 in Bangalore.

Veröffentlicht in: Ingenieurwesen
  • Loggen Sie sich ein, um Kommentare anzuzeigen.

Evolution of MySQL Parallel Replication

  1. 1. Evolution of MySQL Parallel replication Mydbops Database Meetup (Bangalore) Presenter Karthik P R Founder Mydbops www.mydbops.com info@mydbops.com
  2. 2. About Mydbops ● Founded in 2015, HQ in Bangalore India with 150+ customer base across the globe. ● Mydbops is on Database Consulting with core specialization on MySQL and MongoDB Administration and Support. ● We have expert team with 20+ certified DBA’s providing full time support and currently managing 300+ servers. ● Mydbops was created with a motto of developing a DevOPS model for Database administration offering 24*7 expert remote DBA support. ● We help organisations to architect and scale systems in MySQL/MongoDB by implementing the advanced technologies in industry which are completely open source.
  3. 3. Mydbops is into MySQL/MongoDB Support and Consulting. It is founded by experts who have scaled database at Yahoo! ,Percona and Datavail. We are providing an expert level support and 24*7 monitoring for MySQL databases and its related technologies like MariaDB , Percona ( also clustering ) . We support modern database technologies in MySQL which includes Galera ( Clustering ), Group Replication , SQL aware Load balancers like Maxscale / ProxySQL. About Mydbops
  4. 4. ` mysqlsupport@mydbops.com Our Clients www.mydbops.com
  5. 5. CEO / DB Architect About Me
  6. 6. ● Introduction : MySQL Replication ● Basics of Binary log . ● Parallel Replication ● Summary Table of Contents
  7. 7. ● One of the most popular feature ● Native and inbuilt ● Logical Replication ● Asynchronous in state. ● Semi Sync and Flow control ( Group Replication ) ● Read Scalability ● Native high availability feature. Introduction : MySQL Replication
  8. 8. Introduction : MySQL Replication
  9. 9. ● Binary log is the key for replication ● Filtered Replication ○ Binlog filter ○ Relay log filter ● Possible Architecture ○ Master - Slave ○ Master - Master ○ Multi master ( Multi Source ) ○ Intermediate Slave ○ Group Replication ● No parallelism by default ( Single threaded ) Introduction : MySQL Replication
  10. 10. ● Journal of the master writes. ● Contains the DDL and other writes. ● Records the statements in the order of transaction commit. ● Works independent of the engine. ● Formats : ROW and Statement ● Reproduced on the slaves as relay log Basics of Binary log
  11. 11. Basics of Binary log Execute Binlog Commit Two Phase Commit (2PC) 1. Changes are collected in transaction cache ( per connection ) 2. Transaction is committed to storage engine 3. Transaction cache is written to binary log (commit).
  12. 12. Sample binlog content ( MySQL 8.0 ) BEGIN /*!*/; # at 825 #180803 23:57:16 server id 101 end_log_pos 893 CRC32 0xbc8bc6b1 Table_map: `sbtest1`.`sbtest6` mapped to number 82 # at 893 #180803 23:57:16 server id 101 end_log_pos 1118 CRC32 0xdfabee92 Write_rows: table id 82 flags: STMT_END_F ### INSERT INTO `sbtest1`.`sbtest6` ### SET ### @1=1000001 /* INT meta=0 nullable=0 is_null=0 */ ### @2=501910 /* INT meta=0 nullable=0 is_null=0 */ ### @3='93213299669-86838228032-85626473166-33684960785-51919150459-30647403687-71837739046-18605717775-51951633177-01 652473758' /* STRING(480) meta=61152 nullable=0 is_null=0 */ ### @4='10076769068-66969333322-72197014465-67237974816-92730387311' /* STRING(240) meta=65264 nullable=0 is_null=0 */ # at 1118 #180803 23:57:16 server id 101 end_log_pos 1149 CRC32 0x8a6136cc Xid = 7532 COMMIT/*!*/; # at 1149 Basics of Binary log
  13. 13. Parallel Replication
  14. 14. Need for parallel replication. ● Effective usage of multi core machines ( avoid single core writes in slaves ) ● Use the modern disk and storage efficiently ○ RAID 1 ( 2 disks 2 IOs parallel) ○ SSD ( write IOs in parallel ) ● Faster replication ( minimal lag ) Slowness happens at SQL_thread ( SQL applier ) Parallel Replication
  15. 15. MySQL 5.6 MySQL 5.7 MySQL 8.0 Parallel Replication WriteSet Transaction writing different tuples Schema Based Schema local Transactions Logical Clock Binary log group commits
  16. 16. Schema based
  17. 17. MySQL 5.6 ( Schema based ) ● Two transactions of different schema can be parallelised ● Transaction are distributed based on per database basis. ● The transaction ordering can be different master and slave. ○ Master A1, B1, A2, B2, A3,B3 ○ Slave A1, B1, A2, A3, B2, B3 Parallel Replication
  18. 18. Parallel Replication Parallel Threads = N Coordinator Thread = 1 Worker Threads = (N-1) Schema based parallel replication
  19. 19. Schema based ( MySQL 5.6) ● How to Facilitate ? ○ slave_parallel_workers=N ( N > 1) ○ slave_parallel_type=”DATABASE” ( default ) ● Impact ○ Gaps in transaction order ○ Replication Crash Recovery ( GTID can be a saviour ) ○ Beware of backup used. Parallel Replication
  20. 20. Logical Clock
  21. 21. Logical clock ( MySQL 5.7) ● Based on binlog group commit ● Binlog_group_commit_sync_delay and binlog_group_commit_sync_no_delay_count. ● Parallelism interval is based on last_committed and sequence number in binary logs ● Tuning need binlog analysis ( no counters inbuilt ) ● Slave_preserve_commit_order avoids gaps. https://jfg-mysql.blogspot.com/2017/02/metric-for-tuning-parallel-replication-mysql-5-7.html Parallel Replication
  22. 22. Logical clock ( MySQL 5.7) Parallel Replication Last_committed : Start with 0 and reset at end of binlog Sequence_number : Start with 1 and reset at end of binlog
  23. 23. Schema based ( MySQL 5.7) ● How to Facilitate ? ○ slave_parallel_workers=N ( N > 1) ○ slave_parallel_type=”LOGICAL_CLOCK” ( default ) ○ Tune the binlog group commit. ● Impact ○ Can slow down the master writes. ○ Gaps in transaction order ( Slave_preserve_commit_order can avoid it ). ○ slave_preserve_commit order needs log_slave_updates. ○ longer transactions can block or slow down Parallel Replication
  24. 24. Writeset
  25. 25. Write set ( MySQL 8.0 and 5.7.22) ● Transaction that affect different rows ( tuples ) can be parallelized. ● The dependency of each transaction is tracked ( history ). ● The sequence number of the last transaction which updates current row is tracked. ● Faster and better mode of parallel replication. write set : Any transaction with different tuples can be parallelized. write set session : Updates from same session can’t be reordered Parallel Replication
  26. 26. Parallel Replication Master - Writes
  27. 27. Parallel Replication Slave ( Writeset )
  28. 28. Parallel Replication Slave ( Writeset Session)
  29. 29. Write set ( MySQL 8.0) ● How to Facilitate ? ○ slave_parallel_workers=N ( N > 1) ○ slave_parallel_type=”LOGICAL_CLOCK” ( default ) ○ binlog_transaction_dependency_tracking= writeset/writeset_session ○ transaction_writeset_extraction= XXHASH64/MURMUR32 To Disable write set : ● binlog_transaction_dependency_tracking= COMMIT_ORDER Parallel Replication
  30. 30. Limitations : ● Primary key is needed on all table. ● Do not work well with tables with foreign keys. ● Works only with the ROW based replication ( Default in 8.0). Additional Info : ● Used in group replication ● 5.7.22 has writeset ( backported from MySQL 8.0) Parallel Replication
  31. 31. Summary
  32. 32. Summary : ● Parallel replication minimize the lag. ● It’s not easy to setup ( tune ) but worth the efforts. ● Schema based can be used in multi tenant environments. ● Logical Clock (5.7) needs more tuning. ● Writeset (8.0) gives better performance. Summary
  33. 33. General tips over parallel replication: ● Use only InnoDB tables. ● Enable GTID’s for crash safe replication. ● relay_log_recovery should be enabled. ● master_info and slave_info_repositorty should be in table. ● ROW based replication is faster and better for most cases. ● Ensure your tables have a primary key. General Tips
  34. 34. ● https://jfg-mysql.blogspot.com Jean-Francois Slides and blogs on parallel replication. ● https://mysqlhighavailability.com/improving-the-parallel-applier-with-writeset-ba sed-dependency-tracking/ Writeset replication. ● http://mysqlmusings.blogspot.com/2012/06/binary-log-group-commit-in-mysql-5 6.html Binlog Group commit. Resources
  35. 35. info@mydbops.com www.mydbops.com 080-48505683 Contact us
  36. 36. Thank You !!!

×