Lightning Talk slide for Cloudpack Night #6

1. CUSTOM NON-RDS MULTI-AZ
MYSQL REPLICATION
Cloudpack Night #6
5/17/2013

2. ABOUT ME (@IJIN)
• Michael H. Oshita
• Japanese American
• In Japan since 2002
• Software/Infra/Cloud Engineer
• http://ijin.github.io

3. CERTIFIEDTODAY!

4. COMPONENTS
• AWSVPC (Routing-Based HA Pattern)
• MySQL Master-Slave Replication
• MHA (Master HA Manager for MySQL)
• no RDS!!

5. MHA
Master
Slave
Slave
Manager
Manager detects failure on master

6. Master
Master
Slave
Manager
MHA
Slave is promoted to master after binlogs are applied
binlog
binlog
binlog

7. "Starting master failover."
"* Phase 1: Configuration Check Phase..n"
"* Phase 2: Dead Master Shutdown Phase..n"
==> Delete routing table (master_ip_failover)
"* Phase 3: Master Recovery Phase..n"
"* Phase 3.1: Getting Latest Slaves Phase..n"
"* Phase 3.2: Saving Dead Master's Binlog Phase..n"
"* Phase 3.3: Determining New Master Phase..n"
==> Change routing table (master_ip_failover)
"* Phase 4: Slaves Recovery Phase..n"
MHA
FAILOVER SEQUENCE

8. “SHARE A PRIVATE IP ACROSS AVAILABILITY ZONES”
http://d.hatena.ne.jp/c9katayama/20111225/1324837509
#ヤマン ++

9. • Storage engines & distributions
• Instance types
• Fast
• Efficient
WHY NOT RDS?
→ FLEXIBILITY!!

10. • Percona Server (XtraDB)
• MariaDB
• TokuDB
• Mroonga
STORAGE ENGINES &
DISTRIBUTIONS

11. • hi1.xlarge
• m3.xlarge
• c1.xlarge
• etc.
INSTANCETYPES

12. • RoutingTable changes pretty quickly
• Total failover in roughly <20s (RDS takes 3-6m)
• Non-DNS based
• Apps can be configured with a single IP
• no need to worry about internal DNS problems
FAST FAILOVER

13. • Minimum of 2 (web+db) servers
• Warm up (replication booster)
• Fine tuning
• Server Logs
EFFECIENT

14. DEMO

15. CONSIDERATIONS
• Backups → Xtrabackup, copy binlogs to s3
• Point inTime Recovery → Automate with Chef
• Provisioning Replicas → Automate with Chef
• Some learning curve
• API backplane a SPoF

16. THANKYOU!