Weitere Ă€hnliche Inhalte Ăhnlich wie Tungsten University: MySQL Multi-Master Operations Made Simple With Tungsten Replicator 2.1.1 (20) KĂŒrzlich hochgeladen (20) Tungsten University: MySQL Multi-Master Operations Made Simple With Tungsten Replicator 2.1.12. ©Continuent 2013
Introducing Continuent
2
âą The leading provider of clustering and
replication for open source DBMS
âą Our Product: Continuent Tungsten
âą Clustering - Commercial-grade HA, performance
scaling and data management for MySQL
âą Replication - Flexible, high-performance data
movement
4. ©Continuent 2013
Tungsten Replicator Overview
4
Master
(Transactions + Metadata)
Slave
THL
DBMS
Logs
Replicator
(Transactions + Metadata)
THLReplicator
Download
transactions
via network
Apply using JDBC
5. ©Continuent 2013
Master Replication Service
5
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
MySQL
Master
Transaction
History Log
In-Memory
Queue
Slave
ReplicatorsBinlog
tcp/ip
6. ©Continuent 2013
Slave Replication Service
6
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
Transaction
History Log
In-Memory
Queue
Slave
DBMS
Master
Replicator
tcp/ip
9. ©Continuent 2013
Tungsten Replicator 2.1.1
âą Releases as GA on August 21, 2013
âą Main features:
âą installation with tpm
âą security layer
âą full support for MySQL 5.6
âą improved management tools
âą trepctl properties
âą trepctl services [-full] [-json]
âą trepctl clients
âą trepctl status [-json]
âą thl list [-headers] [-json]
9
11. ©Continuent 2013
Overview of Installation Process
1. Set up hosts
2. Prepare MySQL replicas
3. Download software
4. Install using tungsten-installer & con!gure-
service tpm
11
Amazon Setup:
https://docs.continuent.com/wiki/display/TEDOC/
Preparing+EC2+Servers
17. ©Continuent 2013
4 nodes all-masters
17
host1
host4
host2
host3
alpha
alpha
bravo
bravo
charlie
charlie
delta
delta
bravobravo
alpha
delta
delta
charlie
charlie
alpha
18. ©Continuent 2013 18
Multi-master installation
(4 nodes) until TR 2.1.0
~ 8 minutes
tungsten-installer
host1 - alpha 1
tungsten-installer
host2 - bravo 2
tungsten-installer
host3 - charlie 3
tungsten-installer
host4 - delta 4
con!gure-service
host1 - bravo 5
con!gure-service
host1 - charlie 6
con!gure-service
host1 - delta 7
con!gure-service
host2 - alpha 8
con!gure-service
host2 - charlie 9
con!gure-service
host2 - delta 10
con!gure-service
host3 - alpha 11
con!gure-service
host3 - bravo 12
con!gure-service
host3 - delta 13
con!gure-service
host4 - alpha 14
con!gure-service
host4 - bravo 15
con!gure-service
host4 - charlie 16
using tungsten-installer
19. ©Continuent 2013 19
./tools/tpm install my_cluster
--topology=all-masters
--home-directory=$TUNGSTEN_HOME
--replication-user=tungsten
--replication-password=secret
--masters=host1,host2
--master-services=alpha,bravo
--start
Multi-master installation
(2 nodes) with TR 2.1.1
~ 40 seconds
using tpm
20. ©Continuent 2013 20
./tools/tpm install my_cluster
--topology=all-masters
--home-directory=$TUNGSTEN_HOME
--replication-user=tungsten
--replication-password=secret
--masters=host1,host2,host3,host4
--master-services=alpha,bravo,charlie,delta
--start
Multi-master installation
(4 nodes) with TR 2.1.1
~ 45 seconds
using tpm
21. ©Continuent 2013
One more reason why tpm is better
âą Security:
âą all data in transit (transaction history logs, or THL) is
encrypted using SSL
âą all administrative tra"c is encrypted with SSL
âą Transparent to the user
âą Independent of the database server (works also for
heterogeneous replication)
21
24. ©Continuent 2013
Installing with security
1: create keys
24
./cookbook/create-security-keys HarryPotter
...
...
# Options for tpm (stored in ./cookbook/security.options)
--thl-ssl=true
--rmi-ssl=true
--rmi-authentication=true
--rmi-user=cookbook
--java-keystore-password=HarryPotter
--java-truststore-password=HarryPotter
--java-truststore-path=$PWD/cookbook/truststore.ts
--java-keystore-path=$PWD/cookbook/keystore.jks
--java-jmxremote-access-path=$PWD/cookbook/jmxremote.access
--java-passwordstore-path=$PWD/cookbook/passwords.store
25. ©Continuent 2013
Installing with security - 2: run tpm
25
./tools/tpm install alpha
--topology=master-slave
--home-directory=/opt/continuent/replicator
--replication-user=tungsten
--replication-password=secret
--master=host1
--slaves=host2,host3,host4
$(cat ./cookbook/security.options)
--start
./tools/tpm install my_cluster
--topology=all-masters
--home-directory=$TUNGSTEN_HOME
--replication-user=tungsten
--replication-password=secret
--masters=host1,host2,host3,host4
$(cat ./cookbook/security.options)
--master-services=alpha,bravo,charlie,delta
--start
29. ©Continuent 2013
master/slave using tpm
29
./tools/tpm install alpha
--topology=master-slave
--home-directory=/opt/continuent/replicator
--replication-user=tungsten
--replication-password=secret
--master=host1
--slaves=host2,host3,host4
--start
31. ©Continuent 2013
fan-in using tpm
31
./tools/tpm install many_towns
--replication-user=tungsten
--replication-password=secret
--home-directory=/opt/continuent/replication
--masters=host1,host2,host3
--slaves=host4
--master-services=alpha,bravo,charlie
--topology=fan-in
--start
33. ©Continuent 2013
multi-master using tpm
33
../tools/tpm install musketeers
--reset
--topology=all-masters
--home-directory=/opt/continuent/replicator
--replication-user=tungsten
--replication-password=secret
--masters=host1,host2
--master-services=alpha,bravo
--start
36. ©Continuent 2013
extending multi-master
(2: add a node)
36
./tools/tpm configure musketeers
--topology=all-masters
--home-directory=/opt/continuent/replicator
--replication-user=tungsten
--replication-password=secret
--masters=host1,host2,host3
--master-services=alpha,bravo,charlie
--start
./tools/tpm update
37. ©Continuent 2013
Extending further ...
37
host1
host4
host2
host3
alpha
alpha
bravo
bravo
charlie
charlie
delta
delta
bravobravo
alpha
delta
delta
charlie
charlie
alpha
host1
host2
host3
alpha
alpha
bravo
bravo
charlie
charlie
bravo
alpha
charlie
38. ©Continuent 2013
extending multi-master
(3: add one more node)
38
./tools/tpm configure musketeers
--topology=all-masters
--home-directory=/opt/continuent/replicator
--replication-user=tungsten
--replication-password=secret
--masters=host1,host2,host3,host4
--master-services=alpha,bravo,charlie,delta
--start
./tools/tpm update
39. ©Continuent 2013
Install a Star Topology
39
./cookbook/install_star
host1
host4
host2
host3
alpha
bravo
charlie
charlie
delta
bravo
alpha
delta
charlie
charlie
hub
40. ©Continuent 2013
star using tpm
40
./tools/tpm install constellation
--replication-user=tungsten
--replication-password=secret
--home-directory=/opt/continuent/replication
--masters=host1,host2,host4
--hub=host3
--hub-service=charlie
--master-services=alpha,bravo,delta
--topology=star
--start
42. ©Continuent 2013
all-masters with attached slaves
42
host1
host4
host2
host3
alpha
alpha
bravo
bravo
bravo
alpha
data from goes to
host1 host2,host3
host2 host1,host4
43. ©Continuent 2013
all-masters with attached slaves
43
host1
host4
host2
host3
alpha
alpha
bravo
bravo
charlie
charlie
delta
delta
bravobravo
alpha
delta
delta
charlie
charlie
alpha
host6host5 deltacharlie
data from goes to
host1 host2,host3,host4
host2 host1,host3,host4
host3 host1,host2,host4,host5
host4 host1,host2,host3,host6
44. ©Continuent 2013
four all-masters with attached slaves
1: con!gure defaults
44
#1
./tools/tpm configure defaults
--reset
--replication-user=tungsten
--replication-password=secret
--home-directory=/home/tungsten/installs/cookbook
--start
45. ©Continuent 2013
four all-masters with attached slaves
2: con!gure main all-masters
45
#2
./tools/tpm configure four_musketeers
--masters=host1,host2,host3,host4
--master-services=alpha,bravo,charlie,delta
--topology=all-masters
47. ©Continuent 2013
four all-masters with attached slaves
4: con!gure second slave
47
#4
./tools/tpm configure delta
--hosts=host4,host6
--slaves=host6
--master=host4
#5
./tools/tpm install
49. ©Continuent 2013
all-masters with attached mongoDB slave
49
host1
host4
host2
host3
alpha
alpha
bravo
bravo
charlie
charlie
delta
delta
bravobravo
alpha
delta
delta
charlie
charlie
alpha
host5 alpha
50. ©Continuent 2013
all-masters with mongodb slave
1: install masters with heterogeneous options
50
./tools/tpm configure all_masters
--reset
--log=timestamp
--replication-port=3306
--replication-user=tungsten
--replication-password=secret
--home-directory=/home/tungsten/installs/cookbook
--repl-disable-relay-logs=true -a
--masters=ubuntu1,ubuntu2,ubuntu3,ubuntu4
--master-services=alpha,bravo,charlie,delta
--topology=all-masters
--java-file-encoding=UTF8
--mysql-use-bytes-for-string=false
--mysql-enable-enumtostring=true
--mysql-enable-settostring=true
--svc-extractor-filters=colnames,pkey
--start-and-report
51. ©Continuent 2013
all-masters with mongodb slave
2: add mongodb slave and install
51
./tools/tpm configure alpha
--datasource-type=mongodb
--hosts=host1
--master=ubuntu1
--replication-port=6700
--home-directory=~/deploy/mongodb
--slaves=host1
--role=slave
--master-thl-port=2112
--thl-port=2113
--rmi-port=10100 --start-and-report
./tools/tpm install --verbose
57. ©Continuent 2013
Tungsten Cookbook
$ cat USER_VALUES.sh
# User defined values for the cluster to be
installed.
export TUNGSTEN_BASE=$HOME/installs/cookbook
export DATABASE_USER=tungsten
export BINLOG_DIRECTORY=/var/lib/mysql
export MY_CNF=/etc/my.cnf
export DATABASE_PASSWORD=secret
export DATABASE_PORT=3306
export TUNGSTEN_SERVICE=cookbook
export RMI_PORT=10000
export THL_PORT=2112
export START_OPTION=start
57
58. ©Continuent 2013
Sample all-masters Installation
âą edit cookbook/COMMON_NODES.sh
âą edit cookbook/USER_VALUES.sh
âą run cookbook/install_all_masters
âą and then:
âą run cookbook/show_cluster
âą run cookbook/test_cluster
58
61. ©Continuent 2013
Tungsten Replicator is Open Source
61
âą Project home:
http://code.google.com/p/tungsten-replicator/
âą Log bugs, !nd builds, post in replicator discussion
group
âą Documentation:
https://docs.continuent.com/wiki/display/TEDOC/
Tungsten+Documentation+Home
63. ©Continuent 2013
Tungsten University Sessions
63
âą Con!gure & Provision Tungsten Clusters
- Thursday September 5th
- Tuesday September 10th
âą Setup & Operate Tungsten Replicator
- Thursday October 17th
- Tuesday October 22nd
âą Send any feedback to: tu@continuent.com
64. ©Continuent 2012.
Continuent Web Page:
http://www.continuent.com
Tungsten Replicator 2.1.1:
http://code.google.com/p/tungsten-replicator
Our Blogs:
http://scale-out-blog.blogspot.com
http://datacharmer.blogspot.com
http://www.continuent.com/news/blogs
560 S. Winchester Blvd., Suite 500
San Jose, CA 95128
Tel +1 (866) 998-3642
Fax +1 (408) 668-1009
e-mail: sales@continuent.com