Part II of our free self-training slides on MySQL Cluster. In this part we cover 'Detailed Concepts': * Data Distribution & Partitioning * Two Phase Commit Protocol * Transaction Resources

1. MySQL Cluster Training
presented by severalnines.com
Address:
Contact: SeveralninesAB
Jean-Jérôme Schmidt c/o SICS, Box 1263
Email: services@severalnines.com Isafjordsgatan22
SE-164-29 Kista
Copyright 2011 Severalnines AB Control your database infrastructure 1

2. Introduction
• At Severalnines, we believe in sharing information
and knowledge; we all come from an open source
background
• We know a lot of things about MySQL Cluster and
think that MySQL Cluster is a great technology
• These free MySQL Cluster Training slides are a
contribution of ours to the knowledge and information
sharing that‟s common practice in the open source
community
• If you have any questions on these slides or would
like to book an actual training class, please contact
us at: services@severalnines.com
Copyright 2011 Severalnines AB Control your database infrastructure 2

3. Training Slides - Concept
• Over the coming weeks we will be chronologically
releasing slides for the different sections of our
MySQL Cluster Training program on our website.
• The full agenda of the training with all of its modules
is outlined in the next slides so that you can see what
topics will be covered over the coming weeks.
• Particularly specialised topics such as Cluster/J or
NDB API are not fully covered in the slides. We
recommend our instructor-led training classes for
such topics.
• Please contact us for more details:
services@severalnines.com
Copyright 2011 Severalnines AB Control your database infrastructure 3

4. Full Training Agenda (1/4)
• MySQL Cluster Introduction
– MySQL eco system
– Scale up, scale out, and sharding
– MySQL Cluster Architecture
– Use cases
– Features
– Node types and Roles
• Detailed Concepts
– Data Distribution
– Verifying data distribution
– Access Methods
– Partitioning
– Node failures and failure detection
– Network Partitioning
– Transactions and Locking
– Consistency Model
– Redo logging and Checkpointing
Copyright 2011 Severalnines AB Control your database infrastructure 4

5. Agenda (2/4)
• Installing MySQL Cluster
– Setting up MySQL Cluster
– Starting/stopping nodes
– Recovery and restarts
– Upgrading configuration
– Upgrading Cluster
• Performance Tuning (instructor-led only; contact us at services@severalnines.com)
– Differences compared to Innodb/MyISAM
– Designing efficient and fast applications
– Identifying bottlenecks
– Tweaking configuration (OS and MySQL Cluster)
– Query Tuning
– Schema Design
– Index Tuning
Copyright 2011 Severalnines AB Control your database infrastructure 5

6. Agenda (3/4)
• Management and Administration
– Backup and Restore
– Geographical Replication
– Online and offline operations
– Ndbinfo tables
– Reporting
– Single user mode
– Scaling Cluster
• Disk Data
– Use cases
– Limitations
– Best practice configuration
• Designing a Cluster
– Capacity Planning and Dimensioning
– Hardware recommendations
– Best practice Configuration
– Storage calculations
Copyright 2011 Severalnines AB Control your database infrastructure 6

7. Agenda (4/4)
• Resolving Issues
– Common problems
– Error logs and Tracefiles
– Recovery and Escalation procedures
• Connectivity Overview
– NDBAPI
– Cluster/J
– LDAP
• Severalnines Tools
– Monitoring and Management
– Benchmarking
– Sandboxes
– Configuration and capacity planning
• Conclusion
Copyright 2011 Severalnines AB Control your database infrastructure 7

8. Agenda: Lab Exercises
(only applicable to instructor-led training classes)
• Lab Exercises
– Installing and Loading data into MySQL Cluster
– Starting/stopping nodes, recovery
– Query tuning
– Backup and Restore
– Configuration Upgrade
• Would you like to try something particular?
– This is possible too, speak with your instructor
Copyright 2011 Severalnines AB Control your database infrastructure 8

9. Prerequisites
• Readers / Participants have understanding of SQL and basic database concepts.
• Laptops/PCs for hands-on exercises
• Linux: 1GB RAM
• Windows: 2GB RAM
• Approx. 20GB disk space and Virtualbox installed.
• Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads
• MySQL Cluster version 7.1 or later
Copyright 2011 Severalnines AB Control your database infrastructure 9

10. Part 2
Section 2: Detailed Concepts
Copyright 2011 Severalnines AB Control your database infrastructure 10

11. Topics covered in Part 2
• Data Distribution & Partitioning
• Two Phase Commit Protocol
• Transaction Resources
Copyright 2011 Severalnines AB Control your database infrastructure 11

12. Data Distribution and Partitioning
Copyright 2011 Severalnines AB Control your database infrastructure 12

13. MySQL Cluster Architecture
Application Application Application
R/I/U/D R/I/U/D R/I/U/D
Java, C++, LDAP
Application
R/I/U/D
Native API
DATA DATA
NODE NODE
Management
Management
Server
DATA DATA
Server
NODE NODE
Copyright 2011 Severalnines AB Control your database infrastructure 13

14. Data Distribution
• MySQL Cluster automatically partitions tables and
distributes the data on the partitions.
– It is possible to use user-defined partitioning.
• A table is divided into as many partitions as you have
data nodes and data is “striped” on the partitions.
• Hashing of the Primary Key determines the Partition
that should hold the data - automatic distribution.
• It is possible to hash on part of the primary key –
great for sharding and locality of data!
Copyright 2011 Severalnines AB Control your database infrastructure 14

15. Partitioning and Data Distribution
• With two data nodes, data node 1 and data node 2,
each table is divided into two Partitions.
• Subidis the Primary Key.
• Hashing on the Primary Key (subid) determines the
Partition. It is also possible to hash on part of the PK.
– Odd PKs Green
– Even PKs Red
STORAGE LAYER
subid data
DATA DATA
NODE 1 NODE 2 1 A Partition 0
3 B
2 C Partition 1
P0 P1 4 D
Copyright 2011 Severalnines AB Control your database infrastructure 15

16. Replicas
• To provide for redundancy and fast failover partitions
are synchronouslyreplicated.
• Two replicas (two copies of data) is most common
– it is possible to have one (previous slide), two, three and four
replicas.
– NoOfReplicas=2
STORAGE LAYER
DATA DATA
NODE 1 NODE 2
P0 S0
Copyright 2011 Severalnines AB Control your database infrastructure 16

17. Replicas
• To provide for redundancy and fast failover partitions
are synchronouslyreplicated from a PRIMARY to a
SECONDARY partition.
– When a change ( ) is made on P0 it is synchronously
replicated to S0.
– The change is persisted at transaction COMMIT.
– Either both P0 or S0 is update or NOTHING AT ALL
STORAGE LAYER
DATA DATA
NODE 1 NODE 2
P0 S0
Copyright 2011 Severalnines AB Control your database infrastructure 17

18. Data Distribution - disk logging
• After the COMMIT, the data is in main memory.
– But changes are REDO LOGGED, but the REDO LOG is
synced to disk every xth ms (1000 ms is recommended).
• Controlled by TimeBetweenGlobalCheckpoints
• Similar to innodb-flush-log-at-trx_commit=2
– The data is also checkpointed to disk.
• More on this later.
• The disk logging makes it possible to recovery a
completely failed cluster.
Copyright 2011 Severalnines AB Control your database infrastructure 18

19. Replicas
• To provide for redundancy and fast failover partitions
are synchronouslyreplicated.
• Two replicas (two copies of data) is most common
– it is possible to have one (previous slide), two, three and four
replicas.
– NoOfReplicas=2
STORAGE LAYER
subid data
DATA DATA
NODE 1 NODE 2 1 A Partition 0
3 B
2 C Partition 1
P0 P1
4 D
S1 S0
Px == PRIMARY Partition x
Sx == SECONDARY Partition x
Copyright 2011 Severalnines AB Control your database infrastructure 19

20. Node groups
• Data Nodes sharing the same data data belongs to a
node group.
• A node group contains as many nodes as you have
replicas (NoOfReplicas=2 is used in the picture
below).
STORAGE LAYER
subid data
DATA DATA
NODE 1 NODE 2 1 A Partition 0
3 B
2 C Partition 1
P0 P1
4 D
S1 S0
Node group 0 Px == PRIMARY Partition x
Sx == SECONDARY Parttionx
Copyright 2011 Severalnines AB Control your database infrastructure 20

21. Two Replicas – Four Data Nodes
• Four data nodes – four partitions – two replicas
• Four data nodes and two replicas  two node groups
– No of node groups = #data nodes / #replicas
STORAGE LAYER subid data
1 A
Partition 0
2 B
DATA DATA DATA DATA
NODE 1 NODE 2 NODE 3 NODE 4 3 C
Partition 1
4 D
P0 P1 P2 P3 5 E
Partition 2
6 F
S1 S0 S3 S2
7 G
Partition 3
Node group 0 Node group 1 8 H
Px == PRIMARY Partition x
Sx == SECONDARY Parttionx
Copyright 2011 Severalnines AB Control your database infrastructure 21

22. Three Replicas – Three Data Nodes
• Three data nodes – Three partitions
– More replicas gives slower writes.
• Three data nodes and three replicas  one node
group. This is not common.
– Two replicas is common practice/best practice
STORAGE LAYER
DATA DATA DATA subid data
NODE 1 NODE 2 NODE 3
1 A Partition 0
2 B
P0 P1 P2 3 C Partition 1
S1 S0 S0 4 D
S2 S2 S1 5 E Partition
6 F 2
Node group 0
Px == PRIMARY Partition x
Sx == SECONDARY
Parttionx
Copyright 2011 Severalnines AB Control your database infrastructure 22

23. Recommendations
• Two Replicas recommended - gives best tradeoff
between Performance and Availability
• Three Replicas (or Four) will give slower writes (, is
less tested (not so many deployments), and less
deployments are made with this.
• Cost of write (approx)
– 1 replica (no redundancy): cost X
– 2 replicas: cost 2X
– 3 replicas: cost 3X
Copyright 2011 Severalnines AB Control your database infrastructure 23

24. Verifying Data Distribution
• A helper program called ndb_desc shows how many
records are in each partition:
ndb_desc –c<mgmt_host:1186> –d<database>
-p<tablename>
Copyright 2011 Severalnines AB Control your database infrastructure 24

25. Two Phase Commit Protocol
Copyright 2011 Severalnines AB Control your database infrastructure 25

26. View of the Data Node
• Each node has as active Transaction Coordinator
(TC).
– Every Transaction starts at the TC
– One transaction consists of one or more Operations.
• Operation is SELECT, INSERT, UPDATE, DELETE
• Each node has 1-4 Local Query Handlers (LQH)
– Executes the operations
TC
• ACCess stores a hash table
LQH
– Primary Key and Unique
• TUPle stores data records ACC TUP
– ACC hash index links to record in TUP
Copyright 2011 Severalnines AB Control your database infrastructure 26

27. Two Phase Commit Protocol
• 2PC facilitates the synchronous replication
– To safeguard that both Primary and Secondary Partitions are
modified in a Consistent Way
– Locking (Row level)
• 2PC consists of two phases
– Prepare Phase
• Grab locks
• Make a update to a shadow copy
– Commit Phase
• Make the shadow copy the real copy
• Release locks
• 2PC protocol is only involved in
– Update, Delete, Insert
• Reads are not 2PC (would be very slow and
pointless)
Copyright 2011 Severalnines AB Control your database infrastructure 27

28. Two Phase Commit Protocol
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
– Two data nodes - PK hashes to Primary P0.
– The same applies to UPDATES and DELETEs
TC TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 28

29. Phase 1 - Prepare Phase
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
• The protocol starts at the TC that has the Primary
Partition for the PK
TC TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 29

30. Phase 1 - Prepare Phase
• TC starts by calculating a hash (md5sum) on the
Entire Primary key.
– md5sum(PK)
– This gives 128 bit hash value, divided up in 2 x 64-bit
INDEX_HASH PARTITION_HASH
– Actually further calculations are made
• PARTITION= PARTITION_HASH % NO_OF_PARTITIONS
• INDEX_POS=INDEX_HASH % NO_OF_BUCKETS
– LH*3 Algorithm is what is really being used.
• Now MySQL Cluster will know what nodes and
partitions are involved.
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
– INDEX_POS=23
– PARTITION=P0
Copyright 2011 Severalnines AB Control your database infrastructure 30

31. Phase 1 - Prepare Phase
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
• Prepare on the local LQH
TC TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 31

32. Phase 1 - Prepare Phase
• Insert into the hash table in ACC and write record in
TUP (allocate space for the record).
– The Index Entry is locked now
TC TC
PREPARE
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello
POS=23
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 32

33. Phase 1 - Prepare Phase
• Send PREPARE message to the other LQH for the
secondary Partition (S0), and do exactly the same
thing.
TC TC
PREPARE
PREPARE
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 33

34. Phase 1 - Prepare Phase
• Send back PREPARE OK to TC
TC PREPARE OK TC
PREPARE
PREPARE
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 34

35. Phase 2 - Commit Phase
• Send COMMIT message to REMOTE LQH first
– Release locks, simplifies error handling, ops waiting on lock
can resume faster.
TC COMMIT TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 35

36. Phase 2 - Commit Phase
• Send COMMIT to LQH of P0
– Release locks
TC COMMIT TC
COMMIT
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 36

37. Phase 2 - Commit Phase
• Send COMMIT OK to TC
TC COMMIT TC
COMMIT OK
COMMIT
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 37

38. Phase 2 - Commit Phase
• Send COMMIT ACK to application
COMMIT ACK
TC COMMIT TC
COMMIT OK
COMMIT
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 38

39. Failure Handling
• If the Transaction Coordinator fails another TC will
take over.
• The new TC will ask the nodes “Send me a list of
outstanding transactions from the failed TC”
• The new TC will
– ABORT transactions in the Prepare Phase
– Force COMMIT of transactions IFF another node has seen a
COMMIT message.
• The Node that has failed will get the changes when it recovers.
Copyright 2011 Severalnines AB Control your database infrastructure 39

40. Transaction Timers
• Deadlock detection – waiting for lock
– If a transaction is waiting too long time for a lock it is aborted
• TransactionDeadLockDetectionTimeout=3000 [ms]
• Inactive Transactions – holding a lock
– If a transaction is holding a lock too long it is aborted
• TransactionInactiveTimeout=60000 [ms]
– Never set TransactioInactiveTimeout=0
Copyright 2011 Severalnines AB Control your database infrastructure 40

41. Locking
• Committed Read Isolation level is used
• This means that a transaction
– Will read last committed data unless it has itself modified this
data.
• Locks are acquired in the Prepare Phase and
released in the Commit Phase.
Copyright 2011 Severalnines AB Control your database infrastructure 41

42. Transaction Resources
Copyright 2011 Severalnines AB Control your database infrastructure 42

43. Transaction and Operation Records
• Each transaction requires on Transaction Record
(TR)
– When it was started
– Controlled by MaxNoOfConcurrentTransactions
• Also each Operation requires one Operation Record
(OR) in TC and one Local Operation Record (LOR)
in LQH
– What transaction it belongs to
– Operation Type
– Controlled by MaxNoOfConcurrentOperations
– and MaxNoOfLocalOperations
Copyright 2011 Severalnines AB Control your database infrastructure 43

44. Transaction and Operation Records
• Assume
– MaxNoOfConcurrentTransactions=3
– MaxNoOfConcurrentOperations=3
– MaxNoOfLocalOperations=4
• By default there are 10% more Local Ops than Conc Ops.
• Query
– INSERT INTO t1 (id,data) VALUES(1,‟x‟);
• One transaction
• One operation
Copyright 2011 Severalnines AB Control your database infrastructure 44

45. Transaction Starts (prepare phase)
• INSERT INTO t1 (id,data) VALUES(1,‟x‟);
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 45

46. Transaction Starts (prepare phase)
• Allocate a Transaction Record at TC
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 46

47. Transaction Starts (prepare phase)
• Allocate a Operation Record at TC
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 47

48. Transaction Starts (prepare phase)
• Allocate a Local Operation Record at LQH
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 48

49. Transaction Starts (prepare phase)
• With many parallel transactions
TR TR
TC TC
OR OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 49

50. Transaction Starts (prepare phase)
• Wrongly dimensioned for failover
• Not enough TRs / ORs
TR TR
TC TC
OR OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 50

51. Transaction and Operation Records
• Define
– MaxNoOfConcurrentTransactions= 2 x<max_threads>
– MaxNoOfConcurrentOperations=
MaxConcurrentTransactionsx<ops in trans>
– MaxNoOfLocalOperations=1.1 x
MaxNoOfConcurrentOperations
• 1.1x is by default
• Seldom needed to change this.
Copyright 2011 Severalnines AB Control your database infrastructure 51

52. Coming next in Part 3:
Detailed Concepts
Access Methods
Redo Logging and Check-pointing
Recovery
Node Failures and Error Detection
Network Partitioning
Copyright 2011 Severalnines AB Control your database infrastructure 52

53. We hope these training slides are
useful to you!
Please visit our website to view the
next section of this training.
For any questions, comments, feedback or to
book a training class, please contact us at:
services@severalnines.com
Thank you!
Copyright 2011 Severalnines AB Control your database infrastructure 53

54. Disclaimer
© Copyright 2011 Severalnines AB. All rights reserved.
Severalnines& the Severalnineslogo(s) are trademarks of Severalnines AB.
MySQL is a registered trademark of Oracle and/or its affiliates.
Other names may be trademarks of their respective owners.
Copyright 2011 Severalnines AB Control your database infrastructure 54