From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Severalnines Self-Training: MySQL® Cluster - Part V
1. MySQL Cluster Training
presented by severalnines.com
Address:
Contact: SeveralninesAB
Jean-Jérôme Schmidt c/o SICS, Box 1263
Email: services@severalnines.com Isafjordsgatan22
SE-164-29 Kista
Copyright 2011 Severalnines AB Control your database infrastructure 1
2. Introduction
• At Severalnines, we believe in sharing information
and knowledge; we all come from an open source
background
• We know a lot of things about MySQL Cluster and
think that MySQL Cluster is a great technology
• These free MySQL Cluster Training slides are a
contribution of ours to the knowledge and information
sharing that’s common practice in the open source
community
• If you have any questions on these slides or would
like to book an actual training class, please contact
us at: services@severalnines.com
Copyright 2011 Severalnines AB Control your database infrastructure 2
3. Training Slides - Concept
• Over the coming weeks we will be chronologically
releasing slides for the different sections of our
MySQL Cluster Training program on our website.
• The full agenda of the training with all of its modules
is outlined in the next slides so that you can see what
topics will be covered over the coming weeks.
• Particularly specialised topics such as Cluster/J or
NDB API are not fully covered in the slides. We
recommend our instructor-led training classes for
such topics.
• Please contact us for more details:
services@severalnines.com
Copyright 2011 Severalnines AB Control your database infrastructure 3
4. Full Training Agenda (1/4)
• MySQL Cluster Introduction
– MySQL eco system
– Scale up, scale out, and sharding
– MySQL Cluster Architecture
– Use cases
– Features
– Node types and Roles
• Detailed Concepts
– Data Distribution
– Verifying data distribution
– Access Methods
– Partitioning
– Node failures and failure detection
– Network Partitioning
– Transactions and Locking
– Consistency Model
– Redo logging and Checkpointing
• Internals
– NDB Design Internals
Copyright 2011 Severalnines AB Control your database infrastructure 4
5. Agenda (2/4)
• Installing MySQL Cluster
– Setting up MySQL Cluster
– Starting/stopping nodes
– Recovery and restarts
– Upgrading configuration
– Upgrading Cluster
• Performance Tuning (instructor-led only; contact us at services@severalnines.com)
– Differences compared to Innodb/MyISAM
– Designing efficient and fast applications
– Identifying bottlenecks
– Tweaking configuration (OS and MySQL Cluster)
– Query Tuning
– Schema Design
– Index Tuning
Copyright 2011 Severalnines AB Control your database infrastructure 5
6. Agenda (3/4)
• Management and Administration
– Backup and Restore
– Geographical Replication
– Online and offline operations
– Ndbinfo tables
– Reporting
– Single user mode
– Scaling Cluster
• Disk Data
– Use cases
– Limitations
– Best practice configuration
• Designing a Cluster
– Capacity Planning and Dimensioning
– Hardware recommendations
– Best practice Configuration
– Storage calculations
Copyright 2011 Severalnines AB Control your database infrastructure 6
7. Agenda (4/4)
• Resolving Issues
– Common problems
– Error logs and Tracefiles
– Recovery and Escalation procedures
• Connectivity Overview
– NDBAPI
– Cluster/J
– LDAP
• Severalnines Tools
– Monitoring and Management
– Benchmarking
– Sandboxes
– Configuration and capacity planning
• Conclusion
Copyright 2011 Severalnines AB Control your database infrastructure 7
8. Agenda: Lab Exercises
(only applicable to instructor-led training classes)
• Lab Exercises
– Installing and Loading data into MySQL Cluster
– Starting/stopping nodes, recovery
– Query tuning
– Backup and Restore
– Configuration Upgrade
• Would you like to try something particular?
– This is possible too, speak with your instructor
Copyright 2011 Severalnines AB Control your database infrastructure 8
9. Prerequisites
• Readers / Participants have understanding of SQL and basic database concepts.
• Laptops/PCs for hands-on exercises
• Linux: 1GB RAM
• Windows: 2GB RAM
• Approx. 20GB disk space and Virtualbox installed.
• Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads
• MySQL Cluster version 7.1 or later
Copyright 2011 Severalnines AB Control your database infrastructure 9
10. 5th Installment
Severalnines Cluster Self-Training
Part 3: Internals
Copyright 2011 Severalnines AB Control your database infrastructure 10
11. Topics covered in Installment 5
• NDB Design Internals
Copyright 2011 Severalnines AB Control your database infrastructure 11
13. Software Model
• Software model is inherited from Ericsson AXE
swtiches.
• The kernel is composed of several software
blocks
• A block owns a functionality
– Transaction handling, index handling etc.
• Blocks communicates with each other using
signals
– Signals can also be sent to blocks in other nodes
(distributed)
• No data sharing between blocks!
Copyright 2011 Severalnines AB Control your database infrastructure 13
14. Blocks
• Blocks are software modules with pre-allocated
data structures (dynamic memory allocation will
be added later)
• No pointers
• Logical indexes into data structures with run-time
checks of index out of bounds
• Each block is a separate C++ class with one
generic parent (SimulatedBlock)
• Specific entry methods for all incoming signals
• A Virtual Machine schedules the execution of
signals inside a block.
Copyright 2011 Severalnines AB Control your database infrastructure 14
15. Blocks Overview
• DBACC
– Access control, lock manager
• DBDICT
– Meta data management, table and index defintions
• DBDIH
– Data distribution management, data fragmentation and replicas, Local
Checkpoints (LCP) and Global Checkpoints (GCP), system restart
• DBLQH
– Local query handler, local transaction management, local data
operations
• DBTC
– Transcation coordinator, distributed transaction management, global
data operations
• DBTUP
– Tuple manager, manages physical storage of data (read, insert,
update, delete, and monitoring changes of tuples)
Copyright 2011 Severalnines AB Control your database infrastructure 15
16. Blocks Overview
• BACKUP
– On-line backup
– LCP
• CMVMI
– Configuration management, interacting with
management server, interaction between blocks and
virtual machine
• DBTUX
– Local management of ordered indexes
• DBUTIL
– Internal interface to transactions and data operations
• NDBCNTR
– Ndb Cluster manager, adaption to logical cluster
(QMGR), initialization and configuration of blocks
Copyright 2011 Severalnines AB Control your database infrastructure 16
17. Blocks Overview
• NDBCNTR
– Ndb Cluster manager, adaption to logical cluster (QMGR),
initialization and configuration of blocks
• NDBFS
– Abstraction layer on-top of local file system
• QMGR
– Cluster manager, handles logical cluster, cluster membership
• RESTORE
– Supports restoring on-line backup
• SUMA
– Subscription manager, data and meta-data event monitoring
• TRIX
– On-line unique index build
Copyright 2011 Severalnines AB Control your database infrastructure 17
18. Virtual Machine
• The Virtual Machine
– Job schedulation mechanism (job== process signal)
– Jobs represented by signal buffer (Job Buffer)
– Hides OS details from blocks
– Single threaded implementation with job switching as
concurrency model
• very inexpensive context switching
• requires that blocks cooperate and relinquish control
within a resonable time slot -> more effort for
programmer.
Copyright 2011 Severalnines AB Control your database infrastructure 18
19. Virtual Machine and Signals
• Signal types
– Local signal
• signal to another local block, to be executed when
associated job is scheduled by the Virtual Machine
Equivilent to function call. No scheduling
– Remote (distributed) signal
• signal to a block on another node, to be executed when
remote associated job is scheduled by the remote Virtual
Machine
– Delayed signal
• to be executed after a certain time, e.g. 10ms.
• Signals can be fragmented and be up to 12 GB (non
fragmented signals up to 100B)
Copyright 2011 Severalnines AB Control your database infrastructure 19
20. Sample Signal
void Backup::execBACKUP_REQ(Signal* signal)
{
jamEntry();
BackupReq * req = (BackupReq*)signal->getDataPtr();
const Uint32 senderData = req->senderData;
const BlockReferencesenderRef = signal->senderBlockRef();
const Uint32 dataLen32 = req->backupDataLen; // In 32 bit words
const Uint32 flags = signal->getLength() > 2 ? req->flags : 2;
if( dataLen32==0) {jam(); abort()}
if(getOwnNodeId() != getMasterNodeId()) {
jam();
sendBackupRef(senderRef, flags, signal, senderData, BackupRef::IAmNotMaster);
return;
}//if
Copyright 2011 Severalnines AB Control your database infrastructure 20
21. Signal Types
• There are a number of different signal types. Here
are som common.
– REQ request signals
– CONF confirmation signals (ack)
– REF refusal signals
– REP report signals
– ORD order signals (no reply)
Copyright 2011 Severalnines AB Control your database infrastructure 21
22. Virtual Machine and Signals
• select() on all communication interfaces (called transporters
in NDB)
• Receive Signals from all communication interfaces
• Check Timed Signals
• Execute Signals
• Send Signals in buffers beloning to communication
interfaces (transporters).
Copyright 2011 Severalnines AB Control your database infrastructure 22
23. Scheduler
• ThreadConfig.cpp
• FastScheduler.cpp
• Two levels of scheduling
– Scheduling between performing jobs and send/receive
on transporters (ThreadConfig.cpp)
– Scheduling between jobs of different priorites
(FastScheduler.cpp)
Copyright 2011 Severalnines AB Control your database infrastructure 23
24. Scheduler
• Signals have priorities:
– Priority A Signals executed first
– Priority B Signals executed then
– Priority C Signals executed then (not really used)
– Priority D, used to buffer Delayed Signals that are then
put into B job buffer
Copyright 2011 Severalnines AB Control your database infrastructure 24
25. Transporters
• Transporters facilitates the communication
between nodes.
– Point to point – each node has one transporter to any
other node.
– Hides the underlying communication media
– Different transporters can have different characteristics
(latency, bandwidth, etc.).
– Currently TCP/IP sockets, Shared Memory, and SCI
Copyright 2011 Severalnines AB Control your database infrastructure 25
26. Threads in NDB
• Data node consists of a number of threads
– Main thread handling the execution (Execution Thread)
• Transactions and operations are executed in a single
thread
• Please note that the Execution Thread is really a set of
threads in MySQL Cluster 7.x (when using the multi-
thredaded daemon, see next few slides).
– Watchdog thread
• Makes sure the main thread is not stuck somewhere
– Filesystem threads
• Handles async i/o such as writing Local and Global
checkpoints
Copyright 2011 Severalnines AB Control your database infrastructure 26
27. Real-time Extensions
• The Threads can be bound to CPU cores
– reduce context switching
• Bind Maintenance threads to one Core
– Filesystem threads, watchdog threads
– LockMaintThreadsToCPU=<cpuid>
• Bind Execution (main thread) to another core
– LockExecutionThreadToCPU=<cpuid>
• Use cat /proc/interrupts to find out which CPU to
avoid
– On some Oss CPU 0 is used for interrupt handling of
eth0
Copyright 2011 Severalnines AB Control your database infrastructure 27
28. Real-time Extensions
• SchedulerSpinTimer=200 (us)
– Perform select(t=0) (non-blocking) for 200us
– Only in MySQL Cluster 6.3 (or non-multithreaded data
node)
• SchedulerExecutionTimer=50 (us)
– Receive and execute more signals before sending
– Only in MySQL Cluster 6.3 (or non-multithreaded data
node)
• RealtimeScheduler=1
Copyright 2011 Severalnines AB Control your database infrastructure 28
29. Protocols
• MySQL Cluster is a small and fast democracy with
a president (master)
• Protocols in MySQL Cluster is based on
Consensus
– President initiates a request and sends to all Participants
• (REQ signal)
– President expects to get a CONF back from the
Participants
– If a participant sends a REF, then it will most likely be
exclued
Copyright 2011 Severalnines AB Control your database infrastructure 29
30. Protocols
• Central protocols
– Two phase commit protocol (2PC)
– Global checkpoint protocol (GCP)
– Local checkpoint protocol (LCP)
– Heartbeat protocol (HB)
Copyright 2011 Severalnines AB Control your database infrastructure 30
31. Multi-threaded Data Node
• From MySQL Cluster 7.0 the data node is multi-
threaded
– MaxNoOfExecutionThreads
• Set to 8 (max) for 8 core machines.
– The data node will then have
• 1 TC thread
• 4 LQH threads (workers)
• 1 CMVMI thread (communication)
Copyright 2011 Severalnines AB Control your database infrastructure 31
32. Multi-threaded Data Node
TC
TC thread
LQH LQH LQH LQH
Worker ACC TUP ACC TUP ACC TUP ACC TUP
threads
P0 P0 P1 P1 P1 P2 P3 P3
Index Data Index Data Index Data Index Data
Memory Memory Memory Memory Memory Memory Memory Memory
P0 P1 P2 P3
REDO LOG
D8 D9 D10 D11
Copyright 2011 Severalnines AB Control your database infrastructure 32
33. Multi-threaded Data node
• Each worker has one or more partition
– Depends on the number of workers:
• 1 worker -> 4 partitions / worker
• 2 workers -> 2 partitions / worker
• 4 workers -> 1 partition / worker
• Each partition maps to one Redo log segment
• Communication between threads is efficient
– Uses the instruction set available in modern CPUs to have
“lock free” communication
• Each thread has its own scheduler
• Typically it is either the TC thread or the CMVMI
thread that gets overloaded, unless you use a lot of
scans/range scans then it is the Workers that
become the bottleneck first.
Copyright 2011 Severalnines AB Control your database infrastructure 33
34. Coming next in Section 6:
Part 4: Installing MySQL Cluster
Copyright 2011 Severalnines AB Control your database infrastructure 34
35. We hope these training slides are
useful to you!
Please visit our website to view the
next section of this training.
For any questions, comments, feedback or to
book a training class, please contact us at:
services@severalnines.com
Thank you!
Copyright 2011 Severalnines AB Control your database infrastructure 35
Dear Jury, Just as the electric grid revolutionized access to electricity 100 years ago, we at Severalnines believe that Cloud Computing will revolutionize IT where organizations will be able to plug into extremely powerful computing resources over the network. We have already seen the beginnings of this new wave, where the current infrastructure stack is being challenged and disrupted by a whole set of new technologies. For instance, in the database market, over 40 startups have received funding over the past 18 months. Severalnines is not building yet another database product, we believe there are already a lot of good technologies available. To manage a database costs 4 times the purchase price, and yet, very few companies are addressing this problem. Severalnines focuses on solutions to address this underserved segment. The founders of the company have a solid background in databases, having been at MySQL since 2003. The company develops a management platform which is database and cloud agnostic.We are database independent since we do not know who, if anybody, will be the next MySQL of the cloud.We are cloud independent, since we do not want to depend on any cloud vendor (e.g. Amazon or Rackspace) to avoid vendor lock-in. After the Amazon EC2 downtime during the Easter break, hundreds of affected companies have realized the importance of this. There is also a commercial aspect for avoiding vendor lock-in. Severalnines enhances productivity of organizations by attacking the biggest cost associated with database systems. We are a Swedish startup, hosted by SICS in Kista. There is also a small but very efficient development capacity in Singapore. Just as MySQL became a major brand and placed Sweden on the global software infrastructure map, we believe Severalnines can become a serious global player in the emerging Cloud space. We are very thankful that an organization like Eurocloud exists, and would like to thank the jury for considering our application. Kind regards,Vinay Joosery Severalnines AB
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com