MySQL Cluster 7.2 e 7.3 - por Andrew Morgan

MySQL Cluster Introduction
& Update
Andrew Morgan
MySQL High Availability – Product Management
@andrewmorgan
www.clusterdb.com

17th October 2012

Safe Harbour Statement

The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated into any
contract.
It is not a commitment to deliver any material, code, or functionality, and
should not be relied upon in making purchasing decisions. The development,
release, and timing of any features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.

2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. | 17t h October 2012 | Oracle reserves the right to change the timing and content of any future release

Table of Contents
 MySQL Cluster – Introduction

 Customer case studies

 Architecture

 MySQL Cluster 7.2 (GA)

 MySQL Cluster 7.3 (DMR & labs)

 Commercial additions

 The future


MySQL Cluster introduction
Carrier Grade • Shared-nothing in-memory parallel database server
Database • ACID compliant relational database

• Five nines (99.999%) availability
Highly Available • Self-healing, sub-second failover

Real-time • High load, real-time performance
Performance • Predictable low latency, bounded access times

• Incrementally grow out with application demands
Linearly Scalable • Linearly scale with distribution awareness

• Open Source, multiple data access
Open Development • High performance APIs (C++, Java, SQL, LDAP)


NoSQL – Why the demand?

2.1BN USERS
 Web applications 8X DATA GROWTH IN 5 YRS 900M USERS
demanding: 40% DATA
70+ NEW DOMAINS EVERY 60 SECONDS 20M APPS PER DAY

 Development velocity: GROWTH PER
YEAR 60 HOURS
Simplicity & flexibility of UPLOADED
EVERY MINUTE
data model & APIs
 Scalability & $1TR BY 2014
350m TWEETS
performance: high write PER DAY $700BN IN 2011

throughput and
Key/Value access
 Support for “Big Data” 5.9BN MOBILE SUBS IN 2011
1 BILLION iOS & ANDROID 85% HANDSETS
SHIPPED WITH A
APPS DOWNLOADED PER WEEK
BROWSER


What NoSQL must deliver

 Massive scalability
 No application-level sharding
 Performance
Scalability
 High Availability/Fault Tolerance
Performance
 Ease of use
HA
 Simple operations/administration
Ease of use
 Simple APIs
 Quickly evolve application &
schema


Still a role for the RDBMS?

 No best single solution
fits all NoSQL RDBMS
 Mix and match
Simple access patterns Complex queries with joins
Scalability
Compromise on consistency
Performance ACID transactions
for performance

HA
Ad-hoc data format Well defined schemas
Ease of use

SQL/Joins Simple operation Rich set of tools

ACID Transactions


Scalability a
Performance a
HA a
Ease of use a
SQL/Joins a
ACID Transactions a

Who’s Using MySQL Cluster?


CUSTOMER PERSPECTIVE

COMPANY OVERVIEW “MySQL Cluster won the performance test
 Leading provider of communications hands-down, and it fitted our needs perfectly.
platforms, solutions & services We evaluated shared-disk clustered
 €15.2bn Revenues (2009), 77k employees databases, but the cost would have been at
across 130 countries least 10x more.”
-- François Leygues, Systems Manager
CHALLENGES / OPPORTUNITIES
 Converged services driving migration to next
generation HLR / HSS systems Results
 New IMS platforms for Unified
 Scale out on standard ATCA hardware to
support 60m+ subscribers on a single
Communications platform
 Reduce cost per subscriber and accelerate  Low latency, high throughput with 99.999%+
time to value availability
 Enabled customers to reduce cost per
SOLUTION subscriber and improve margins
 MySQL Cluster Carrier Grade Edition
 Delivered data management solution at 10x
 MySQL Support & Consulting Services
less cost than alternatives

http://www.mysql.com/why-mysql/case-studies/mysql-alcatel-casestudy.php
11 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. | 17t h October 2012 | Oracle reserves the right to change the timing and content of any future release 11

PLAYFUL PLAY OVERVIEW
CUSTOMER PERSPECTIVE
 Developer of Latin America’s most popular FaceBook game "The MySQL support service has been essential in
 Based on El Chavo, massive success in LatAM, US and Spain helping us for troubleshooting and giving
recommendations for the production cluster, Thanks"
CHALLENGES / OPPORTUNITIES Carlos Morales – DBA, Playfulplay.com México
 2m users in 4 months, growing at 30k per day
 Targeting 50m users in 5 years SOLUTIONS
• MySQL Cluster Carrier Grade Edition
 Massive scale, especially of database writes
• MySQL Cluster Manager
 99.999% uptime
• MySQL Support & Consulting Services
 Data integrity
DATABASE REQUIREMENTS
RESULTS
 10k TPS on commodity hardware, in the cloud
• 45% improvement in performance
 Managing user avatars and sessions
• 80% reduction in DBA overhead
 In-App Purchases • 99.999% uptime
 Digital marketing + user response data
https://blogs.oracle.com/MySQL/entry/mysql_cluster_powers_el_chavo

Playful Play Architecture

MySQL Cluster

Co-Located MySQL Servers + Data Nodes

https://apps.facebook.com/lavecindaddeelchavo/

USER PERSPECTIVE
“MySQL Cluster exceeds our requirements for low
latency, high throughput performance with
COMPANY OVERVIEW continuous availability, in a single solution that
 Division of Docudesk minimizes complexity and overall cost.”
 Deliver Document Management SaaS
-- Casey Brown, Manager of Dev & DBA Services,
CHALLENGES / OPPORTUNITIES Docudesk
 Provide a single repository for customers to
manage, archive, and distribute documents Results
 Implement scalable, fault tolerant, real time data  Successfully deployed document
management back-end management solution, eliminating paper
 PHP session state cached for in-service trails from legal processes
personalization  Integrate caching and database into one
 Store document meta-data, text (as BLOBs), layer, reducing complexity & cost
ACL, job queues and billing data  Support workload with 50:50 read/write
ratio
 Data volumes growing at 2% per day  Low latency for real-time user experience
and document time-stamping
SOLUTION
 Continuous database availability
 MySQL Cluster deployed on EC2
http://www.mysql.com/customers/view/?id=1160
14 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. | 17t h October 2012 | Oracle reserves the right to change the timing and content of any future release 14

MySQL Cluster Architecture

Clients

Application Layer

Data Layer

Management

MySQL Cluster Data Nodes



Clients

Application Layer

Data Layer

Management
Management



On-line Scheduled Maintenance

Scale Backup

Evolve Upgrade


Automatic Data Partitioning
4 Partitions * 2 Replicas = 8 Fragments
Table T1 Data Node 1

Px Partition
P1
Data Node 2
P2

P3 Data Node 3

P4

Data Node 4
A fragment is a copy of a partition
Number of fragments = # of partitions * # of replicas


F1
Px Partition
P1
Data Node 2
P2

P3 Data Node 3

P4

Fx Primary Fragment Data Node 4

Fx Secondary Fragment


F1
Px Partition
P1
Data Node 2
P2 F1

P3 Data Node 3

P4




F1
Px Partition
P1
Data Node 2
P2 F3 F1

P3 Data Node 3

P4




F1 F3
Px Partition
P1
Data Node 2
P2 F3 F1

P3 Data Node 3

P4




F1 F3
Px Partition
P1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2
P4




F1 F3
Px Partition
P1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2
P4

F2


F1 F3
Px Partition
P1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2
P4

F4 F2


F1 F3
Px Partition
P1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2 F4
P4

F4 F2


F1 F3
Px Partition
P1 Node Group 1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2 F4
P4

F4 F2


F1 F3
Px Partition
P1 Node Group 1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2 F4
P4
Node Group 2
- Node groups are created automatically
F4 F2
- # of groups = # of data nodes / # of
replicas


F1 F3
Px Partition
P1 Node Group 1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2 F4
P4
Node Group 2
As long as one data node in each F4 F2
node group Fx Secondary Fragment
is running we have a complete copy of
the data


F1 F3
Px Partition
P1 Node Group 1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2 F4
P4
Node Group 2
the data


F1 F3
Px Partition
P1 Node Group 1
Data Node 2
P2 F3 F1

P3 Data Node 3
F2 F4
P4
Node Group 2
- No complete copy of the Fx Primary Fragment Data Node 4
data F4 F2
- Cluster shutdowns
automatically


writes


MySQL Cluster 7.2 GA
Enabling Next Generation Web Services

• 70x Higher Complex Query Performance: Adaptive Query Localization
• Native Memcached API
• MySQL 5.5 Server Integration
• Multi-Threaded Data Node Extensions
• VM Support

Enhancing Cross Data Center Scalability

• Multi-Site Clustering
• Simplified Active / Active Replication

Ease of Use

• MySQL Cluster Manager 1.1
• Consolidated Privileges


Comparing MySQL Cluster Performance
8x Higher Performance per Node
20

• 1 Billion+ Reads per 18

Minute, 8 node Intel

Reads per Second (Millions)
16

Xeon cluster 14

12

• Multi-Threaded Data 10
Node Extensions 8

6
• NoSQL C++ API, 4
flexaSynch benchmark 2

0
MySQL Cluster 7.1 MySQL Cluster 7.2


1.2 Billion UPDATEs per Minute
25

Millions of UPDATEs per Second
20
• 30 x Intel E5-2600 Intel
Servers
15

• NoSQL C++ API,
flexaSynch benchmark 10

• ACID Transactions, with 5
Synchronous
Replication 0
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30


Adaptive Query Localization 70x
Scaling Distributed Joins More
Performance
• Perform Complex Queries
across Shards
• JOINs pushed down to
data nodes mysqld
• Executed in parallel Data Nodes
A
• Returns single result set
Q
to MySQL
L
• Opens Up New Use-Cases
• Real-time analytics
• Recommendations
engines mysqld
• Analyze click-streams Data Nodes


MySQL Cluster 7.2 AQL Test Query
Web-Based Content Management System


Web-Based CMS
70x
Query Execution Time Seconds More
100
Performance
90
80
87.23 seconds
70
60
50
40
30
20
10 1.26 seconds
0
MySQL Cluster 7.1 MySQL Cluster 7.2

Must Analyze tables for best results
mysql> ANALYZE TABLE <tab-name>;


NoSQL Access to MySQL Cluster data

Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps

JPA
ClusterJPA

PHP PERL Python Ruby JDBC ClusterJ JSON Apache Memcached

MySQL JNI Node.js mod-ndb ndb-eng

NDB API (C++)



MySQL Cluster 7.1: ClusterJ/JPA
• Domain Object Model Persistence API (ClusterJ):
• Java API
• High performance, low latency
• Feature rich
• JPA interface built upon this new Java layer:
• Java Persistence API compliant
• Implemented as an OpenJPA plugin
• Uses ClusterJ where possible, reverts to JDBC for
some operations
• Higher performance than JDBC
• More natural for most Java designers
• Easier Cluster adoption for web applications


MySQL Cluster 7.2: Memcached

• Memcached is a distributed memory based hash-
key/value store with no persistence to disk
• NoSQL, simple API, popular with developers
• MySQL Cluster already provides scalable, in-
memory performance with NoSQL (hashed) access
as well as persistence
• Provide the Memcached API but map to NDB
API calls
• Writes-in-place, so no need to invalidate cache
• Simplifies architecture as caching & database
integrated into 1 tier
• Access data from existing relational tables


Traditional Memcached Architecture
Two levels of hashing

httpd memcached

hash key
PHP/Perl memcached
to find data
Memcache
friends:12389 memcached
memcache key


Memcache Protocol Elements

• Key Up to 250 characters
• Expire Time A number in seconds, up to 30 days
• Flags 32-bit number for application use
• CAS ID 64-bit incrementing version number on value
• Value Value stored for key; traditionally up to 1 MB.


Memcache Protocol Commands (1)

• get key Fetch value for this key
• set key value Store this data
• add key value Store this data (strict insert)
• replace key value Store this data (strict update)
• cas key value cas-id Update with version number check


Memcache Protocol Commands (2)

• append key value Append text to current value
• prepend key value Insert text before current value
• incr key amount Add to current stored integer value
• decr key amount Subtract from current stored value
• delete key Delete a key and value
• flush_all Delete all stored values on the server
• stats stat-name Fetch server statistics


NDB & Memcache Architecture:
Memcache protocol + NDB storage

MySQL
Application Cluster
memcached Data Node

Memcache
NDB Engine MySQL
Client
Cluster
Data Node


Goals

• Access stored data directly from memcache client
• Memcached perspective:
• MySQL Cluster is a write-scalable, replicated data store
• with reliable in-memory storage,
• plus on-disk storage when data is too big for memory.
• MySQL Cluster perspective:
• memcache is a high performance API
• providing easy access to in-memory data,
• plus an extra layer of caching when data is on disk.


Memcached/MySQL Cluster latency
memcachetest -t 2 -M 7000 -c 25000


Cluster & Memcached – Schema-Free

key value
<town:maidenhead,SL6>
Application view
SQL view key value


Key Value
town:maidenhead SL6
generic table


Cluster & Memcached – Configured Schema

key value
Application view
SQL view prefix key value


Prefix Table Key-col Val-col policy town ... code ...
town: map.zip town code cluster maidenhead ... SL6 ...

Config tables map.zip


Multi-Site Clustering

 Split data nodes across data
centers
Node Group 1 – Synchronous replication
Data Node 1 Data Node 2
and auto-failover between
Synchronous sites
Replication
– Improved heartbeating to
Node Group 2 handle network partitions
Data Node 3 Data Node 4  Extends HA Options
 Active/Active with no need
for conflict handling


Active/Active Geographic Replication

 Replicating complete clusters
across data centers
– DR & data locality
– No passive resources
Geographic
 Simplified Active / Active
Replication
Replication
– Eliminates requirement for
application & schema changes
– Transaction-level rollback


Simplified Provisioning & Maintenance
User Privilege Consolidation


DMR1


EA


MySQL Cluster 7.3 DMR1: Foreign Keys
 Brings MySQL Cluster to a broader range of workloads
– Packaged apps, custom projects
 Adds powerful functionality while reducing complexity
– App logic & data model
 Enabled by default
 Enforced for SQL
& NoSQL APIs
 On-line add and drop


 Implementation goal: compatibility with InnoDB
– Easy migration of solutions already working with InnoDB
 Foreign keys are natively implemented in the storage
engine
 Created in SQL
 Enforced in SQL & NoSQL (C++, ClusterJ, memcached,
node.js)


Downloading MySQL Cluster 7.3 DMR
Development Release Tab

http://dev.mysql.com/downloads/cluster/

MySQL Cluster 7.3 EA: Node.js NoSQL API
 Native JavaScript access to MySQL Cluster
– End-to-End JavaScript: browser to the app and
Clients database
– Storing and retrieving JavaScript objects directly
in MySQL Cluster
– Eliminate SQL transformation
V8 JavaScript Engine  Implemented as a module for node.js
MySQL Cluster Node.js Module – Integrates full Cluster API library within the web
app
 Couple high performance, distributed apps, with
high performance distributed database


MySQL Cluster 7.3 EA: Auto-Installer

 Fast configuration Specify Auto-
Workload Discover
 Auto-discovery
 Workload optimized
 Repeatable best
practices
 For MySQL Cluster Define
Deploy Topology
7.2 + 7.3


Define Hosts


Auto-Discovery


Modify hosts


Define nodes


Refine config


Deploy


Downloading MySQL Cluster EA

http://labs.mysql.com/


MySQL Cluster Manager
Enhancing DevOps Agility, Reducing Downtime

Automated
Self-Healing HA Operations
Management

• Start / Stop node or • Node monitoring • Cluster-wide
whole cluster configuration
• Auto-recovery
• On-Line Scaling extended to SQL consistency
• On-Line + mgmt nodes • Persistent
Reconfiguration configurations
• On-Line Upgrades • HA Agents


How Does MySQL Cluster Manager Help?
Example: Initiating upgrade from MySQL Cluster 7.0 to 7.2

Before MySQL Cluster Manager With MySQL Cluster Manager
• 1 x preliminary check of cluster state upgrade cluster --package=7.1 mycluster;
• 8 x ssh commands per server Total: 1 Command -
• 8 x per-process stop commands
• 4 x scp of configuration files (2 x mgmd & 2 x Unattended Operation
mysqld)
• 8 x per-process start commands  Results
• 8 x checks for started and re-joined processes – Reduces the overhead and complexity
• 8 x process completion verifications of managing database clusters
• 1 x verify completion of the whole cluster. – Reduces the risk of downtime resulting
• Excludes manual editing of each configuration from administrator error
file. – Automates best practices in database
Total: 46 commands - cluster management
2.5 hours of attended operation


MySQL Cluster 7.2 e 7.3 - por Andrew Morgan

MySQL Cluster 7.2 e 7.3 - por Andrew Morgan

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (14)

Ähnlich wie MySQL Cluster 7.2 e 7.3 - por Andrew Morgan

Ähnlich wie MySQL Cluster 7.2 e 7.3 - por Andrew Morgan (20)

Mehr von MySQL Brasil

Mehr von MySQL Brasil (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

MySQL Cluster 7.2 e 7.3 - por Andrew Morgan