SlideShare ist ein Scribd-Unternehmen logo
1 von 67
Ulf Wendel, Oracle




    MySQL? Load?
Clustering! Balancing!
      PECL/mysqlnd_ms 1.4
The speaker says...
Clustering databases is a main stream technology.
MySQL users can choose between a variety of clustering
options. The opions range from asynchronous primary copy
to synchronous update anywhere.


Let's quickly recap what MySQL has to offer.
MySQL Replication                              New:
                                        Global Transaction
                                            Identifier
Asynchronous read scale-out
  Master: writes, Slaves: reads
  Various topologies supported
  Other use cases: backup, HA, OLTP/warehousing

                 Client

   Writes                      Reads

   Master          Slave        Slave         Slave
The speaker says...
MySQL Replication is used for read scale-out. Writes
are send to a MySQL master, reads are send to a MySQL
slave. The master sends updates to the slaves in an
asynchronous way. Slaves may not have the latest
changes. This kind of replication is also called primary copy.
Where goes the update: to the primary. When/how do
updates happen: lazy, asynchronous.


MySQL Replication can also be used for:
Backup – perfom blocking backup on slave
High Availablility – for example, remote slaves
Warehousing – OLTP reporting on the slaves
MySQL Cluster                                  1.2 Billion
                                              UPDATEs/min
Real-time transactional write scale-out
  99,999% availability
  Auto sharding, shared nothing architecture
  Web use cases: session server, tele communication
                            Client

                 SQL Node             SQL Node



  Cluster Data Node   Cluster Data Node   Cluster Data Node
The speaker says...
MySQL Cluster is used for real-time transactional
read and write scale-out. Data is auto-partitioned on
cluster data nodes resulting in 99.999% availability. The
cluster is accessed through MySQL Application Cluster
Nodes. The most basic MySQL Application Cluster Node is a
SQL Node – a MySQL Server (mysqld).


If MySQL Cluster is new to you, think of MySQL Cluster as a
storage engine for the network. Like InnoDB and MyISAM
are storage engines for disks. This kind of replication is also
called eager (when?) update anywhere (where?).
Application using any DB cluster
Tasks:
  Choose servers to execute statement
  Load balance within candidates
  Maintain connection pool
  Automatic (client) fail over for High Availability

                          Client



 MySQL Server 1      MySQL Server 2       MySQL Server n
The speaker says...
All PHP applications talking to a cluster of MySQL
database servers are facing the same tasks.


Replacing a single database with a database cluster means
changing the 1:1 relation between the application and the
database into a 1:n relation. 1:n puts an additional task on
the application: find the right n, find the right server to
talk to.
The plugin for all of you
    WordPress, Drupal, Symfony, Oxid, ZendFramework, ...


                     mysql, mysqli, PDO_MYSQL

                             mysqlnd

                     PECL/mysqlnd_ms plugin

Statement Redirection Balancing Failover Connection Pooling



    MySQL Server 1        MySQL Server 2        MySQL Server n
The speaker says...
PECL/mysqlnd_ms is a plugin for the MySQL native
driver for PHP (mysqlnd) library. The mysqlnd library is
part of PHP as of version 5.3. As of 5.4 mysqlnd is a default.


All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can
be compiled to use the mysqlnd library, thus all existing
PHP MySQL application are supported by the plugin.


From an applications point of view a mysqlnd plugin
can act as a transparent proxy. Depending on the use
case, no code changes are needed. PECL/mysqlnd_ms
can be a drop-in solution.
Wait - what's wrong with this?
Go for it! But don't miss our goodies...
class DBRepl {
  private static $master_list = array(...);
  private static $master = NULL;
  private static $slave_list = array(...);
  private static $slave = NULL;
  public static function WriteConn() {
    if (self::$master)
      return self::$master;
    /* pick and initialize ...*/
  }
  public static function ReadConn() {
    if (self::$slave)
      return self::$slave;
    /* pick and initialize ...*/
  }
}
The speaker says...
A client-side approach to the problem is promising, if
client applications can be changed. It is no drop-in
solution. The load balancer is part of the client
application. It scales by client and it fails by client.
There is no single point of failure as it is the case with a
proxy based approach. No additional latency occurs like with
a proxy. Load balancing can be adaptive for good
resource usage. DBMS node failures do not block
clients, fail over is possible.


PECL/mysqlnd_ms has all this, is semi-transparent
and offers many more features!
PECL/mysqlnd_ms 1.4
Anything missing?
  All PHP MySQL APIs, semi-transparent, drop-in
  Read write splitting, transaction aware
  Load balancing, fail over (optional: automatic)
  Service levels: consistency, slave lag limit, trottling
  Optional, automatic caching of read results
  User can control all automatic actions
  75+ pages documentation: hands on and reference
  Stable, more test code than C code
  Some 10 bug reports since 1.0
The speaker says...
No worry, it is not complicated. All the magic is beneath the
hood, in the C code of PECL/mysqlnd_ms. The user
interface is short and comprehensive.


This is all what PECL/mysqlnd_ms is about: make using
any kind of MySQL Cluster easier! Let the load balancer
do the work.


Provide a cluster-aware API. Clustering is mainstream. We
need new APIs.
Read/Write splitting
Default: mysqlnd_ms.disable_rw_split = 0
  read-only: statement begins with SELECT
  read-only: statement begins with slave SQL hint
  custom: callback picks slave for statement

                  Client

   Writes             Reads (SELECT, SQL hint)

   Master           Slave         Slave        Slave
The speaker says...
Read/Write splitting must be done for primary copy clusters,
like MySQL Replication. Clients must direct all writes to the
primary (master). PECL/mysqlnd_ms automatically
directs statements that begin with SELECT or a
certain SQL hint to one of the configured secondaries
(slaves).


An update anywhere cluster, such as MySQL Cluster, allows
writes to be performed on all nodes. No read write splitting
is needed. Thus, you can disable it on a global level using
the configuration directive mysqlnd_ms.disable_rw_split.
Redirection and transactions
 No switch allowed during a transaction
   API calls controlling transactions monitored (5.4+)
   SQL not checked, protocol does not annnounce state
   Solution: use API (5.4+) or SQL hints!

                       Client

UPDATE SET col=@my           BEGIN; SELECT @my=1;

    Master           Slave        Slave        Slave
The speaker says...
Statement redirection for load balancing bares a
pitfall: connection state. Session variables and
transactions are part of the state. A load balancer must not
switch servers in the middle of a transaction. When using
primary copy, all transactions shall go to the primary
(master) because it is unknown at the beginning of a
transaction whether it includes updates.


The MySQL Client Server protocol does not hint whether a
transaction is active. Load balancers must either parse SQL
to catch BEGIN, monitor API calls or be hinted by the
application. PECL/mysqlnd_ms monitors the API and
understands hints.
Transaction aware code
Use API call over SQL to control transactions
    For MySQL Replication, set: trx_stickiness = master
    Check master_on_write=1 or mysqlnd_ms_set_qos()
$mysqli = new mysqli("myapp", "username", "password", "database");

/* Disable autocommit, master used, no server switch allowed */
$mysqli->autocommit(FALSE);
/* ... */

if (!$mysqli->commit()) {
  die(sprintf("[%d] %sn", $mysqli->errno, $mysqli->error));
}

/* Transaction has ended, load balancing begins again */
$mysqli->autocommit(TRUE);
The speaker says...
Use the API calls to control transactions. Give the
database driver a chance to help you. Virtually any state
change that you perform through API calls is handled
by PECL/mysqlnd_ms. For example, if you change the
charset with mysqli_set_charset(), the plugin makes sure
that all opened connections use the same charset. We hide
complexity, make cluster use easier.


Consider using master_on_write=1. This will keep all
reads followed by the first write on the master. Handy, if
you use asynchronous cluster and eventual consistency is
not enough. Wait for quality-of-service, its getting
better!
Is this your code?
PECL/mysqlnd_ms does not look worse, does it?
$mysqli = DBRel::ReadConn();
/* ... */
$mysqli = DBRepl::WriteConn()
$mysqli->autocommit(FALSE);
/* ... */
if (!$mysqli->commit()) {
  die(sprintf("[%d] %sn", $mysqli->errno, $mysqli->error));
}
/* ... */
$mysqli->autocommit(TRUE);
$mysqli = DBRepl::ReadConn();
The speaker says...
The differences to the classic approach become obvious.
The plugin code has less „hints“, the abstraction is less often
informed of the need of a certain type of database.


The classic code is specifically written for a certain kind of
cluster. Once you migrate the database from a lazy primary
copy system (e.g. MySQL Replication) to an eager update
anywhere solution (e.g. MySQL Cluster) you have additional
calls in your code that are no more needed.
The plugin code needs nothing but a configuration
file change to switch from MySQL Replication to
MySQL Cluster.
Load Balancing
Built-in or user-defined through callback
  Round robin, Random, Random once
  1.3 – kind of adaptive by setting limits for slave lag
  1.4 – nodes can be assigned a weight/probability

                    Client

   Writes                Load Balancing of Reads

   Master             Slave         Slave         Slave
 Weight = 1        Weight = 2    Weight = 1     Weight = 1
The speaker says...
The best and default load balancing stategy is the
one with the lowest impact on the connection state:
random once. Minimize switches for the life-span of your
PHP script. Pick a random slave during startup and use it for
all reads. The life-span of PHP is the of a web request. It is
short. Two web requests will end up on two random slaves.


The upcoming 1.4 release will allow you to assign a weight
to a node. Nodes with a weight of two will get twice
as many requests as nodes with a weight of one. This
is not only great if your nodes have different
computing power but also to prefer local nodes over
remote ones by setting a higher weight.
Pooling: connection failover
Caution: connection state change
  Default: disabled = raise exception
  Optional: silently failover master
  1.4 – remember failed, try more slaves before master

                  Client

   Writes              Reads

   Master           Slave        Slave        Slave
The speaker says...
Connection state changes are what speaks against
automatic, silent fail over once a connection has
been established and fails. If your application is written
in a way that automatic reconnect upon failure is allowed,
you can enable it. Otherwise, handle the error, do not
close the connection but run the next query to
trigger a new load balancing attempt! Failed nodes can
be remembered for the duration of a request.
Automatic failover can either try to connect to the
master or try to use another slave before failing over
to the master. If using MySQL Cluster, the search logic
still applies with the only difference that only masters are to
be configured.
Pooling: lazy connections
Do not open connection before needed
  Delay open until statement execution
  Reduce number of connections, reduce load
  Great for legacy apps – even without load balancing

                  Client

   Writes                      Reads

   Master           Slave         Slave        Slave
The speaker says...
By default, to reduce the connection pool size of
every PHP web request, connections are not opened
before executing a statement. Imagine you have 50
concurrent web requests, you configured four slaves and,
you are using random once. If not using lazy connections,
the plugin opens 50 x 4 = 200 slave connections but uses at
max 50 of them (random once). Every connection occupies
resources – ever seen a PHP application opening
connections before using? There are many badly written
ones...


Many connection state changing API calls are buffered and
replayed when opening lazy to prevent pitfalls.
Quality of service
Don't bother about node selection – MS does!
  mysqlnd_ms_set_qos()
  Eventual, session and strong consistency
  Allow only certain slave lag, enable caching

                            Client

                     PECL/mysqlnd_ms


               Database cluster as a service

        Node         Node            Node        Node
The speaker says...
The basics (load balancing, failover) are solved. Abstraction
can start. We can start to see the cluster as a service. The
load balancer knows how to use the cluster to achieve a
certain quality of service. The application formulates its
service quality requirements, the load balancer takes
appropriate action. For example, state the consistency
level needed. Is reading stale data from a slave
allowed (eventual consistency) and if so, is there any
limit on freshness?


Having API calls to know the client demands is a
great help for load balancer and cluster vendors!
Eventual consistency
Nodes may or may not serve the latest copies
  Nodes may serve stale copies or not have copies at all
  Default consistency level of MySQL Replication



SET X = 1         MySQL Node



        MySQL Node          MySQL Node          GET X, X = 0



        GET X, NULL
The speaker says...
An eventual consistent node may or may not serve
the latest copy. In fact, there is no promise that a
particular copy is available from every node. Many
systems that default to eventual consistency reach strong
consistency over time. Eventually all nodes get
synchronized. This model is similar to that of a cache.


Eventual consistency is good enoug for browsing product
catalogs or other infrequently changing contents in areas
where stale data is acceptable. It is the default consistency
level with MySQL Replication. Why bother ?! Can't vendors
provide proper solutions?
Clustering databases is complex
There is no one-fits-all replication solution
  The dangers of replication and a solution (Jim Gray, Pat
  Helland, 1996): a ten-fold increase in nodes and traffic
  gives a thousand fold increase in deadlocks or
  reconciliations
  CAP theorem (Eric Brewer, 2000): consistency,
  availability and paritition tolerance can't be achieved
  together
  … vendors are forced to offer compromises
  … vendors are faced with a highly complex problem
  … you are forced to know about consistency :-(
The speaker says...
I would love to stop talking about consistency. But
replication has limits. Keep on pushing but know the
limits. Every synchronous, update anywhere solution has a
scalability limit. Partitioning (sharding) only shifts the limits
and creates other issues (rebuilding aggregates). Until its
solved...
Applications shall state their QoS/consistency needs:
To allow vendors to relax consistency for performance
reasons, but only if feasible, if allowed by the app
To allow load balancers to make an optimal node choice


= PECL/mysqlnd_ms 1.2+ build around GTIDs...
Setting QoS for consistency
Better than master_on_write = 1
$mysqli = new mysqli("myapp", "username", "password", "database");
/* read-write splitting: master used */
$mysqli->query("INSERT INTO orders(item) VALUES ('elePHPant')");
/* Request session consistency: read your writes */
mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_SESSION);
/* Plugin picks a node which has the changes, here: master */
$res = $mysqli->query("SELECT item FROM orders WHERE order_id = 1");
var_dump($res->fetch_assoc());
/* Back to eventual consistency: stale data allowed */
mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL);
/* Plugin picks any slave, stale data is allowed */
$res = $mysqli->query("SELECT item, price FROM specials");
The speaker says...
An eager update anywhere replication system such as
MySQL Cluster delivers strong consistency. Classic
users of MySQL Replication achieve different level of
consistency using the simple rules: eventual consistency
– any slave, session and strong consistency – master
only.
Using the master for any level beyond eventual
consistency has a negative impact on read scale out.
MySQL 5.6 Global Transactions Identifiers help to
achieve session consistency even when reading from
slaves – reliable read your writes has become
possible. PECL/mysqlnd_ms implements the node
selection logic.
Global transaction identifier
Combination of server id and sequence number
  Emulation: PECL/mysqlnd_ms 1.2+, MySQL Proxy
  Built-in: MySQL 5.6


                         MySQL Master

            Log 7, Pos 34, GTID M:1: UPDATE x=1
            Log 7, Pos 35, GTID M:2: UPDATE x=9


         MySQL Slave 1                  MySQL Slave 2

    … , GTID M:1: UPDATE x=1     … , GTID M:1: UPDATE x=1
    … , GTID M:2: UPDATE x=9
The speaker says...
A global transaction identifier is a cluster-wide
unique transaction identifier. MySQL 5.6 can generate it
automatically. MySQL Proxy and PECL/mysqlnd_ms 1.2
feature client-side emulations for use with any MySQL
version. SQL can be used to access the GTIDs.


GTIDs have been been created to make MySQL Replication
failover easier. However, they are useful for load balancing
as well in a primary copy system.
MySQL Replication: strong con.
With or without GTID: primary only
  Only the primary has all comitted transactions




 Client A                              Client B
                 MySQL Master
SET X = 1                            GET X, X = 1




        MySQL Slave         MySQL Slave
The speaker says...
Configuring PECL/mysqlnd_ms for use with a MySQL
Replication cluster and calling mysqlnd_ms_set_qos($conn,
MYSQLND_MS_QOS_CONSISTENCY_STRONG) instructs
PECL/mysqlnd_ms to use only the MySQL Replication
master server for requests.


In a lazy primary copy system there is only one node that is
guaranteed to have all comitted updates: the primary.
Please note, that its possible to achieve higher consistency
levels than eventual consistency in an lazy primary copy
system by appropriately choosing nodes.
MySQL Replication: session con.
  Use GTID to find „synchronous“ slave
     Check slave status using SQL
     Reduce read load on master

               SET X = 9         MySQL Master

                            ..., GTID M:1: UPDATE x=1
                            ..., GTID M:2: UPDATE x=9


GET X, X = 9        MySQL Slave 1               MySQL Slave 2

               … , GTID M:1: UPDATE x=1   … , GTID M:1: UPDATE x=1
               … , GTID M:2: UPDATE x=9
The speaker says...
Global transaction identifier help to find „up-to-date“
slaves that have already replicated the latest
updates of a client. Thus, session consistency can
now be achieved by reading from the master and
selected „up-to-date“ slaves. This works with the GTID
emulation of PECL/mysqlnd_ms 1.2+ and any MySQL
version. And, it works with MySQL 5.6 that has built-in
GTIDs.


PECL/mysqlnd_ms 1.4 can either wait for a slave to
catch up or search all slaves before considering the
master. The wait effectively means throttling slaves
to reduce slave lag!
MySQL Replication: eventual c.
With or without GTID: all slaves
  Optional QoS level: upper slave lag limit
  MySQL estimates slave lag!


               SET X = 9         MySQL Master



GET X, X = 8        MySQL Slave 1             MySQL Slave 2

                  Slave lag = 1 second   Slave lag = 7 seconds
The speaker says...
A MySQL Replication slave is eventual consistent – it
may or may not have the latest updates. There is no
need to filter nodes with regards to consistency.


Slaves can be filtered by replication lag:
mysqlnd_ms_set_qos($conn,
MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL,
MYSQLND_MS_QOS_OPTION_AGE, 5) filters out all
slaves that report an estimated lag of more than five
seconds.
Slave selection logic
Same logic whenever slaves are to be filtered
  applied for: session consistency + GTID
  applied for: eventual consistency + Lag
  Stage 1: send SQL to check status to all slaves
  Stage 2: fetch replies in order
  Stage 3: apply filter logic
  SQL is documented in the manual
The speaker says...
Whenever PECL/mysqlnd_ms has to check the slave status
for filtering slaves, the check is done in two stages. First, all
slaves are queried. Then, replies are fetched from the slaves
in order. Usually, many requests can be send out in stage
one before the servers reply.


The SQL details are documented at php.net/mysqlnd_ms.
QoS: Cache Integration
 Automatically cache data no older than n seconds
    Reduce slave load
    Caches: Memcache, process memory, …
    PHP 5.4, PECL/mysqlnd_qc 1.1


1st GET X, X = 8     MySQL Slave 1          MySQL Slave 2

                   Slave lag = 1 second   Slave lag = 7 seconds
                   Max_TTL = 10 – 1 = 1
                   TTL = MAX_AGE - 9      TTL = MAX_AGE - 7


2nd GET X, X = 8          Cache
The speaker says...
If the load balancer knows that eventual consistency
is good and a certain degree of stale data (age, slave
lag) is allowed, a slave access can be transparently
replaced with a cache access. This lowers the overall
slave load in the cluster and reduces latency.


Replacing a slave access with a cache access is
transparent from an application point of view.
PECL/mysqlnd_ms is using PECL/mysqlnd_qc (qc = query
cache) for the caching. PECL/mysqlnd_qc can also be used
as a standalone cache to store query results in Memcache,
process memory and many more caches.
Transparent caching

$mysqli = new mysqli("myapp", "username", "password", "database");


/* no caching */
$res = $mysqli->query("SELECT headline FROM latest_news");
var_dump($res->fetch_all(MYSQLI_ASSOC));


/* use cache, if slave lag allows */
mysqlnd_ms_set_qos($mysqli,
 MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL,
 MYSQLND_MS_QOS_OPTION_CACHE, 60));


$res = $mysqli->query("SELECT headline FROM old_news");
var_dump($res->fetch_all(MYSQLI_ASSOC));
The speaker says...
Complicated logic behind but easy to use – PHP should
look out for new cluster aware APIs.


Let's close with an example of a minimal configuration for
use with MySQL Replication.
Speaking code: configuration
/* php.ini */
mysqlnd_ms.enable=1
mysqlnd_ms.config_file=/path/to/mysqlnd_ms_plugin_json.conf


/* /path/to/mysqlnd_ms_plugin_json.conf contents */
{ "myapp": {
  "master": {
    "master_0": {
        "host": "localhost",
        "socket": "/tmp/mysql.sock"
  }},
  "slave": {
  "slave_0": {
       "host": "192.168.2.27",
       "port": "3306"
}}}}
Connect and use... - as ever
$mysqli = new mysqli("myapp", "username", "password", "database");
$pdo = new PDO('mysql:host=myapp;dbname=database',
   'username', 'password');
$mysql = mysql_connect("myapp", "username", "password");


/* Statements will be run on the master */
$mysqli->query("DROP TABLE IF EXISTS test");
$mysqli->query("CREATE TABLE test(id INT)");
$mysqli->query("INSERT INTO test(id) VALUES (1)");


/* Goes to the slave - don't let slave lag fool you! */
$res = $mysqli->query("SELECT id FROM test");
var_dump($res->fetch_assoc());


$mysqli->close();
The speaker says...
Imagine we would have failover that could be bound to
certain error codes. For example, assume the SELECT
returns „Error: 1051 SQLSTATE: 42S02
(ER_BAD_TABLE_ERROR), Message: Unknown table '%s'
„ because you forgot about possiblity of a lag between the
CREATE and on the master and the SELECT on the slave.
What if PECL/mysqlnd_ms would automatically fail over to
the master to retry the SELECT... Sure, the better solution is
to as for session consistency to avoid this issue at all.


Let us know what additional features you would like
to see in PECL/mysqlnd_ms. We are ready to code...
THE END


Contact: ulf.wendel@oracle.com, @Ulf_Wendel
Kvack?
 Bonus slides!
Load Balancing layers overview
Whole stack cluster
Simple – 1:1 mapping!
  Requires synchronous cluster
  No fault tolerance: node failure = stack failure
  Inefficient resource usage: no balancing


  HTTP Server          HTTP Server         HTTP Server

App Server (PHP)     App Server (PHP)    App Server (PHP)




  MySQL Node           MySQL Node          MySQL Node
The speaker says...
In a synchronous cluster, for example, if using MySQL
Cluster, all nodes have all the data. Thus, every application
server could be assigned to one DBMS node. Easy, fast
and good for MySQL Cluster but with limits. No good
for asynchronous MySQL Replication.

Limit: DBMS node failure includes application node
failure. Client should have additional fail over logic.

Limit: Over- and undercapacity of a DBMS node
cannot be compensated. Clients cannot switch to more
powerful nodes. Overcapacity of a MySQL node cannot be
used by other clients. Different hardware size is ignored.
Proxy approach - Pro
Free and open source MySQL Proxy
  No application changes, free 3rd party proxy scripts
  MySQL node failure does not stop application server
  Good resource usage, dynamic adoption possible


  HTTP Server         HTTP Server         HTTP Server

App Server (PHP)    App Server (PHP)    App Server (PHP)

                     MySQL Proxy


  MySQL Node          MySQL Node          MySQL Node
The speaker says...
A transparent MySQL Proxy based solution requires
no application changes. Clients connect to the Proxy
using MySQL Client Server Protocol, as if the MySQL Proxy
was a MySQL Server.


Proxy can compensate for DBMS failures. It can react
to temporary and permant outages.


Proxy can adapt load balancing dynamically. Taking
into account node hardware size, current node load, latency,
location, … fantasy sets the limits here for self-written or 3rd
party Proxy scripts.
Proxy approach - Con
Additional component
  Complex architecture, can be single point of failure
  C and Lua are no natural match for PHP lovers
  Adds latency: from client to proxy to node


  HTTP Server         HTTP Server          HTTP Server

App Server (PHP)    App Server (PHP)     App Server (PHP)

                     MySQL Proxy


  MySQL Node          MySQL Node          MySQL Node
The speaker says...
MySQL Proxy is a great performer!

But, … MySQL Proxy adds complexity to the stack.
MySQL Proxy needs to be managed. MySQL Proxy is build
around C and Lua. C and PHP would be a better match for
PHP users. Wrongly used, MySQL Proxy becomes a single
point of failure. It is single threaded. This bares the risk of
tasks (Proxy scripts) blocking each other.


But, … MySQL Proxy adds latency. Although, it can be
minimized significantly running MySQL Proxy on the App
Server to avoid use of TCP/IP between PHP and Proxy.
Client-side load balancer - Pro
Client application handles load balancing
  No single point of failure
  No additional latency
  Highly configurable and adaptive


  HTTP Server        HTTP Server        HTTP Server

App Server (PHP)   App Server (PHP)   App Server (PHP)

  <?php .. ?>         <?php ...?>        <?php ?>


  MySQL Node         MySQL Node         MySQL Node
The speaker says...
A client-side approach to the problem is promising, if
client applications can be changed. It has most Pro's of
the previous approaches.

The load balancer is part of the client application. It
scales by client and it fails by client. Scalability is given
and there is no single point of failure. No additional latency
occurs.


Load balancing can be adaptive for good resource
usage. DBMS node failures do not block clients, fail
over is possible.
Client-side load balancer - Con
Client application handles load balancing
  Application must be designed for (specific) cluster use
  No drop-in solution for existing applications
  PHP is stateless: adaptive load balancing is difficult


  HTTP Server         HTTP Server          HTTP Server

App Server (PHP)    App Server (PHP)     App Server (PHP)

  <?php .. ?>          <?php ...?>          <?php ?>


  MySQL Node          MySQL Node           MySQL Node
The speaker says...
The major downside of a client-side application
based solution is the need to change the application.
Application changes must be done even for basic cluster use
cases. Modifications to source code may not be possible.
They may complicate upgrades of standard software. They
may cause lots of work thus become expensive.


Load balancing is part of PHP thus stateless. This is both a
Pro and a Con. It is difficult to hint other clients about node
failures. On the other hand, shared nothing is not a bad idea
either.
Is driver based the solution?
Client-side and driver based: PECL/mysqlnd_ms
  Pro: all previous
  Synchronous cluster: no application changes
  Asynchronous cluster: no change for basic cases


   HTTP Server       HTTP Server         HTTP Server

App Server (PHP)   App Server (PHP)    App Server (PHP)

PECL/mysqlnd_ms    PECL/mysqlnd_ms    PECL/mysqlnd_ms


  MySQL Node         MySQL Node          MySQL Node
The speaker says...
A driver based client-side solution has all Pro's!

Considering load balancing aspect only, no application
changes are required. If using MySQL Cluster, a
synchronous cluster, no application changes are
required. If using MySQL Replication, an
asynchronous cluster that needs read-write splitting,
the need for application changes is significantly
reduced!

The PECL mysqlnd_ms installation can be part of the next
PHP deployment, done anyway for security considerations.
No new procedure. If any issues, simply disable extension.

Weitere ähnliche Inhalte

Was ist angesagt?

Award-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cacheAward-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cacheUlf Wendel
 
PHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing pluginPHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing pluginUlf Wendel
 
MySQL 5.7 Fabric: Introduction to High Availability and Sharding
MySQL 5.7 Fabric: Introduction to High Availability and Sharding MySQL 5.7 Fabric: Introduction to High Availability and Sharding
MySQL 5.7 Fabric: Introduction to High Availability and Sharding Ulf Wendel
 
MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011
MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011
MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011Ulf Wendel
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group ReplicationUlf Wendel
 
NoSQL in MySQL
NoSQL in MySQLNoSQL in MySQL
NoSQL in MySQLUlf Wendel
 
Highly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlndHighly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlndJervin Real
 
Built-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIsBuilt-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIsUlf Wendel
 
MySQL Multi Master Replication
MySQL Multi Master ReplicationMySQL Multi Master Replication
MySQL Multi Master ReplicationMoshe Kaplan
 
MySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveMySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveUlf Wendel
 
HTTP Plugin for MySQL!
HTTP Plugin for MySQL!HTTP Plugin for MySQL!
HTTP Plugin for MySQL!Ulf Wendel
 
Methods of Sharding MySQL
Methods of Sharding MySQLMethods of Sharding MySQL
Methods of Sharding MySQLLaine Campbell
 
Introduction to Galera
Introduction to GaleraIntroduction to Galera
Introduction to GaleraHenrik Ingo
 
Mysql high availability and scalability
Mysql high availability and scalabilityMysql high availability and scalability
Mysql high availability and scalabilityyin gong
 
High-Availability using MySQL Fabric
High-Availability using MySQL FabricHigh-Availability using MySQL Fabric
High-Availability using MySQL FabricMats Kindahl
 
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationPercona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationKenny Gryp
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group ReplicationBogdan Kecman
 
Galera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction SlidesGalera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction SlidesSeveralnines
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group ReplicationManish Kumar
 
Using and Benchmarking Galera in different architectures (PLUK 2012)
Using and Benchmarking Galera in different architectures (PLUK 2012)Using and Benchmarking Galera in different architectures (PLUK 2012)
Using and Benchmarking Galera in different architectures (PLUK 2012)Henrik Ingo
 

Was ist angesagt? (20)

Award-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cacheAward-winning technology: Oxid loves the query cache
Award-winning technology: Oxid loves the query cache
 
PHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing pluginPHP mysqlnd connection multiplexing plugin
PHP mysqlnd connection multiplexing plugin
 
MySQL 5.7 Fabric: Introduction to High Availability and Sharding
MySQL 5.7 Fabric: Introduction to High Availability and Sharding MySQL 5.7 Fabric: Introduction to High Availability and Sharding
MySQL 5.7 Fabric: Introduction to High Availability and Sharding
 
MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011
MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011
MySQL native driver for PHP (mysqlnd) - Introduction and overview, Edition 2011
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group Replication
 
NoSQL in MySQL
NoSQL in MySQLNoSQL in MySQL
NoSQL in MySQL
 
Highly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlndHighly Available MySQL/PHP Applications with mysqlnd
Highly Available MySQL/PHP Applications with mysqlnd
 
Built-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIsBuilt-in query caching for all PHP MySQL extensions/APIs
Built-in query caching for all PHP MySQL extensions/APIs
 
MySQL Multi Master Replication
MySQL Multi Master ReplicationMySQL Multi Master Replication
MySQL Multi Master Replication
 
MySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspectiveMySQL 5.7 clustering: The developer perspective
MySQL 5.7 clustering: The developer perspective
 
HTTP Plugin for MySQL!
HTTP Plugin for MySQL!HTTP Plugin for MySQL!
HTTP Plugin for MySQL!
 
Methods of Sharding MySQL
Methods of Sharding MySQLMethods of Sharding MySQL
Methods of Sharding MySQL
 
Introduction to Galera
Introduction to GaleraIntroduction to Galera
Introduction to Galera
 
Mysql high availability and scalability
Mysql high availability and scalabilityMysql high availability and scalability
Mysql high availability and scalability
 
High-Availability using MySQL Fabric
High-Availability using MySQL FabricHigh-Availability using MySQL Fabric
High-Availability using MySQL Fabric
 
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group ReplicationPercona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
Percona XtraDB Cluster vs Galera Cluster vs MySQL Group Replication
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group Replication
 
Galera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction SlidesGalera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction Slides
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group Replication
 
Using and Benchmarking Galera in different architectures (PLUK 2012)
Using and Benchmarking Galera in different architectures (PLUK 2012)Using and Benchmarking Galera in different architectures (PLUK 2012)
Using and Benchmarking Galera in different architectures (PLUK 2012)
 

Andere mochten auch

HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLUlf Wendel
 
MySQL High Availability with Group Replication
MySQL High Availability with Group ReplicationMySQL High Availability with Group Replication
MySQL High Availability with Group ReplicationNuno Carvalho
 
MySQL High Availability Solutions - Feb 2015 webinar
MySQL High Availability Solutions - Feb 2015 webinarMySQL High Availability Solutions - Feb 2015 webinar
MySQL High Availability Solutions - Feb 2015 webinarAndrew Morgan
 
Libmysqld Introduction
Libmysqld IntroductionLibmysqld Introduction
Libmysqld Introductionpapablues
 
99.999% Available OpenStack Cloud - A Builder's Guide
99.999% Available OpenStack Cloud - A Builder's Guide99.999% Available OpenStack Cloud - A Builder's Guide
99.999% Available OpenStack Cloud - A Builder's GuideDanny Al-Gaaf
 
High Availability with MySQL
High Availability with MySQLHigh Availability with MySQL
High Availability with MySQLThava Alagu
 
MySQL Fabric: High Availability using Python/Connector
MySQL Fabric: High Availability using Python/ConnectorMySQL Fabric: High Availability using Python/Connector
MySQL Fabric: High Availability using Python/ConnectorVishal Yadav
 
MySQL Proxy: Architecture and concepts of misuse
MySQL Proxy: Architecture and concepts of misuseMySQL Proxy: Architecture and concepts of misuse
MySQL Proxy: Architecture and concepts of misuseweigon
 
MySQL Proxy. From Architecture to Implementation
MySQL Proxy. From Architecture to ImplementationMySQL Proxy. From Architecture to Implementation
MySQL Proxy. From Architecture to ImplementationRonald Bradford
 
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!ScaleBase
 
MySQL HA Solutions
MySQL HA SolutionsMySQL HA Solutions
MySQL HA SolutionsMat Keep
 
MySQL Proxy. A powerful, flexible MySQL toolbox.
MySQL Proxy. A powerful, flexible MySQL toolbox.MySQL Proxy. A powerful, flexible MySQL toolbox.
MySQL Proxy. A powerful, flexible MySQL toolbox.Miguel Araújo
 
MySQL High Availability Deep Dive
MySQL High Availability Deep DiveMySQL High Availability Deep Dive
MySQL High Availability Deep Divehastexo
 
MySQL High Availability and Disaster Recovery with Continuent, a VMware company
MySQL High Availability and Disaster Recovery with Continuent, a VMware companyMySQL High Availability and Disaster Recovery with Continuent, a VMware company
MySQL High Availability and Disaster Recovery with Continuent, a VMware companyContinuent
 
MySQL Group Replication - an Overview
MySQL Group Replication - an OverviewMySQL Group Replication - an Overview
MySQL Group Replication - an OverviewMatt Lord
 
MySQL Replication: What’s New in MySQL 5.7 and Beyond
MySQL Replication: What’s New in MySQL 5.7 and BeyondMySQL Replication: What’s New in MySQL 5.7 and Beyond
MySQL Replication: What’s New in MySQL 5.7 and BeyondAndrew Morgan
 
Using MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling OutUsing MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling OutOSSCube
 
ProxySQL - High Performance and HA Proxy for MySQL
ProxySQL - High Performance and HA Proxy for MySQLProxySQL - High Performance and HA Proxy for MySQL
ProxySQL - High Performance and HA Proxy for MySQLRené Cannaò
 

Andere mochten auch (20)

HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
 
MySQL High Availability with Group Replication
MySQL High Availability with Group ReplicationMySQL High Availability with Group Replication
MySQL High Availability with Group Replication
 
MySQL High Availability Solutions - Feb 2015 webinar
MySQL High Availability Solutions - Feb 2015 webinarMySQL High Availability Solutions - Feb 2015 webinar
MySQL High Availability Solutions - Feb 2015 webinar
 
Libmysqld Introduction
Libmysqld IntroductionLibmysqld Introduction
Libmysqld Introduction
 
99.999% Available OpenStack Cloud - A Builder's Guide
99.999% Available OpenStack Cloud - A Builder's Guide99.999% Available OpenStack Cloud - A Builder's Guide
99.999% Available OpenStack Cloud - A Builder's Guide
 
High Availability with MySQL
High Availability with MySQLHigh Availability with MySQL
High Availability with MySQL
 
MySQL Fabric: High Availability using Python/Connector
MySQL Fabric: High Availability using Python/ConnectorMySQL Fabric: High Availability using Python/Connector
MySQL Fabric: High Availability using Python/Connector
 
MySQL Proxy: Architecture and concepts of misuse
MySQL Proxy: Architecture and concepts of misuseMySQL Proxy: Architecture and concepts of misuse
MySQL Proxy: Architecture and concepts of misuse
 
MySQL highav Availability
MySQL highav AvailabilityMySQL highav Availability
MySQL highav Availability
 
MySQL Proxy. From Architecture to Implementation
MySQL Proxy. From Architecture to ImplementationMySQL Proxy. From Architecture to Implementation
MySQL Proxy. From Architecture to Implementation
 
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
 
MySQL Proxy tutorial
MySQL Proxy tutorialMySQL Proxy tutorial
MySQL Proxy tutorial
 
MySQL HA Solutions
MySQL HA SolutionsMySQL HA Solutions
MySQL HA Solutions
 
MySQL Proxy. A powerful, flexible MySQL toolbox.
MySQL Proxy. A powerful, flexible MySQL toolbox.MySQL Proxy. A powerful, flexible MySQL toolbox.
MySQL Proxy. A powerful, flexible MySQL toolbox.
 
MySQL High Availability Deep Dive
MySQL High Availability Deep DiveMySQL High Availability Deep Dive
MySQL High Availability Deep Dive
 
MySQL High Availability and Disaster Recovery with Continuent, a VMware company
MySQL High Availability and Disaster Recovery with Continuent, a VMware companyMySQL High Availability and Disaster Recovery with Continuent, a VMware company
MySQL High Availability and Disaster Recovery with Continuent, a VMware company
 
MySQL Group Replication - an Overview
MySQL Group Replication - an OverviewMySQL Group Replication - an Overview
MySQL Group Replication - an Overview
 
MySQL Replication: What’s New in MySQL 5.7 and Beyond
MySQL Replication: What’s New in MySQL 5.7 and BeyondMySQL Replication: What’s New in MySQL 5.7 and Beyond
MySQL Replication: What’s New in MySQL 5.7 and Beyond
 
Using MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling OutUsing MySQL Fabric for High Availability and Scaling Out
Using MySQL Fabric for High Availability and Scaling Out
 
ProxySQL - High Performance and HA Proxy for MySQL
ProxySQL - High Performance and HA Proxy for MySQLProxySQL - High Performance and HA Proxy for MySQL
ProxySQL - High Performance and HA Proxy for MySQL
 

Ähnlich wie MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4

MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMakerKris Buytaert
 
MySQL HA Alternatives 2010
MySQL  HA  Alternatives 2010MySQL  HA  Alternatives 2010
MySQL HA Alternatives 2010Kris Buytaert
 
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesMySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesKenny Gryp
 
Master master vs master-slave database
Master master vs master-slave databaseMaster master vs master-slave database
Master master vs master-slave databaseWipro
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
 
MySQL 101 PHPTek 2017
MySQL 101 PHPTek 2017MySQL 101 PHPTek 2017
MySQL 101 PHPTek 2017Dave Stokes
 
RDS for MySQL, No BS Operations and Patterns
RDS for MySQL, No BS Operations and PatternsRDS for MySQL, No BS Operations and Patterns
RDS for MySQL, No BS Operations and PatternsLaine Campbell
 
Database Mirror for the exceptional DBA – David Izahk
Database Mirror for the exceptional DBA – David IzahkDatabase Mirror for the exceptional DBA – David Izahk
Database Mirror for the exceptional DBA – David Izahksqlserver.co.il
 
ConFoo MySQL Replication Evolution : From Simple to Group Replication
ConFoo  MySQL Replication Evolution : From Simple to Group ReplicationConFoo  MySQL Replication Evolution : From Simple to Group Replication
ConFoo MySQL Replication Evolution : From Simple to Group ReplicationDave Stokes
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlsqlhjalp
 
MySQL no Paypal Tesla e Uber
MySQL no Paypal Tesla e UberMySQL no Paypal Tesla e Uber
MySQL no Paypal Tesla e UberMySQL Brasil
 
The power of mysqlnd plugins
The power of mysqlnd pluginsThe power of mysqlnd plugins
The power of mysqlnd pluginsUlf Wendel
 
MySQL High Availability Solutions
MySQL High Availability SolutionsMySQL High Availability Solutions
MySQL High Availability SolutionsLenz Grimmer
 
Mysqlhacodebits20091203 1260184765-phpapp02
Mysqlhacodebits20091203 1260184765-phpapp02Mysqlhacodebits20091203 1260184765-phpapp02
Mysqlhacodebits20091203 1260184765-phpapp02Louis liu
 
MySQL High Availability Solutions
MySQL High Availability SolutionsMySQL High Availability Solutions
MySQL High Availability SolutionsLenz Grimmer
 
MySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoMySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoKeith Hollman
 

Ähnlich wie MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4 (20)

MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMaker
 
MySQL HA Alternatives 2010
MySQL  HA  Alternatives 2010MySQL  HA  Alternatives 2010
MySQL HA Alternatives 2010
 
Introduction to Galera Cluster
Introduction to Galera ClusterIntroduction to Galera Cluster
Introduction to Galera Cluster
 
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best PracticesMySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
MySQL InnoDB Cluster - New Features in 8.0 Releases - Best Practices
 
Master master vs master-slave database
Master master vs master-slave databaseMaster master vs master-slave database
Master master vs master-slave database
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
 
MySQL 101 PHPTek 2017
MySQL 101 PHPTek 2017MySQL 101 PHPTek 2017
MySQL 101 PHPTek 2017
 
RDS for MySQL, No BS Operations and Patterns
RDS for MySQL, No BS Operations and PatternsRDS for MySQL, No BS Operations and Patterns
RDS for MySQL, No BS Operations and Patterns
 
Mysql
MysqlMysql
Mysql
 
Database Mirror for the exceptional DBA – David Izahk
Database Mirror for the exceptional DBA – David IzahkDatabase Mirror for the exceptional DBA – David Izahk
Database Mirror for the exceptional DBA – David Izahk
 
ConFoo MySQL Replication Evolution : From Simple to Group Replication
ConFoo  MySQL Replication Evolution : From Simple to Group ReplicationConFoo  MySQL Replication Evolution : From Simple to Group Replication
ConFoo MySQL Replication Evolution : From Simple to Group Replication
 
Mysql ppt
Mysql pptMysql ppt
Mysql ppt
 
Mysql tutorial 5257
Mysql tutorial 5257Mysql tutorial 5257
Mysql tutorial 5257
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdl
 
MySQL no Paypal Tesla e Uber
MySQL no Paypal Tesla e UberMySQL no Paypal Tesla e Uber
MySQL no Paypal Tesla e Uber
 
The power of mysqlnd plugins
The power of mysqlnd pluginsThe power of mysqlnd plugins
The power of mysqlnd plugins
 
MySQL High Availability Solutions
MySQL High Availability SolutionsMySQL High Availability Solutions
MySQL High Availability Solutions
 
Mysqlhacodebits20091203 1260184765-phpapp02
Mysqlhacodebits20091203 1260184765-phpapp02Mysqlhacodebits20091203 1260184765-phpapp02
Mysqlhacodebits20091203 1260184765-phpapp02
 
MySQL High Availability Solutions
MySQL High Availability SolutionsMySQL High Availability Solutions
MySQL High Availability Solutions
 
MySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoMySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & Demo
 

Mehr von Ulf Wendel

Data massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodesData massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodesUlf Wendel
 
PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011Ulf Wendel
 
Mysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark reportMysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark reportUlf Wendel
 
mysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handlermysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handlerUlf Wendel
 
Mysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuningMysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuningUlf Wendel
 
Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008Ulf Wendel
 

Mehr von Ulf Wendel (6)

Data massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodesData massage: How databases have been scaled from one to one million nodes
Data massage: How databases have been scaled from one to one million nodes
 
PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011PHPopstar der PHP Unconference 2011
PHPopstar der PHP Unconference 2011
 
Mysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark reportMysqlnd query cache plugin benchmark report
Mysqlnd query cache plugin benchmark report
 
mysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handlermysqlnd query cache plugin: user-defined storage handler
mysqlnd query cache plugin: user-defined storage handler
 
Mysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuningMysqlnd query cache plugin statistics and tuning
Mysqlnd query cache plugin statistics and tuning
 
Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008Mysqlnd Async Ipc2008
Mysqlnd Async Ipc2008
 

Kürzlich hochgeladen

The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 

Kürzlich hochgeladen (20)

The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 

MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4

  • 1. Ulf Wendel, Oracle MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4
  • 2. The speaker says... Clustering databases is a main stream technology. MySQL users can choose between a variety of clustering options. The opions range from asynchronous primary copy to synchronous update anywhere. Let's quickly recap what MySQL has to offer.
  • 3. MySQL Replication New: Global Transaction Identifier Asynchronous read scale-out Master: writes, Slaves: reads Various topologies supported Other use cases: backup, HA, OLTP/warehousing Client Writes Reads Master Slave Slave Slave
  • 4. The speaker says... MySQL Replication is used for read scale-out. Writes are send to a MySQL master, reads are send to a MySQL slave. The master sends updates to the slaves in an asynchronous way. Slaves may not have the latest changes. This kind of replication is also called primary copy. Where goes the update: to the primary. When/how do updates happen: lazy, asynchronous. MySQL Replication can also be used for: Backup – perfom blocking backup on slave High Availablility – for example, remote slaves Warehousing – OLTP reporting on the slaves
  • 5. MySQL Cluster 1.2 Billion UPDATEs/min Real-time transactional write scale-out 99,999% availability Auto sharding, shared nothing architecture Web use cases: session server, tele communication Client SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node
  • 6. The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out. Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node – a MySQL Server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks. This kind of replication is also called eager (when?) update anywhere (where?).
  • 7. Application using any DB cluster Tasks: Choose servers to execute statement Load balance within candidates Maintain connection pool Automatic (client) fail over for High Availability Client MySQL Server 1 MySQL Server 2 MySQL Server n
  • 8. The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a 1:n relation. 1:n puts an additional task on the application: find the right n, find the right server to talk to.
  • 9. The plugin for all of you WordPress, Drupal, Symfony, Oxid, ZendFramework, ... mysql, mysqli, PDO_MYSQL mysqlnd PECL/mysqlnd_ms plugin Statement Redirection Balancing Failover Connection Pooling MySQL Server 1 MySQL Server 2 MySQL Server n
  • 10. The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library. The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thus all existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy. Depending on the use case, no code changes are needed. PECL/mysqlnd_ms can be a drop-in solution.
  • 11. Wait - what's wrong with this? Go for it! But don't miss our goodies... class DBRepl { private static $master_list = array(...); private static $master = NULL; private static $slave_list = array(...); private static $slave = NULL; public static function WriteConn() { if (self::$master) return self::$master; /* pick and initialize ...*/ } public static function ReadConn() { if (self::$slave) return self::$slave; /* pick and initialize ...*/ } }
  • 12. The speaker says... A client-side approach to the problem is promising, if client applications can be changed. It is no drop-in solution. The load balancer is part of the client application. It scales by client and it fails by client. There is no single point of failure as it is the case with a proxy based approach. No additional latency occurs like with a proxy. Load balancing can be adaptive for good resource usage. DBMS node failures do not block clients, fail over is possible. PECL/mysqlnd_ms has all this, is semi-transparent and offers many more features!
  • 13. PECL/mysqlnd_ms 1.4 Anything missing? All PHP MySQL APIs, semi-transparent, drop-in Read write splitting, transaction aware Load balancing, fail over (optional: automatic) Service levels: consistency, slave lag limit, trottling Optional, automatic caching of read results User can control all automatic actions 75+ pages documentation: hands on and reference Stable, more test code than C code Some 10 bug reports since 1.0
  • 14. The speaker says... No worry, it is not complicated. All the magic is beneath the hood, in the C code of PECL/mysqlnd_ms. The user interface is short and comprehensive. This is all what PECL/mysqlnd_ms is about: make using any kind of MySQL Cluster easier! Let the load balancer do the work. Provide a cluster-aware API. Clustering is mainstream. We need new APIs.
  • 15. Read/Write splitting Default: mysqlnd_ms.disable_rw_split = 0 read-only: statement begins with SELECT read-only: statement begins with slave SQL hint custom: callback picks slave for statement Client Writes Reads (SELECT, SQL hint) Master Slave Slave Slave
  • 16. The speaker says... Read/Write splitting must be done for primary copy clusters, like MySQL Replication. Clients must direct all writes to the primary (master). PECL/mysqlnd_ms automatically directs statements that begin with SELECT or a certain SQL hint to one of the configured secondaries (slaves). An update anywhere cluster, such as MySQL Cluster, allows writes to be performed on all nodes. No read write splitting is needed. Thus, you can disable it on a global level using the configuration directive mysqlnd_ms.disable_rw_split.
  • 17. Redirection and transactions No switch allowed during a transaction API calls controlling transactions monitored (5.4+) SQL not checked, protocol does not annnounce state Solution: use API (5.4+) or SQL hints! Client UPDATE SET col=@my BEGIN; SELECT @my=1; Master Slave Slave Slave
  • 18. The speaker says... Statement redirection for load balancing bares a pitfall: connection state. Session variables and transactions are part of the state. A load balancer must not switch servers in the middle of a transaction. When using primary copy, all transactions shall go to the primary (master) because it is unknown at the beginning of a transaction whether it includes updates. The MySQL Client Server protocol does not hint whether a transaction is active. Load balancers must either parse SQL to catch BEGIN, monitor API calls or be hinted by the application. PECL/mysqlnd_ms monitors the API and understands hints.
  • 19. Transaction aware code Use API call over SQL to control transactions For MySQL Replication, set: trx_stickiness = master Check master_on_write=1 or mysqlnd_ms_set_qos() $mysqli = new mysqli("myapp", "username", "password", "database"); /* Disable autocommit, master used, no server switch allowed */ $mysqli->autocommit(FALSE); /* ... */ if (!$mysqli->commit()) { die(sprintf("[%d] %sn", $mysqli->errno, $mysqli->error)); } /* Transaction has ended, load balancing begins again */ $mysqli->autocommit(TRUE);
  • 20. The speaker says... Use the API calls to control transactions. Give the database driver a chance to help you. Virtually any state change that you perform through API calls is handled by PECL/mysqlnd_ms. For example, if you change the charset with mysqli_set_charset(), the plugin makes sure that all opened connections use the same charset. We hide complexity, make cluster use easier. Consider using master_on_write=1. This will keep all reads followed by the first write on the master. Handy, if you use asynchronous cluster and eventual consistency is not enough. Wait for quality-of-service, its getting better!
  • 21. Is this your code? PECL/mysqlnd_ms does not look worse, does it? $mysqli = DBRel::ReadConn(); /* ... */ $mysqli = DBRepl::WriteConn() $mysqli->autocommit(FALSE); /* ... */ if (!$mysqli->commit()) { die(sprintf("[%d] %sn", $mysqli->errno, $mysqli->error)); } /* ... */ $mysqli->autocommit(TRUE); $mysqli = DBRepl::ReadConn();
  • 22. The speaker says... The differences to the classic approach become obvious. The plugin code has less „hints“, the abstraction is less often informed of the need of a certain type of database. The classic code is specifically written for a certain kind of cluster. Once you migrate the database from a lazy primary copy system (e.g. MySQL Replication) to an eager update anywhere solution (e.g. MySQL Cluster) you have additional calls in your code that are no more needed. The plugin code needs nothing but a configuration file change to switch from MySQL Replication to MySQL Cluster.
  • 23. Load Balancing Built-in or user-defined through callback Round robin, Random, Random once 1.3 – kind of adaptive by setting limits for slave lag 1.4 – nodes can be assigned a weight/probability Client Writes Load Balancing of Reads Master Slave Slave Slave Weight = 1 Weight = 2 Weight = 1 Weight = 1
  • 24. The speaker says... The best and default load balancing stategy is the one with the lowest impact on the connection state: random once. Minimize switches for the life-span of your PHP script. Pick a random slave during startup and use it for all reads. The life-span of PHP is the of a web request. It is short. Two web requests will end up on two random slaves. The upcoming 1.4 release will allow you to assign a weight to a node. Nodes with a weight of two will get twice as many requests as nodes with a weight of one. This is not only great if your nodes have different computing power but also to prefer local nodes over remote ones by setting a higher weight.
  • 25. Pooling: connection failover Caution: connection state change Default: disabled = raise exception Optional: silently failover master 1.4 – remember failed, try more slaves before master Client Writes Reads Master Slave Slave Slave
  • 26. The speaker says... Connection state changes are what speaks against automatic, silent fail over once a connection has been established and fails. If your application is written in a way that automatic reconnect upon failure is allowed, you can enable it. Otherwise, handle the error, do not close the connection but run the next query to trigger a new load balancing attempt! Failed nodes can be remembered for the duration of a request. Automatic failover can either try to connect to the master or try to use another slave before failing over to the master. If using MySQL Cluster, the search logic still applies with the only difference that only masters are to be configured.
  • 27. Pooling: lazy connections Do not open connection before needed Delay open until statement execution Reduce number of connections, reduce load Great for legacy apps – even without load balancing Client Writes Reads Master Slave Slave Slave
  • 28. The speaker says... By default, to reduce the connection pool size of every PHP web request, connections are not opened before executing a statement. Imagine you have 50 concurrent web requests, you configured four slaves and, you are using random once. If not using lazy connections, the plugin opens 50 x 4 = 200 slave connections but uses at max 50 of them (random once). Every connection occupies resources – ever seen a PHP application opening connections before using? There are many badly written ones... Many connection state changing API calls are buffered and replayed when opening lazy to prevent pitfalls.
  • 29. Quality of service Don't bother about node selection – MS does! mysqlnd_ms_set_qos() Eventual, session and strong consistency Allow only certain slave lag, enable caching Client PECL/mysqlnd_ms Database cluster as a service Node Node Node Node
  • 30. The speaker says... The basics (load balancing, failover) are solved. Abstraction can start. We can start to see the cluster as a service. The load balancer knows how to use the cluster to achieve a certain quality of service. The application formulates its service quality requirements, the load balancer takes appropriate action. For example, state the consistency level needed. Is reading stale data from a slave allowed (eventual consistency) and if so, is there any limit on freshness? Having API calls to know the client demands is a great help for load balancer and cluster vendors!
  • 31. Eventual consistency Nodes may or may not serve the latest copies Nodes may serve stale copies or not have copies at all Default consistency level of MySQL Replication SET X = 1 MySQL Node MySQL Node MySQL Node GET X, X = 0 GET X, NULL
  • 32. The speaker says... An eventual consistent node may or may not serve the latest copy. In fact, there is no promise that a particular copy is available from every node. Many systems that default to eventual consistency reach strong consistency over time. Eventually all nodes get synchronized. This model is similar to that of a cache. Eventual consistency is good enoug for browsing product catalogs or other infrequently changing contents in areas where stale data is acceptable. It is the default consistency level with MySQL Replication. Why bother ?! Can't vendors provide proper solutions?
  • 33. Clustering databases is complex There is no one-fits-all replication solution The dangers of replication and a solution (Jim Gray, Pat Helland, 1996): a ten-fold increase in nodes and traffic gives a thousand fold increase in deadlocks or reconciliations CAP theorem (Eric Brewer, 2000): consistency, availability and paritition tolerance can't be achieved together … vendors are forced to offer compromises … vendors are faced with a highly complex problem … you are forced to know about consistency :-(
  • 34. The speaker says... I would love to stop talking about consistency. But replication has limits. Keep on pushing but know the limits. Every synchronous, update anywhere solution has a scalability limit. Partitioning (sharding) only shifts the limits and creates other issues (rebuilding aggregates). Until its solved... Applications shall state their QoS/consistency needs: To allow vendors to relax consistency for performance reasons, but only if feasible, if allowed by the app To allow load balancers to make an optimal node choice = PECL/mysqlnd_ms 1.2+ build around GTIDs...
  • 35. Setting QoS for consistency Better than master_on_write = 1 $mysqli = new mysqli("myapp", "username", "password", "database"); /* read-write splitting: master used */ $mysqli->query("INSERT INTO orders(item) VALUES ('elePHPant')"); /* Request session consistency: read your writes */ mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_SESSION); /* Plugin picks a node which has the changes, here: master */ $res = $mysqli->query("SELECT item FROM orders WHERE order_id = 1"); var_dump($res->fetch_assoc()); /* Back to eventual consistency: stale data allowed */ mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL); /* Plugin picks any slave, stale data is allowed */ $res = $mysqli->query("SELECT item, price FROM specials");
  • 36. The speaker says... An eager update anywhere replication system such as MySQL Cluster delivers strong consistency. Classic users of MySQL Replication achieve different level of consistency using the simple rules: eventual consistency – any slave, session and strong consistency – master only. Using the master for any level beyond eventual consistency has a negative impact on read scale out. MySQL 5.6 Global Transactions Identifiers help to achieve session consistency even when reading from slaves – reliable read your writes has become possible. PECL/mysqlnd_ms implements the node selection logic.
  • 37. Global transaction identifier Combination of server id and sequence number Emulation: PECL/mysqlnd_ms 1.2+, MySQL Proxy Built-in: MySQL 5.6 MySQL Master Log 7, Pos 34, GTID M:1: UPDATE x=1 Log 7, Pos 35, GTID M:2: UPDATE x=9 MySQL Slave 1 MySQL Slave 2 … , GTID M:1: UPDATE x=1 … , GTID M:1: UPDATE x=1 … , GTID M:2: UPDATE x=9
  • 38. The speaker says... A global transaction identifier is a cluster-wide unique transaction identifier. MySQL 5.6 can generate it automatically. MySQL Proxy and PECL/mysqlnd_ms 1.2 feature client-side emulations for use with any MySQL version. SQL can be used to access the GTIDs. GTIDs have been been created to make MySQL Replication failover easier. However, they are useful for load balancing as well in a primary copy system.
  • 39. MySQL Replication: strong con. With or without GTID: primary only Only the primary has all comitted transactions Client A Client B MySQL Master SET X = 1 GET X, X = 1 MySQL Slave MySQL Slave
  • 40. The speaker says... Configuring PECL/mysqlnd_ms for use with a MySQL Replication cluster and calling mysqlnd_ms_set_qos($conn, MYSQLND_MS_QOS_CONSISTENCY_STRONG) instructs PECL/mysqlnd_ms to use only the MySQL Replication master server for requests. In a lazy primary copy system there is only one node that is guaranteed to have all comitted updates: the primary. Please note, that its possible to achieve higher consistency levels than eventual consistency in an lazy primary copy system by appropriately choosing nodes.
  • 41. MySQL Replication: session con. Use GTID to find „synchronous“ slave Check slave status using SQL Reduce read load on master SET X = 9 MySQL Master ..., GTID M:1: UPDATE x=1 ..., GTID M:2: UPDATE x=9 GET X, X = 9 MySQL Slave 1 MySQL Slave 2 … , GTID M:1: UPDATE x=1 … , GTID M:1: UPDATE x=1 … , GTID M:2: UPDATE x=9
  • 42. The speaker says... Global transaction identifier help to find „up-to-date“ slaves that have already replicated the latest updates of a client. Thus, session consistency can now be achieved by reading from the master and selected „up-to-date“ slaves. This works with the GTID emulation of PECL/mysqlnd_ms 1.2+ and any MySQL version. And, it works with MySQL 5.6 that has built-in GTIDs. PECL/mysqlnd_ms 1.4 can either wait for a slave to catch up or search all slaves before considering the master. The wait effectively means throttling slaves to reduce slave lag!
  • 43. MySQL Replication: eventual c. With or without GTID: all slaves Optional QoS level: upper slave lag limit MySQL estimates slave lag! SET X = 9 MySQL Master GET X, X = 8 MySQL Slave 1 MySQL Slave 2 Slave lag = 1 second Slave lag = 7 seconds
  • 44. The speaker says... A MySQL Replication slave is eventual consistent – it may or may not have the latest updates. There is no need to filter nodes with regards to consistency. Slaves can be filtered by replication lag: mysqlnd_ms_set_qos($conn, MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL, MYSQLND_MS_QOS_OPTION_AGE, 5) filters out all slaves that report an estimated lag of more than five seconds.
  • 45. Slave selection logic Same logic whenever slaves are to be filtered applied for: session consistency + GTID applied for: eventual consistency + Lag Stage 1: send SQL to check status to all slaves Stage 2: fetch replies in order Stage 3: apply filter logic SQL is documented in the manual
  • 46. The speaker says... Whenever PECL/mysqlnd_ms has to check the slave status for filtering slaves, the check is done in two stages. First, all slaves are queried. Then, replies are fetched from the slaves in order. Usually, many requests can be send out in stage one before the servers reply. The SQL details are documented at php.net/mysqlnd_ms.
  • 47. QoS: Cache Integration Automatically cache data no older than n seconds Reduce slave load Caches: Memcache, process memory, … PHP 5.4, PECL/mysqlnd_qc 1.1 1st GET X, X = 8 MySQL Slave 1 MySQL Slave 2 Slave lag = 1 second Slave lag = 7 seconds Max_TTL = 10 – 1 = 1 TTL = MAX_AGE - 9 TTL = MAX_AGE - 7 2nd GET X, X = 8 Cache
  • 48. The speaker says... If the load balancer knows that eventual consistency is good and a certain degree of stale data (age, slave lag) is allowed, a slave access can be transparently replaced with a cache access. This lowers the overall slave load in the cluster and reduces latency. Replacing a slave access with a cache access is transparent from an application point of view. PECL/mysqlnd_ms is using PECL/mysqlnd_qc (qc = query cache) for the caching. PECL/mysqlnd_qc can also be used as a standalone cache to store query results in Memcache, process memory and many more caches.
  • 49. Transparent caching $mysqli = new mysqli("myapp", "username", "password", "database"); /* no caching */ $res = $mysqli->query("SELECT headline FROM latest_news"); var_dump($res->fetch_all(MYSQLI_ASSOC)); /* use cache, if slave lag allows */ mysqlnd_ms_set_qos($mysqli, MYSQLND_MS_QOS_CONSISTENCY_EVENTUAL, MYSQLND_MS_QOS_OPTION_CACHE, 60)); $res = $mysqli->query("SELECT headline FROM old_news"); var_dump($res->fetch_all(MYSQLI_ASSOC));
  • 50. The speaker says... Complicated logic behind but easy to use – PHP should look out for new cluster aware APIs. Let's close with an example of a minimal configuration for use with MySQL Replication.
  • 51. Speaking code: configuration /* php.ini */ mysqlnd_ms.enable=1 mysqlnd_ms.config_file=/path/to/mysqlnd_ms_plugin_json.conf /* /path/to/mysqlnd_ms_plugin_json.conf contents */ { "myapp": { "master": { "master_0": { "host": "localhost", "socket": "/tmp/mysql.sock" }}, "slave": { "slave_0": { "host": "192.168.2.27", "port": "3306" }}}}
  • 52. Connect and use... - as ever $mysqli = new mysqli("myapp", "username", "password", "database"); $pdo = new PDO('mysql:host=myapp;dbname=database', 'username', 'password'); $mysql = mysql_connect("myapp", "username", "password"); /* Statements will be run on the master */ $mysqli->query("DROP TABLE IF EXISTS test"); $mysqli->query("CREATE TABLE test(id INT)"); $mysqli->query("INSERT INTO test(id) VALUES (1)"); /* Goes to the slave - don't let slave lag fool you! */ $res = $mysqli->query("SELECT id FROM test"); var_dump($res->fetch_assoc()); $mysqli->close();
  • 53. The speaker says... Imagine we would have failover that could be bound to certain error codes. For example, assume the SELECT returns „Error: 1051 SQLSTATE: 42S02 (ER_BAD_TABLE_ERROR), Message: Unknown table '%s' „ because you forgot about possiblity of a lag between the CREATE and on the master and the SELECT on the slave. What if PECL/mysqlnd_ms would automatically fail over to the master to retry the SELECT... Sure, the better solution is to as for session consistency to avoid this issue at all. Let us know what additional features you would like to see in PECL/mysqlnd_ms. We are ready to code...
  • 55. Kvack? Bonus slides! Load Balancing layers overview
  • 56. Whole stack cluster Simple – 1:1 mapping! Requires synchronous cluster No fault tolerance: node failure = stack failure Inefficient resource usage: no balancing HTTP Server HTTP Server HTTP Server App Server (PHP) App Server (PHP) App Server (PHP) MySQL Node MySQL Node MySQL Node
  • 57. The speaker says... In a synchronous cluster, for example, if using MySQL Cluster, all nodes have all the data. Thus, every application server could be assigned to one DBMS node. Easy, fast and good for MySQL Cluster but with limits. No good for asynchronous MySQL Replication. Limit: DBMS node failure includes application node failure. Client should have additional fail over logic. Limit: Over- and undercapacity of a DBMS node cannot be compensated. Clients cannot switch to more powerful nodes. Overcapacity of a MySQL node cannot be used by other clients. Different hardware size is ignored.
  • 58. Proxy approach - Pro Free and open source MySQL Proxy No application changes, free 3rd party proxy scripts MySQL node failure does not stop application server Good resource usage, dynamic adoption possible HTTP Server HTTP Server HTTP Server App Server (PHP) App Server (PHP) App Server (PHP) MySQL Proxy MySQL Node MySQL Node MySQL Node
  • 59. The speaker says... A transparent MySQL Proxy based solution requires no application changes. Clients connect to the Proxy using MySQL Client Server Protocol, as if the MySQL Proxy was a MySQL Server. Proxy can compensate for DBMS failures. It can react to temporary and permant outages. Proxy can adapt load balancing dynamically. Taking into account node hardware size, current node load, latency, location, … fantasy sets the limits here for self-written or 3rd party Proxy scripts.
  • 60. Proxy approach - Con Additional component Complex architecture, can be single point of failure C and Lua are no natural match for PHP lovers Adds latency: from client to proxy to node HTTP Server HTTP Server HTTP Server App Server (PHP) App Server (PHP) App Server (PHP) MySQL Proxy MySQL Node MySQL Node MySQL Node
  • 61. The speaker says... MySQL Proxy is a great performer! But, … MySQL Proxy adds complexity to the stack. MySQL Proxy needs to be managed. MySQL Proxy is build around C and Lua. C and PHP would be a better match for PHP users. Wrongly used, MySQL Proxy becomes a single point of failure. It is single threaded. This bares the risk of tasks (Proxy scripts) blocking each other. But, … MySQL Proxy adds latency. Although, it can be minimized significantly running MySQL Proxy on the App Server to avoid use of TCP/IP between PHP and Proxy.
  • 62. Client-side load balancer - Pro Client application handles load balancing No single point of failure No additional latency Highly configurable and adaptive HTTP Server HTTP Server HTTP Server App Server (PHP) App Server (PHP) App Server (PHP) <?php .. ?> <?php ...?> <?php ?> MySQL Node MySQL Node MySQL Node
  • 63. The speaker says... A client-side approach to the problem is promising, if client applications can be changed. It has most Pro's of the previous approaches. The load balancer is part of the client application. It scales by client and it fails by client. Scalability is given and there is no single point of failure. No additional latency occurs. Load balancing can be adaptive for good resource usage. DBMS node failures do not block clients, fail over is possible.
  • 64. Client-side load balancer - Con Client application handles load balancing Application must be designed for (specific) cluster use No drop-in solution for existing applications PHP is stateless: adaptive load balancing is difficult HTTP Server HTTP Server HTTP Server App Server (PHP) App Server (PHP) App Server (PHP) <?php .. ?> <?php ...?> <?php ?> MySQL Node MySQL Node MySQL Node
  • 65. The speaker says... The major downside of a client-side application based solution is the need to change the application. Application changes must be done even for basic cluster use cases. Modifications to source code may not be possible. They may complicate upgrades of standard software. They may cause lots of work thus become expensive. Load balancing is part of PHP thus stateless. This is both a Pro and a Con. It is difficult to hint other clients about node failures. On the other hand, shared nothing is not a bad idea either.
  • 66. Is driver based the solution? Client-side and driver based: PECL/mysqlnd_ms Pro: all previous Synchronous cluster: no application changes Asynchronous cluster: no change for basic cases HTTP Server HTTP Server HTTP Server App Server (PHP) App Server (PHP) App Server (PHP) PECL/mysqlnd_ms PECL/mysqlnd_ms PECL/mysqlnd_ms MySQL Node MySQL Node MySQL Node
  • 67. The speaker says... A driver based client-side solution has all Pro's! Considering load balancing aspect only, no application changes are required. If using MySQL Cluster, a synchronous cluster, no application changes are required. If using MySQL Replication, an asynchronous cluster that needs read-write splitting, the need for application changes is significantly reduced! The PECL mysqlnd_ms installation can be part of the next PHP deployment, done anyway for security considerations. No new procedure. If any issues, simply disable extension.