SQL versus NoSQL and how MySQL's InnoDB/Memcached interface can keep you sane.

1. Dave Stokes
MySQL Community Manager
David.Stokes@Oracle.com
SQL & NoSQL:
How 'Big Data' & MySQL Work Together
Presenting with
David.Stokes@Oracle.Com LOGO
1
MySQL Community Manager
Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8

2. Program Agenda
•
WHY SQL / NoSQL
•
Alternatives to SQL
•
Big Data
•
Best of both worlds – InnoDB/memcached
•
MySQL Cluster – NDB/memcached
•
Q&A
Synopsis – How to use MySQL as a relational data store
according to Codd & Date while gaining the ability to
access schema-less data and looking cool while doing it.
2 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

3. SQL – Cod and Date
Images from Wikipedia.com
3 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

4. Codd & Date
Wikipedia: Edgar Frank "Ted"
Codd was an English computer
scientist who invented the
relational model for database
management, the theoretical
basis for relational
databases. ...Codd continued to
develop and extend his
relational model, sometimes in
collaboration with Chris Date
4 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

5. Codd's Relational Model
The purpose of the relational model is to
provide a declarative method for specifying
data and queries: users directly state what
information the database contains and what
information they want from it, and let the
database management system software take
care of describing data structures for storing
the data and retrieval procedures for
answering queries. Wikipedia
5 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

6. Not all data relational or easy to
extract using SQL
Common NoSQL uses
● Document Stores / Fuzzy schemas
● A 'Facebook Query' – find the friends of your friends …
and then their friends
● Data size may be too large for RDMS or OS
● Coolness factor
6 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

7. NoSQL
● Database management without relational
model, schema free
● Does not use SQL (some retrofitting)
● Usually not ACID
● Eventually consistent data
● Distributed, fault-tolerant
● Large amounts of data
7 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

8. Common NoSQL Types
● Key value stores
● Document databases
● Graph databases
● XML databases
● Distributed peers
● Object stores
8 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

9. Infobright's Emerging Database Landscape
http://bit.ly/emerging_db_landscape
Row Based Columnar NoSQL – Key Value NoSQL – Dcoument NoSQL Columnar
store Store Store
Common uses Transaction Historical data Cache for storing Web apps or app Real-time data
processing analysis, data frequently needing scaling w/o logging
warehousing, BI requested data defined schema
Basic Description Data structured in Data structured in Data stored in Persistent storage, Very large data
rows columns memory some SQL-like storage,
querying MapReduce support
Strengths Capturing/inputting Fast query support Scalability, very fast Persistent store, Very high
new records; on data sets; storage/retrieval of scalable; better throughput; strong
Robust, proven compression data query support than partitioning; random
technology key-value stores read-write access
Weaknesses Scale issues Import/export Usually all data Lack of Low level API;
speed; heavy must fit into sophisticated Inability to do
computing resource memory, no capabilities complex queries;
needed complex queries high query latency
Typical Database Several GB to 50 Several GB to Several TB to Several TB to
Size Range TB several TB several PB several PB
Key Players MySQL, Oracle, Infobright, Aster Memacached, MongoDB, Hbase, Big Table,
SQL Server, Sybase Data, Sybase IQ, Amazon S3, Redis, CouchDB, Cassandra
ASE Vertica, ParAccel Voldemort SimpleDB
9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

10. An example using MongoDB
db.cars.insert(
{make: 'Ford',
model: 'F-150',
cylinders: 8})
db.cars.find(
cylinders: {$gte: 8})
10 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

11. Hadoop
Part 1 – Reliable Part 2 – Parallel
data storage using data processing
the Hadoop using map/reduce
Distributed File
System (HDFS)
● Can get expensive for hardware, not for all
data
11 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

12. Tilting relational databases on their side
Columnar databases align by column, not
rows
● High compression possible
● OLAP & Data Warehousing
● MySQL engines
● Calpont's InfiniDB
● Infobright
12 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

13. When not to use NoSQL
Your data is
● Relational
– Some hierarchy
– schema
● Need ACID
● Do not like lots of servers, disk farms
13 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

14. memcached before MySQL 5.6
memcached
is a general-purpose
distributed memory
caching system
--Wikipedia
14 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

15. MySQL's use of memcached for NoSQL
● Innodb or NDB storage engines
● Access same data (same disks) either
through SQL or memcached
● 1,000,000,000+ transactions a minute for
MySQL Cluster
● Many sites already using memcached
- already in use, well known, easy to implement
15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

16. Diagrammatic Overview
16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

17. Why this is cool?
● memcached as a daemon plugin of mysqld: both mysqld and
memcached are running in the same process space, with very
low latency access to data
● Direct access to InnoDB: bypassing SQL parser and
optimizer
● Support standard protocol (memcapable): support both
memcached text-based protocol and binary protocol; all 55
memcapable tests are passed
● Support multiple columns: users can map multiple columns
into “value”. The value is separated by a pre-defined
“separator” (configurable).
17 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

18. Why is this cool? 2
● Optional local caching: three options – “cache-only”, “innodb-
only”, and “caching” (both “cache” and “innodb store”). These
local options can apply to each of four Memcached operations
(set, get, delete and flush).
● Batch operations: user can specify the batch commit size for
InnoDB memcached operations via
“daemon_memcached_r_batch_size” and
“daemon_memcached_w_batch_size” (default 32)
● Support all memcached configure options through MySQL
configure variable “daemon_memcached_option
18 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

19. PHP Example of using memcached
function get_foo(int userid) { function get_foo(int userid) {
data = db_select("SELECT * FROM /* first try the cache */
users WHERE userid = ?", userid);
data = memcached_fetch("userrow:" + userid);
return data;
if (!data) {
}
/* not found : request database */
data = db_select("SELECT * FROM users WHERE
userid = ?", userid);
Rewriten to /* then store in cache until next get */
use memcached_add("userrow:" + userid, data);
memached }
return data;
}
19 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

20. Example InnoDB/memcached
● SQL
● mysql> INSERT INTO demo_test
VALUES ('dave','it works', 10, 200, NULL)g
● Memcached
20 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

21. Getting it running
● Install MySQL 5.6.6
● /scripts/innodb_memcached_config.sql
● Creates test.demo_test
– Key (c1) – CHAR/VARCAHR
– Value (c2) – CHAR/VARCHAR
– Flag (c3) – 32bit Integer
– CAS (c4) – 64bit Integer
– Exp (c5) – 32bit integer
● mysql> install plugin daemon_memcached soname
“libmemcached.so”;
● mysql> set session TRANSACTION ISOLATION
LEVEL read uncommitted; /* ignore batches */
21 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

22. MySQL Cluster –
NDB and/or memcahced
22 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

23. MySQL Cluster quick review
Fault tolerant, auto
sharding, shared
nothing, data on
redundant boxes,
99.999% up time,
ACID,
geographical
replication
between clusters,
& no single point of
failure
23 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

24. Option 1 co-locate the memcached API
with the data nodes
The applications can connect
to any of the memcached API
nodes – if one should fail just
switch to another as it can
access the exact same data
instantly. As you add more data
nodes you also add more
memcached servers and so the
data access/storage layer can
scale out (until you hit the 48
data node limit).
24 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

25. Separate Layer
For maximum
flexibility, you can
have a separate
Memcached layer
so that the
application, the
Memcached API &
MySQL Cluster can
all be scaled
independently.
25 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

26. Co locate with Application
Another simple option is to co-
locate the Memcached API with
the application. In this way, as you
add more application nodes you
also get more Memcached
throughput. If you need more data
storage capacity you can
independently scale MySQL
Cluster by adding more data
nodes. One nice feature of this
approach is that failures are
handled very simply – if one
App/Memcached machine should
fail, all of the other applications
just continue accessing their local
Memcached API.
26 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

27. In all of the last three examples,
there has been a single source for
the data (it’s all in MySQL Cluster)
.
● If you choose, you can still have all or some of the data cached within the
memcached server (and specify whether that data should also be persisted
in MySQL Cluster) – you choose how to treat different pieces of your data.
If for example, you had some data that is written to and read from frequently
then store it just in MySQL Cluster, if you have data that is written to rarely
but read very often then you might choose to cache it in memcached as
well and if you have data that has a short lifetime and wouldn’t benefit from
being stored in MySQL Cluster then only hold it in memcached. The beauty
is that you get to configure this on a per-key-prefix basis (through tables in
MySQL Cluster) and that the application doesn’t have to care – it just uses
the memcached API and relies on the software to store data in the right
place(s) and to keep everything in sync.
● Of course if you want to access the same data through SQL then you’d
make sure that it was configured to be stored in MySQL Cluster.
27 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

28. OTHER OPTIONS
•
There are other options for Big Data
and NoSQL that are beyond the
•
Scope of this presentation ..
•
Although I can not think of
•
Any thing to point to as an
•
Example :-)
ORACLE
PRODUCT
LOGO
28 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8

29. David.Stokes@Oracle.com
@Stoker
Q&A
slideshare.net/davestokes/presentations
29 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8