3. Safe Harbor Agreement
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT
DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY,
AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A
COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR
FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING
PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND
TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR
ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF
ORACLE.
3
7. Relational Databases
● Original Goal was to save data with minimal duplication
● Disks were expensive
● … and slow
● 45 years old
● Introduced the concept of accessing many records with a single command
7
9. Relational Databases
● Need to set up tables BEFORE use
● Relations, indexes, data normalization, query optimizations
● Hard to change on the fly
● Need a DBA or someone who has DBA skills
● This can be a chokepoint
9
11. NoSQL or Document Store
● Schemaless
○ No schema design, no normalization, no foreign keys, no data types, …
○ Very quick initial development
● Flexible data structure
○ Embedded arrays or objects
○ Valid solution when natural data can not be modeled optimally into a
relational model
○ Objects persistence without the use of any ORM - *mapping
object-oriented*
11
12. NoSQL or JSON Document Store
● JSON
● close to frontend
● native in JS
● easy to learn
12
13. How DBAs see data as opposed to how Developers see data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Federation",
"HeadOfState" : "Philippe I"
},
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.8000030517578
},
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}
13
14. What if there was a way to provide both SQL
and NoSQL on one stable platform that has
proven stability on well know technology
with a large Community and a diverse
ecosystem ?
With the MySQL Document
Store it is now an option!
14
15. A Solution for all
Developers:
schemaless
★ rapid prototyping &
simpler APIs
★ document model
★ transactions
Operations:
★ performance
management/visibility
★ robust replication, backup,
restore
★ comprehensive tooling
ecosystem
★ simpler application schema
upgrades
15
Business Owner:
★ don't lose my data == ACID
trx
★ capture all my data =
extensible/schemaless
★ product on schedule/time to
market = rapid development
16. Built on the MySQL JSON Data type and Proven MySQL Server Technology 16
★ Provides a schema flexible JSON Document Store
★ No SQL required
★ No need to define all possible attributes, tables, etc.
★ Uses new MySQL X DevAPI
★ Can leverage generated column to extract JSON values
into materialized columns that can be indexed for fast
SQL searches.
17. Built on the MySQL JSON Data type and Proven MySQL Server Technology 17
★ Document can be ~1GB
○ It's a column in a row of a table
★ Allows use of modern programming styles
○ No more embedded strings of SQL in your
code
○ Easy to read
★ Also works with relational Tables
★ Proven MySQL Technology
18. ★ C++
★ Java
★ .Net
★ Node.js
★ JavaScript
★ Python
★ PHP
○ Working with other Communities to help them support it too 18
Connectors for
19. ★ Command Completion
★ Python, JavaScripts & SQL modes
★ Admin functions
★ New Util object
★ A new high-level session concept that can scale from single MySQL
Server to a multiple server environment
19
New MySQL Shell
20. ★ Non-blocking, asynchronous calls follow common language patterns
★ Send out many queries and process other things until they return
★ Supports CRUD operations
★ Concentrate on basic functions
★ Easily scale from one server to InnoDB cluster w/o changing application!
20
New Model
25. JavaScript 25
// Connecting to MySQL Server and working with a Collection
var mysqlx = require('mysqlx');
// Connect to server
var mySession = mysqlx.getSession( {
host: 'localhost', port: 33060,
user: 'user', password: 'password'} );
var myDb = mySession.getSchema('test');
// Create a new collection 'my_collection'
var myColl = myDb.createCollection('my_collection');
// Insert documents
myColl.add({_id: '1', name: 'Sakila', age: 15}).execute();
myColl.add({_id: '2', name: 'Susanne', age: 24}).execute();
myColl.add({_id: '3', name: 'User', age: 39}).execute();
// Find a document
var docs = myColl.find('name like :param1 AND age < :param2').limit(1).
bind('param1','S%').bind('param2',20).execute();
// Print document
print(docs.fetchOne());
// Drop the collection
myDb.dropCollection('my_collection');
No SQL!!
26. Python 26
# Connecting to MySQL Server and working with a Collection
from mysqlsh import mysqlx
# Connect to server
mySession = mysqlx.get_session( {
'host': 'localhost', 'port': 33060,
'user': 'user', 'password': 'password'} )
myDb = mySession.get_schema('test')
# Create a new collection 'my_collection'
myColl = myDb.create_collection('my_collection')
# Insert documents
myColl.add({'_id': '1', 'name': 'Sakila', 'age': 15}).execute()
myColl.add({'_id': '2', 'name': 'Susanne', 'age': 24}).execute()
myColl.add({'_id': '3', 'name': 'User', 'age': 39}).execute()
# Find a document
docs = myColl.find('name like :param1 AND age < :param2')
.limit(1)
.bind('param1','S%')
.bind('param2',20)
.execute()
# Print document
doc = docs.fetch_one()
print doc
27. Node.JS 27
// Connecting to MySQL Server and working with a Collection
var mysqlx = require('@mysql/xdevapi');
var db;
// Connect to server
mysqlx
.getSession({
user: 'user',
password: 'password',
host: 'localhost',
port: '33060',
})
.then(function (session) {
db = session.getSchema('test');
// Create a new collection 'my_collection'
return db.createCollection('my_collection');
})
.then(function (myColl) {
// Insert documents
return Promise
.all([
myColl.add({ name: 'Sakila', age: 15 }).execute(),
myColl.add({ name: 'Susanne', age: 24 }).execute(),
myColl.add({ name: 'User', age: 39 }).execute()
])
.then(function () {
// Find a document
return myColl
.find('name like :name && age < :age')
.bind({ name: 'S%', age: 20 })
.limit(1)
.execute(function (doc) {
// Print document
console.log(doc);
});
});
})
.then(function(docs) {
// Drop the collection
return db.dropCollection('my_collection');
})
.catch(function(err) {
// Handle error
});
28. C++ 28
// Connect to server
var mySession = MySQLX.GetSession("server=localhost;port=33060;user=user;password=password;");
var myDb = mySession.GetSchema("test");
// Create a new collection "my_collection"
var myColl = myDb.CreateCollection("my_collection");
// Insert documents
myColl.Add(new { name = "Sakila", age = 15}).Execute();
myColl.Add(new { name = "Susanne", age = 24}).Execute();
myColl.Add(new { name = "User", age = 39}).Execute();
// Find a document
var docs = myColl.Find("name like :param1 AND age < :param2").Limit(1)
.Bind("param1", "S%").Bind("param2", 20).Execute();
// Print document
Console.WriteLine(docs.FetchOne());
// Drop the collection
myDb.DropCollection("my_collection");
29. Java 29
// Connect to server
Session mySession = new
SessionFactory().getSession("mysqlx://localhost:33060/test?user=user&password=password");
Schema myDb = mySession.getSchema("test");
// Create a new collection 'my_collection'
Collection myColl = myDb.createCollection("my_collection");
// Insert documents
myColl.add("{"name":"Sakila", "age":15}").execute();
myColl.add("{"name":"Susanne", "age":24}").execute();
myColl.add("{"name":"User", "age":39}").execute();
// Find a document
DocResult docs = myColl.find("name like :name AND age < :age")
.bind("name", "S%").bind("age", 20).execute();
// Print document
DbDoc doc = docs.fetchOne();
System.out.println(doc);
// Drop the collection
myDB.dropCollection("test", "my_collection");
36. For this example, I will use the well known restaurants collection:
We need to dump the data to a file and
we will use the MySQL Shell
with the Python interpreter to load the data.
Migration from MongoDB to MySQL Document Store
36
37. Dump and load using MySQL Shell & Python
This example is inspired by @datacharmer's work: https://www.slideshare.net/datacharmer/mysql-documentstore
$ mongo quiet eval 'DBQuery.shellBatchSize=30000;
db.restaurants.find().shellPrint()'
| perl -pe 's/(?:ObjectId|ISODate)(("[^"]+"))/ $1/g' > all_recs.json
37
38. Or use new bulk loader in 8.0.13
38
Parallel import introduced in 8.0.17
39. BSON Support
Now, it supports the conversion of the following additional BSON types:
■ Date
■ Timestamp
■ NumberDecimal
■ NumberLong
■ NumberInt
■ Regular Expression
■ Binary
39
> util.importJson("/path_to_file/neighborhoods_mongo.json",
{schema: "test", collection: "neighborhoods",
convertBsonTypes: true});
53. What does a collection look like on the server ? 53
54. Every document has a unique identifier called the document ID, which can be
thought of as the equivalent of a table's primary key. The document ID value can
be manually assigned when adding a document.
If no value is assigned, a document ID is generated and assigned to the
document automatically !
Use getDocumentId() or getDocumentIds() to get _ids(s)
_id
54
55. Mapping to SQL Examples
createCollection('mycollection')
versus
CREATE TABLE `test`.`mycoll` (
doc JSON,
_id VARCHAR(32)
GENERATED ALWAYS AS (doc->>'$._id') STORED
PRIMARY KEY
) CHARSET utf8mb4;
55
56. Mapping to SQL Examples
mycollection.add({‘test’: 1234})
versus
INSERT INTO `test`.`mycoll` (doc)
VALUES ( JSON_OBJECT( 'test',1234));
56
57. More Mapping to SQL Examples
mycollection.find("test > 100")
Versus
SELECT doc
FROM `test`.`mycoll`
WHERE (JSON_EXTRACT(doc,'$.test') >100);
57
70. Find the top 10 restaurants by grade for each cuisine 70
WITH cte1 AS
(SELECT doc->>"$.name" AS name,
doc->>"$.cuisine" AS cuisine,
(SELECT AVG(score) FROM
JSON_TABLE(doc, "$.grades[*]" COLUMNS
(score INT PATH "$.score")) AS r) AS avg_score
FROM restaurants)
SELECT *, RANK() OVER
(PARTITION BY cuisine ORDER BY avg_score DESC) AS `rank`
FROM cte1
ORDER BY `rank`, avg_score DESC LIMIT 10;
This query uses a Common Table Expression (CTE) and a Windowing Function to rank the
average scores of each restaurant, by each cuisine assembled in a JSON_TABLE
71. No SQL Consumed In This Query!! 71
$schema = $session->getSchema("world");
$table = $schema->getTable("city");
$row = $table->select('Name','District')
->where('District like :district')
->bind(['district' => 'Texas'])
->limit(25)
->execute()->fetchAll();
73. JSON Validation
The Problem
Unlike strictly types relational databases there is no data normalization or ‘rigor’
applied to that data.
There is also no native way to do range checks
And there is no way have required fields
73
74. JSON-Schema.org
The Problem
Unlike strictly types relational databases there is no data normalization or ‘rigor’
applied to that data.
There is also no native way to do range checks
And there is no way have required fields
JSON-Shema.org is work to fix that
74
80. Index JSON Arrays
{ "user":"Bob", "user_id":31, "zipcode":[94477,94536] }
CREATE TABLE customers
( id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
modified DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
custinfo JSON,
INDEX zips( (CAST(custinfo->'$.zip' AS UNSIGNED ARRAY)) )
);
Mutli Value Indexes allow you to go past the 1:1 relation to index the data in JSON
arrays. And there are three special functions what can make use of MVIs when
used on the right side of a WHERE clause -- MEMBER OF(), JSON_CONTAINS(),
and JSON_OVERLAPS()
80
82. This is the best of the two worlds in one product !
● Data integrity
● ACID Compliant
● Transactions
● SQL
● Schemaless
● flexible data structure
● easy to start (CRUD)
82
86. New in MySQL 8.0
1. True Data Dictionary
2. Default UTF8MB4
3. Windowing Functions, CTEs, Lateral Derived Joins
4. InnoDB SKIPPED LOCK and NOWAIT
5. Instant Add Column
6. Histograms
7. Resource Groups
8. Better optimizer with new temporary table engine
9. True Descending Indexes
10. 3D GIS
11. JSON Enhancements
12. Hash Joins!
86
87. Please buy my book!
If you deal with the JSON
Data Type or have an interest
in the MySQL Document
Store, this text is a great
guide with many examples to
help you understand the
complexities and
opportunities with a native
JSON Data Type – SECOND
EDITION now Available on 87
89. “We have saved around 40% of our
costs and are able to reinvest that
back into the business. And we are
scaling across EMEA, and that’s
basically all because of Oracle.”
—Asser Smidt
CEO and Cofounder, BotSupply
Startups get cloud credits and a 70%
discount for 2 years, global exposure via
marketing, events, digital promotion, and
media, plus access to mentorship,
capital and Oracle’s 430,000+
customers
Customers meet vetted startups in
transformative spaces that help them
stay ahead of their competition
Oracle stays at the competitive edge
of innovation with solutions that
complement its technology stack
Oracle for Startups - enroll at oracle.com/startup
A Virtuous Cycle of Innovation, Everybody Wins.