SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Downloaden Sie, um offline zu lesen
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0:
An Internal Server ComponentThat Matters
Ståle Deraas, Senior Development Manager
Oracle, MySQL
27 Apr 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
What is a Data Dictionary?
Data Dictionary before MySQL 8.0
Transactional Data Dictionary in MySQL 8.0
What’s in it for you?
1
2
3
4
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What is a Data Dictionary?
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary
• Metadata is information about data in an RDBMS
– Column definitions, Index definitions, Foreign key definitions
...
• Data Dictionary is a collection of metadata for all data in an RDBMS
ID NAME WEIGHT HEIGHT GENDER
3 Bob 80 185 M
5 Liz 55 165 F
Metadata
Data
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Server
Data Dictionary
6
Role in Server
Query Executor
Optimizer
Storage
Engine
SQL
statement
Client
Parser
Result Data
Dictionary
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
CREATE PROCEDURE p1(v INT)
SQL SECURITY INVOKER
BEGIN
...
END
Data Dictionary
Types of Metadata
Data Dictionary
Table Definitions SP Definitions
View Definitions Schemas
ACL
CREATE TABLE customers(
id INT AUTO_INCREMENT
...
PRIMARY KEY (id),
INDEX ...
FOREIGN KEY ...
)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Dictionary before MySQL 8.0
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Data Dictionary before MySQL 8.0
• Metadata stored in a mix of files and tables :
– File based
• FRM, TRN, TRG, OPT, PAR
...
– Table based: non-transactional
• mysql.proc
...
– Table based: transactional
• Innodb.SYS
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Data Dictionary before MySQL 8.0
10
Data Dictionary
Files
FRM TRG OPT
System Tables (mysql.)
user procevents
InnoDB System Tables
MyISAM
File system
InnoDB
SQL
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL Data Dictionary before MySQL 8.0
• INFORMATION_SCHEMA queries are slow
• Inconsistencies due to non-transactional storage of metadata
• Inconsistencies between InnoDB metadata and Server metadata
• Showstopper for crash-safe / transactional DDL
• Replication is challenging, as DDLs are not atomic
• Difficult to extend
• No uniform API
11
Problems
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary
MySQL 8.0
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
13
Data Dictionary
InnoDB
SQL
DD TableDD TableDD Table
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
• All metadata stored in tables
• Single metadata repository for all MySQL server subsystems
• Reliable, crash-safe InnoDB tables
• INFORMATION_SCHEMA implemented as views over DD tables
– Queries can be optimized
– Improved performance
– Simpler, uniform implementation, easier to maintain
Main features
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
15
Overview
InnoDB
Data Dictionary
DD Table User Table
INFORMATION
SCHEMA
Views
Archive
User Table
CSV
User Table
Optimizer
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
• The data dictionary schema is based on SQL standard definitions
• Designed for automated upgrade of metadata
• The data dictionary is designed to be easily extended for new requirements
• Designed to allow plugins/components to add INFORMATION_SCHEMA
views and PERFORMANCE_SCHEMA tables
Main features, cont’ed
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
InnoDB
Transactional Data Dictionary in MySQL 8.0
17
Architecture
Query Executor
Parser Optimizer
Data
Dictionary
Tablespace
Data Dictionary API
Storage Engine
PluginUser TableUser Table
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary API in MySQL 8.0
• A single way to deal with Data Dictionary
– For the server core
– For Storage Engines
• A uniform API for all types of metadata
• Provide a way to handle Storage Engine private data
– Storage Engines no longer have to store their own metadata
– The API enables SEs to store key-value pairs
18
Design goals
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
SDI: Serialized Dictionary Information
- Copy of metadata from the Data Dictionary in JSON format
- Stored in data tablespaces(InnoDB) or .sdi file (MyISAM)
- Used for data migration and redundancy
The InnoDB Data Dictionary tablespace is the metadata storage
19
Reliability and Redundancy
InnoDB User Tablespaces
User Table
InnoDB Data Dictionary tablespace
mysql.tables
SDI
ID Name
1 User Table 1
2 User Table 2
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary in MySQL 8.0
What’s in it for you?
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Transactional Data Dictionary Benefits
• INFORMATION SCHEMA
– Improved Performance and Scalability
– Solves longstanding issues
• Reliability
– Atomic DDL
– Serialized Dictionary Information
– Automated Data Dictionary upgrade
• +++
21
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22
MySQLClient
I_S Query Results
MySQL Server
Create temporary table.
Heuristic optimization
Read metadata from File system or
from MyISAM/InnoDB engine.
.
TEMP TABLE
Return rows to user. File system / MyISAM / InnoDB engine
INFORMATION SCHEMA MySQL 5.7: Multiple sources, heuristics, creation of temp tables
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 23
MySQL Client
I_S Query Results
MySQL Server
Optimizer prepares
execution plan
Executor reads metadata from
data dictionary tables
InnoDB storage engine
Return rows to user
INFORMATION SCHEMA MySQL 8.0: Uniform, simple and using server standard features
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA Performance
Confidential – Oracle Internal/Restricted/Highly Restricted 24
Uniform, simpler implemention makes it a lot faster
MySQL Client
I_S Query Results
MySQL Server
Optimizer prepares
execution plan.
Executor reads metadata from
data dictionary tables.
InnoDB storage engine
Return rows to user.
INFORMATION_SCHEMA in 8.0
MySQL Client
I_S Query Results
MySQL Server
Create temporary table.
Heuristic optimization.
Read metadata from File system or
from MyISAM/InnoDB engine.
.
TEMP TABLE
Return rows to user.
INFORMATION_SCHEMA in 5.7
File system / MyISAM
/ InnoDB engine
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA Performance and Scalability
• Typically 30X performance improvements over MySQL 5.7
• More than 100X for some queries like: List all InnoDB table columns
25
I_S queries scale, both with database size and query load
0 20 40 60 80 100 120 140 160
List all InnoDB tables columns 5k tables
List all InnoDB tables columns 10k tables
MySQL 8.0
MySQL 5.7
Time in Seconds (Lower is better)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 26
100 schemas times 50 tables (5000 tables)
Information Schema Performance
0 0.5 1 1.5 2 2.5 3 3.5 4
Count All Schemas
Schema aggregate size stats
All Dynamic Table Info
All Static Table Info
Auto Increments Near Limit
Count All Columns
Count All Indexes
MySQL 8.0
MySQL 5.7
Time in Seconds (Lower is better)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA in MySQL 8.0: Dynamic values
• TABLES.TABLE_ROWS, TABLES.DATA_FREE, …
– Handled by opening table and retrieving data from SE in MySQL 5.7 – expensive!!!
• Two approaches in MySQL 8.0
– information_schema_stats=cached
• Default
• Stored in auxiliary tables
• ANALYZE TABLE
– information_schema_stats=latest
• Native functions in the I_S views
• Uses special SE API (InnoDB) or falls back to opening tables (MyISAM)
27
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
INFORMATION SCHEMA longstanding issues solved in 8.0
– Bug#34921 comparison with information schema tables don’t honor collation
– Bug#48445 Inconsistency with SHOW and SELECT FROM I_S
– Bug#61846 SHOW FULL COLUMNS displays incorrect privileges for table
– Bug#65121 Inconsistent result for select on INFORMATION_SCHEMA.STATISTICS
– Bug#75532 Join between I_S schema tables is case insensitive/returns wrong value
– Bug#81347 unnecessary scanned all databases for information schema
– .....
28
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability – Atomic DDL
• All metadata stored in InnoDB (transactional SE)
• DDL code changed to avoid intermediate commits
– At server level
– At InnoDB level
– At replication level
29
Transactional behavior
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability – Atomic DDL : Use Case - DROP DATABASE
30
MySQL 5.7
• Delete tables
– Metadata, TRN/TRG/FRM files
– Data, InnoDB tables
• Delete stored programs
– Metadata, rows in MyISAM (non-
transactional)
• Delete schema
– Metadata, DB.OPT file
Mix of filesystem, non-
transactional/transactional
storage and multiple commits
MySQL 8.0
• Delete tables
– Metadata, rows in InnoDB
– Data, InnoDB tables
• Delete stored programs
– Metadata, rows in InnoDB
• Delete schema
– Metadata, rows in InnoDB
Updates to transactional storage,
one commit
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability – Disaster Recovery
• MySQL 8.0, metadata and data stored in InnoDB (transactional SE)
– Metadata copy stored in SDI embedded in .ibd
– Data stored in .ibd
• MySQL before 8.0, metadata in files, data in InnoDB
– Metadata stored in .FRM
– Data stored in .ibd
31
Use case – IMPORT using Serialized Dictionary Information, InnoDB
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
MySQL before 8.0
• Moving .FRM and data files around (MyISAM)
• Create tables and do import using SQL (InnoDB)
CREATE TABLE t1 (...)
ALTER TABLE t1 DISCARD TABLESPACE
ALTER TABLE t1 IMPORT TABLESPACE ...
• Easy to do mistakes
MySQL 8.0
• Self-descriptive tablespaces (SDI)
• New IMPORT statement will import
the tablespace:
– Read metadata from the SDI, and
create the tables
– Import the data into these tables
• Worst case, if SDI is corrupt, you
can extract it (JSON format) and
edit
32
Reliability – Disaster Recovery
• Use case – IMPORT using Serialized Dictionary Information
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Reliability and Ease of Use – Automated Dictionary Upgrade
• The Data Dictionary is versioned
• The MySQL Server will support upgrading dictionary tables automatically
– Done directly by the executable and not in a script
– Reduces user and privilege issues for the process doing the upgrade
• The Data Dictionary will be upgraded atomically
33
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Extending Dictionary Tables from Plugins
• API for plugins to extend INFORMATION_SCHEMA &
PERFORMANCE_SCHEMA
– Add new virtual tables/views
• API for plugins to store their specific data in the Data Dictionary
34
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Try for yourself!
• Downloadable 8.0.1 DMR
– dev.mysql.com
• A «hot» labs release right off the press: 8.0.1 + Unified Data Dictionary
• Enjoy and give us your feedback!
• Thank you for listening
• http://mysqlserverteam.com
35

Weitere ähnliche Inhalte

Was ist angesagt?

Percona Xtrabackup - Highly Efficient Backups
Percona Xtrabackup - Highly Efficient BackupsPercona Xtrabackup - Highly Efficient Backups
Percona Xtrabackup - Highly Efficient BackupsMydbops
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterKenny Gryp
 
Running MariaDB in multiple data centers
Running MariaDB in multiple data centersRunning MariaDB in multiple data centers
Running MariaDB in multiple data centersMariaDB plc
 
Using The Mysql Binary Log As A Change Stream
Using The Mysql Binary Log As A Change StreamUsing The Mysql Binary Log As A Change Stream
Using The Mysql Binary Log As A Change StreamLuís Soares
 
MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바NeoClova
 
MySQL Parallel Replication by Booking.com
MySQL Parallel Replication by Booking.comMySQL Parallel Replication by Booking.com
MySQL Parallel Replication by Booking.comJean-François Gagné
 
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL ShellMySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL ShellMiguel Araújo
 
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLMySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLOlivier DASINI
 
Oracle RAC on Engineered Systems
Oracle RAC on Engineered SystemsOracle RAC on Engineered Systems
Oracle RAC on Engineered SystemsMarkus Michalewicz
 
MySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitationsMySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitationsJean-François Gagné
 
M|18 Deep Dive: InnoDB Transactions and Write Paths
M|18 Deep Dive: InnoDB Transactions and Write PathsM|18 Deep Dive: InnoDB Transactions and Write Paths
M|18 Deep Dive: InnoDB Transactions and Write PathsMariaDB plc
 
Load Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesLoad Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesSeveralnines
 
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)Jean-François Gagné
 
MySQL Shell for Database Engineers
MySQL Shell for Database EngineersMySQL Shell for Database Engineers
MySQL Shell for Database EngineersMydbops
 
Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!Ted Wennmark
 
Fine-tuning Group Replication for Performance
Fine-tuning Group Replication for PerformanceFine-tuning Group Replication for Performance
Fine-tuning Group Replication for PerformanceVitor Oliveira
 
MariaDB MaxScale
MariaDB MaxScaleMariaDB MaxScale
MariaDB MaxScaleMariaDB plc
 
MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바NeoClova
 

Was ist angesagt? (20)

Percona Xtrabackup - Highly Efficient Backups
Percona Xtrabackup - Highly Efficient BackupsPercona Xtrabackup - Highly Efficient Backups
Percona Xtrabackup - Highly Efficient Backups
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & ClusterMySQL Database Architectures - InnoDB ReplicaSet & Cluster
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
 
Running MariaDB in multiple data centers
Running MariaDB in multiple data centersRunning MariaDB in multiple data centers
Running MariaDB in multiple data centers
 
Using The Mysql Binary Log As A Change Stream
Using The Mysql Binary Log As A Change StreamUsing The Mysql Binary Log As A Change Stream
Using The Mysql Binary Log As A Change Stream
 
MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바MySQL Administrator 2021 - 네오클로바
MySQL Administrator 2021 - 네오클로바
 
MySQL Parallel Replication by Booking.com
MySQL Parallel Replication by Booking.comMySQL Parallel Replication by Booking.com
MySQL Parallel Replication by Booking.com
 
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL ShellMySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL Shell
 
InnoDb Vs NDB Cluster
InnoDb Vs NDB ClusterInnoDb Vs NDB Cluster
InnoDb Vs NDB Cluster
 
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQLMySQL InnoDB Cluster - A complete High Availability solution for MySQL
MySQL InnoDB Cluster - A complete High Availability solution for MySQL
 
Oracle RAC on Engineered Systems
Oracle RAC on Engineered SystemsOracle RAC on Engineered Systems
Oracle RAC on Engineered Systems
 
MySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitationsMySQL Parallel Replication: inventory, use-case and limitations
MySQL Parallel Replication: inventory, use-case and limitations
 
M|18 Deep Dive: InnoDB Transactions and Write Paths
M|18 Deep Dive: InnoDB Transactions and Write PathsM|18 Deep Dive: InnoDB Transactions and Write Paths
M|18 Deep Dive: InnoDB Transactions and Write Paths
 
Load Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesLoad Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - Slides
 
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
MySQL Parallel Replication: All the 5.7 and 8.0 Details (LOGICAL_CLOCK)
 
MySQL Shell for DBAs
MySQL Shell for DBAsMySQL Shell for DBAs
MySQL Shell for DBAs
 
MySQL Shell for Database Engineers
MySQL Shell for Database EngineersMySQL Shell for Database Engineers
MySQL Shell for Database Engineers
 
Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!Upgrade to MySQL 8.0!
Upgrade to MySQL 8.0!
 
Fine-tuning Group Replication for Performance
Fine-tuning Group Replication for PerformanceFine-tuning Group Replication for Performance
Fine-tuning Group Replication for Performance
 
MariaDB MaxScale
MariaDB MaxScaleMariaDB MaxScale
MariaDB MaxScale
 
MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바MySQL Advanced Administrator 2021 - 네오클로바
MySQL Advanced Administrator 2021 - 네오클로바
 

Ähnlich wie Data dictionary pl17

MySQL 8.0 in a nutshell
MySQL 8.0 in a nutshellMySQL 8.0 in a nutshell
MySQL 8.0 in a nutshellOracleMySQL
 
State ofdolphin short
State ofdolphin shortState ofdolphin short
State ofdolphin shortMandy Ang
 
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)OracleMySQL
 
What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017Ivan Ma
 
MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?Olivier DASINI
 
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...OracleMySQL
 
New data dictionary an internal server api that matters
New data dictionary an internal server api that mattersNew data dictionary an internal server api that matters
New data dictionary an internal server api that mattersAlexander Nozdrin
 
MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015Mario Beck
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 Geir Høydalsvik
 
How to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performanceHow to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performanceoysteing
 
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.Cloud Native Day Tel Aviv
 
Dd and atomic ddl pl17 dublin
Dd and atomic ddl pl17 dublinDd and atomic ddl pl17 dublin
Dd and atomic ddl pl17 dublinStåle Deraas
 
What's New in MySQL 5.7
What's New in MySQL 5.7What's New in MySQL 5.7
What's New in MySQL 5.7Olivier DASINI
 
Mysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesMysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesTarique Saleem
 
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL SupportMysql User Camp
 
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR AnalyticsCedar Consulting
 
MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018Olivier DASINI
 
MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017MySQL Brasil
 
Unlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQLUnlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQLRicky Setyawan
 

Ähnlich wie Data dictionary pl17 (20)

MySQL 8.0 in a nutshell
MySQL 8.0 in a nutshellMySQL 8.0 in a nutshell
MySQL 8.0 in a nutshell
 
State ofdolphin short
State ofdolphin shortState ofdolphin short
State ofdolphin short
 
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
MySQL in Oracle environment : Quick start guide for Oracle DBA (Part 1)
 
What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017
 
MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?MySQL 8.0 - What's New ?
MySQL 8.0 - What's New ?
 
Novinky v Oracle Database 18c
Novinky v Oracle Database 18cNovinky v Oracle Database 18c
Novinky v Oracle Database 18c
 
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
MySQL in oracle_environments(Part 2): MySQL Enterprise Monitor & Oracle Enter...
 
New data dictionary an internal server api that matters
New data dictionary an internal server api that mattersNew data dictionary an internal server api that matters
New data dictionary an internal server api that matters
 
MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015MySQL 5.7: What's New, Nov. 2015
MySQL 5.7: What's New, Nov. 2015
 
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
 
How to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performanceHow to analyze and tune sql queries for better performance
How to analyze and tune sql queries for better performance
 
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
MySQL Shell: the daily tool for devs and admins. By Vittorio Cioe.
 
Dd and atomic ddl pl17 dublin
Dd and atomic ddl pl17 dublinDd and atomic ddl pl17 dublin
Dd and atomic ddl pl17 dublin
 
What's New in MySQL 5.7
What's New in MySQL 5.7What's New in MySQL 5.7
What's New in MySQL 5.7
 
Mysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesMysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New Features
 
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
18. Madhur Hemnani - Result Orientated Innovation with Oracle HR Analytics
 
MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018MySQL 8.0, what's new ? - Forum PHP 2018
MySQL 8.0, what's new ? - Forum PHP 2018
 
MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017MySQL como Document Store PHP Conference 2017
MySQL como Document Store PHP Conference 2017
 
Unlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQLUnlocking big data with Hadoop + MySQL
Unlocking big data with Hadoop + MySQL
 

Kürzlich hochgeladen

Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 

Kürzlich hochgeladen (20)

Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 

Data dictionary pl17

  • 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0: An Internal Server ComponentThat Matters Ståle Deraas, Senior Development Manager Oracle, MySQL 27 Apr 2017 Copyright © 2017, Oracle and/or its affiliates. All rights reserved.
  • 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  • 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Agenda What is a Data Dictionary? Data Dictionary before MySQL 8.0 Transactional Data Dictionary in MySQL 8.0 What’s in it for you? 1 2 3 4
  • 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | What is a Data Dictionary?
  • 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Data Dictionary • Metadata is information about data in an RDBMS – Column definitions, Index definitions, Foreign key definitions ... • Data Dictionary is a collection of metadata for all data in an RDBMS ID NAME WEIGHT HEIGHT GENDER 3 Bob 80 185 M 5 Liz 55 165 F Metadata Data
  • 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Server Data Dictionary 6 Role in Server Query Executor Optimizer Storage Engine SQL statement Client Parser Result Data Dictionary
  • 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | CREATE PROCEDURE p1(v INT) SQL SECURITY INVOKER BEGIN ... END Data Dictionary Types of Metadata Data Dictionary Table Definitions SP Definitions View Definitions Schemas ACL CREATE TABLE customers( id INT AUTO_INCREMENT ... PRIMARY KEY (id), INDEX ... FOREIGN KEY ... )
  • 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Data Dictionary before MySQL 8.0
  • 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Data Dictionary before MySQL 8.0 • Metadata stored in a mix of files and tables : – File based • FRM, TRN, TRG, OPT, PAR ... – Table based: non-transactional • mysql.proc ... – Table based: transactional • Innodb.SYS
  • 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Data Dictionary before MySQL 8.0 10 Data Dictionary Files FRM TRG OPT System Tables (mysql.) user procevents InnoDB System Tables MyISAM File system InnoDB SQL
  • 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL Data Dictionary before MySQL 8.0 • INFORMATION_SCHEMA queries are slow • Inconsistencies due to non-transactional storage of metadata • Inconsistencies between InnoDB metadata and Server metadata • Showstopper for crash-safe / transactional DDL • Replication is challenging, as DDLs are not atomic • Difficult to extend • No uniform API 11 Problems
  • 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary MySQL 8.0
  • 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 13 Data Dictionary InnoDB SQL DD TableDD TableDD Table
  • 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 • All metadata stored in tables • Single metadata repository for all MySQL server subsystems • Reliable, crash-safe InnoDB tables • INFORMATION_SCHEMA implemented as views over DD tables – Queries can be optimized – Improved performance – Simpler, uniform implementation, easier to maintain Main features
  • 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 15 Overview InnoDB Data Dictionary DD Table User Table INFORMATION SCHEMA Views Archive User Table CSV User Table Optimizer
  • 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 • The data dictionary schema is based on SQL standard definitions • Designed for automated upgrade of metadata • The data dictionary is designed to be easily extended for new requirements • Designed to allow plugins/components to add INFORMATION_SCHEMA views and PERFORMANCE_SCHEMA tables Main features, cont’ed
  • 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | InnoDB Transactional Data Dictionary in MySQL 8.0 17 Architecture Query Executor Parser Optimizer Data Dictionary Tablespace Data Dictionary API Storage Engine PluginUser TableUser Table
  • 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary API in MySQL 8.0 • A single way to deal with Data Dictionary – For the server core – For Storage Engines • A uniform API for all types of metadata • Provide a way to handle Storage Engine private data – Storage Engines no longer have to store their own metadata – The API enables SEs to store key-value pairs 18 Design goals
  • 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 SDI: Serialized Dictionary Information - Copy of metadata from the Data Dictionary in JSON format - Stored in data tablespaces(InnoDB) or .sdi file (MyISAM) - Used for data migration and redundancy The InnoDB Data Dictionary tablespace is the metadata storage 19 Reliability and Redundancy InnoDB User Tablespaces User Table InnoDB Data Dictionary tablespace mysql.tables SDI ID Name 1 User Table 1 2 User Table 2
  • 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary in MySQL 8.0 What’s in it for you?
  • 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Data Dictionary Benefits • INFORMATION SCHEMA – Improved Performance and Scalability – Solves longstanding issues • Reliability – Atomic DDL – Serialized Dictionary Information – Automated Data Dictionary upgrade • +++ 21
  • 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22 MySQLClient I_S Query Results MySQL Server Create temporary table. Heuristic optimization Read metadata from File system or from MyISAM/InnoDB engine. . TEMP TABLE Return rows to user. File system / MyISAM / InnoDB engine INFORMATION SCHEMA MySQL 5.7: Multiple sources, heuristics, creation of temp tables
  • 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 23 MySQL Client I_S Query Results MySQL Server Optimizer prepares execution plan Executor reads metadata from data dictionary tables InnoDB storage engine Return rows to user INFORMATION SCHEMA MySQL 8.0: Uniform, simple and using server standard features
  • 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA Performance Confidential – Oracle Internal/Restricted/Highly Restricted 24 Uniform, simpler implemention makes it a lot faster MySQL Client I_S Query Results MySQL Server Optimizer prepares execution plan. Executor reads metadata from data dictionary tables. InnoDB storage engine Return rows to user. INFORMATION_SCHEMA in 8.0 MySQL Client I_S Query Results MySQL Server Create temporary table. Heuristic optimization. Read metadata from File system or from MyISAM/InnoDB engine. . TEMP TABLE Return rows to user. INFORMATION_SCHEMA in 5.7 File system / MyISAM / InnoDB engine
  • 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA Performance and Scalability • Typically 30X performance improvements over MySQL 5.7 • More than 100X for some queries like: List all InnoDB table columns 25 I_S queries scale, both with database size and query load 0 20 40 60 80 100 120 140 160 List all InnoDB tables columns 5k tables List all InnoDB tables columns 10k tables MySQL 8.0 MySQL 5.7 Time in Seconds (Lower is better)
  • 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 26 100 schemas times 50 tables (5000 tables) Information Schema Performance 0 0.5 1 1.5 2 2.5 3 3.5 4 Count All Schemas Schema aggregate size stats All Dynamic Table Info All Static Table Info Auto Increments Near Limit Count All Columns Count All Indexes MySQL 8.0 MySQL 5.7 Time in Seconds (Lower is better)
  • 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA in MySQL 8.0: Dynamic values • TABLES.TABLE_ROWS, TABLES.DATA_FREE, … – Handled by opening table and retrieving data from SE in MySQL 5.7 – expensive!!! • Two approaches in MySQL 8.0 – information_schema_stats=cached • Default • Stored in auxiliary tables • ANALYZE TABLE – information_schema_stats=latest • Native functions in the I_S views • Uses special SE API (InnoDB) or falls back to opening tables (MyISAM) 27
  • 28. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | INFORMATION SCHEMA longstanding issues solved in 8.0 – Bug#34921 comparison with information schema tables don’t honor collation – Bug#48445 Inconsistency with SHOW and SELECT FROM I_S – Bug#61846 SHOW FULL COLUMNS displays incorrect privileges for table – Bug#65121 Inconsistent result for select on INFORMATION_SCHEMA.STATISTICS – Bug#75532 Join between I_S schema tables is case insensitive/returns wrong value – Bug#81347 unnecessary scanned all databases for information schema – ..... 28
  • 29. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability – Atomic DDL • All metadata stored in InnoDB (transactional SE) • DDL code changed to avoid intermediate commits – At server level – At InnoDB level – At replication level 29 Transactional behavior
  • 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability – Atomic DDL : Use Case - DROP DATABASE 30 MySQL 5.7 • Delete tables – Metadata, TRN/TRG/FRM files – Data, InnoDB tables • Delete stored programs – Metadata, rows in MyISAM (non- transactional) • Delete schema – Metadata, DB.OPT file Mix of filesystem, non- transactional/transactional storage and multiple commits MySQL 8.0 • Delete tables – Metadata, rows in InnoDB – Data, InnoDB tables • Delete stored programs – Metadata, rows in InnoDB • Delete schema – Metadata, rows in InnoDB Updates to transactional storage, one commit
  • 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability – Disaster Recovery • MySQL 8.0, metadata and data stored in InnoDB (transactional SE) – Metadata copy stored in SDI embedded in .ibd – Data stored in .ibd • MySQL before 8.0, metadata in files, data in InnoDB – Metadata stored in .FRM – Data stored in .ibd 31 Use case – IMPORT using Serialized Dictionary Information, InnoDB
  • 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | MySQL before 8.0 • Moving .FRM and data files around (MyISAM) • Create tables and do import using SQL (InnoDB) CREATE TABLE t1 (...) ALTER TABLE t1 DISCARD TABLESPACE ALTER TABLE t1 IMPORT TABLESPACE ... • Easy to do mistakes MySQL 8.0 • Self-descriptive tablespaces (SDI) • New IMPORT statement will import the tablespace: – Read metadata from the SDI, and create the tables – Import the data into these tables • Worst case, if SDI is corrupt, you can extract it (JSON format) and edit 32 Reliability – Disaster Recovery • Use case – IMPORT using Serialized Dictionary Information
  • 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reliability and Ease of Use – Automated Dictionary Upgrade • The Data Dictionary is versioned • The MySQL Server will support upgrading dictionary tables automatically – Done directly by the executable and not in a script – Reduces user and privilege issues for the process doing the upgrade • The Data Dictionary will be upgraded atomically 33
  • 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Extending Dictionary Tables from Plugins • API for plugins to extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA – Add new virtual tables/views • API for plugins to store their specific data in the Data Dictionary 34
  • 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Try for yourself! • Downloadable 8.0.1 DMR – dev.mysql.com • A «hot» labs release right off the press: 8.0.1 + Unified Data Dictionary • Enjoy and give us your feedback! • Thank you for listening • http://mysqlserverteam.com 35