Overview and basics of MySQL Database

1. MySQL Database
MySQL 5.7
1
Lalit Choudhary

2. Agenda
2
1 MySQL History
2 MySQL Architecture
3 MySQL Installation
4 MySQL Storage engines and configuration
5 MySQL Administration and Monitoring
6 MySQL Replication
7 MySQL backup and restore
8 Upgrade and Downgrade

3. MySQL History
 The initial release in 1995 had a SQL interface and a dual license model, a free and an embedded
version. David Axmark, Monty and Allen Larrson founded MySQL AB in 1995, it was taken over by
Sun Microsystems in 2008 and Sun itself was taken by Oracle in 2010
 MyISAM engine
 InnoDB transactional engine
 MySQL 5.0 store procedure, triggers etc.
 MySQL 5.1 row-based replication, partitioning
 MySQL development boots in oracle, Including adding and improving existing features.
 MySQL comes with different choices depending on your needs.
- Community Server (GPL License)
- Standard Edition
- Enterprise Edition
3

4. MySQL Architecture
• Physical view of the MySQL (5.7) server architecture
Directory location will be different as per OS and type of installation.
4

5. MySQL Architecture
• Physical view of the MySQL (5.7) server architecture
Data directory
• Server log files
• Status file
• Innodb log files
• Innodb system tablespace
• Innodb log buffer
• Innodb tmp tablespace
Program executable files
Data sub-directory (Per Database)
• Data and index files (.ibd)
• Object structure files(.frm,.opt)
Program log files
5

6. 6
 Configuration files:
◦ data_dir/auto.cnf : Contains server_uuid
◦ /etc/my.cnf : MySQL Configuration file.
 Misc:
• --basedir=dir_name
◦ The path to the MySQL installation directory.
• --datadir=dir_name
◦ The path to the MySQL data directory.
• --pid-file=file_name
◦ The path name of the file in which the server should write its process ID.
• --socket=file_name, -S file_name
◦ On Unix, the name of the Unix socket file to use, for connections made using a named pipe to a local server.
• --log-error=file_name
◦ Log errors and startup messages to this file.
MySQL Architecture

7. MySQL Architecture
• Logical view of the MySQL server architecture
PLUGIN Connectors : ODBC, JDBC, NET,PHP, Python etc.
Connections/Thread handling
Query Cache
Parser
Optimizer
Table Metadata
Cache
Keys Cache
[MyISAM]
inInnoDB MyISAM NDB MEMORY ……
SERVER
Storage Engines
CLIENT
7
[ Storage Engines
buffer/s]

8. MySQL Installation
 Ways to install MySQL
• Source code
• Binaries : ref http://dev.mysql.com/doc/refman/5.7/en/binary-installation.html
• Packages : Ref: http://dev.mysql.com/doc/refman/5.7/en/linux-installation-rpm.html
• MySQL Installer MSI and ZIP Archive
• Yum repository
 Configuration
• Storage engine
• Server Variables
• User management and Access control
8

9. Storage Engines
InnoDB:
• Fully transactional ACID.
• Offers REDO and UNDO for transactions.
• Data storage in tablespace :
 Multiple data files
 Logical object structure using innodb data and log buffer
• Row level locking.
9

10. InnoDB Architecture
10
Log files
Tablespace
MEMORY DISK
Innodb_buffer_pool
[Buffered data
pages,MVCC,UNDO,etc.]
Innodb_log_buffer
[Buffered logs record]
[tablespace IDs and page IDs]
TABLESPACE
ibdata1
ibdata2
ib_logfile0
ib_logfile1
ib_logfile2
REDO
LOG
FILES
Checkpoint
Checkpoint
Commit
Other Memory pools
Ib_buffer_pool file
R
e
c
o
v
e
r
y

11. MySQL Architecture
InnoDB
Ref: http://dev.mysql.com/doc/refman/5.7/en/innodb-storage-engine.html
 Tablespace : ibdata
◦ The file ibdata is the system tablespace for the InnoDB engine. Apart from the table data storage, InnoDB's functionality
requires looking for table metadata and storing and retrieving MVCC info to support ACID compliance and Transaction
Isolation. It contains several types for information for InnoDB objects.
• Table Data Pages
• Table Index Pages
• Data Dictionary
• MVCC Control Data
• Undo Space
• Rollback Segments
• Double Write Buffer (Pages Written in the Background to avoid OS caching)
• Insert Buffer (Changes to Secondary Indexes)
◦ Example:
innodb_data_file_path = /ibdata/ibdata1:10M:autoextend
innodb_data_file_path = /ibdata/ibdata1:988M;/disk2/ibdata2:50M:autoextend
11

12. MySQL Architecture
 Innodb redo log file : ib_logfile
◦ It will read across ib_logfile0 and ib_logfile1 to check for any data changes that were not posted to the double
write buffer in ibdata1. It will replay (redo) those changes. Once they are replayed and stored, mysqld becomes
ready for new DB Connections.
◦ Example :Innodb_log_file_in_group=3
Innodb_log_file_size=150M
 Innodb buffer pool file : ib_buffer_pool
◦ At the time of mysqld startup to reduce the warmup period after restarting the server, it loads few percent of most
recently used pages for each buffer pool at server shutdown and this data i.e. tablespace ID and page ID data
information is derived from the INNODB_BUFFER_PAGE_LRU INFORMATION_SCHEMA table. Tablespace ID and
page ID data is saved in a file named ib_buffer_pool under InnoDB data directory.
◦ The file name and location can be modified using the innodb_buffer_pool_filename configuration parameter.
12

13. MySQL Architecture
 Transactional logs: Binary logs
◦ Variable name : --log-bin
◦ Enable binary logging. The server logs all statements that change data to the binary log, which is used for backup
and replication.
◦ Using mysqlbinlog client tool we can extract and view binlog contents in text format.
13

14. Storage Engines
NDB:
• Fully Transactional and ACID Storage engine.
• Distribution execution of data and using multiple mysqld.
• NDB use logical data with own buffer for each NDB engine.
• Offers REDO and UNDO for transactions.
• Row level locking.
14

15. Storage Engines
MyISAM:
• Non transactional storage engine
• Speed for read
• Data storage in files and use key ,metadata and query cache
 .FRM for table structure
 .MYI for table index
 .MYD for table data
• Table level locking.
15

16. Storage Engines
Memory:
• Non transactional storage engine
• All data store in memory other then table metadata and structure.
• Table level locking.
16

17. Storage Engines
Archive:
• Non transactional storage engine,
• Store large amounts of compressed and unindexed data.
• Allow INSERT, REPLACE, and SELECT, but not DELETE or UPDATE sql operations.
• Table level locking.
17

18. Storage Engines
CSV:
• Stores data in flat files using comma-separated values format.
• Table structure needs be create within MySQL server (.frm)
18

19. MySQL Production ready configuration
 Hardware
• Storage and file types : SSD/HDD and EXT4 or XFS on Linux
• Memory : Optimal performance, Large transaction ,indexes, faster response time for ongoing changes and utilize Disk
IO
• CPU : Faster processors with many cores provide better throughput
 MySQL configuration
◦ innodb_file_per_table
◦ innodb_buffer_pool_size
◦ innodb_buffer_pool_instances
◦ innodb_log_file_size
◦ innodb_data_file_path
◦ Innodb_log_file_in_group
◦ innodb_flush_log_at_trx_commit
◦ innodb_thread_concurrency
◦ innodb_flush_method
19

20. MySQL Administration and Monitoring
 MySQL administrative tools
• Mysql : Login, database access/manipulation ,
• Mysqladmin : Administration purpose
• Mysqldump / mysqlpump : Logical backup utility
• Mysqlbinlog : Utility for processing binary log files and display binlog contents in text format.
• Mysql enterprise backup : Provides enterprise-grade backup and recovery for MySQL.
• Percona xtrabackup : Open source tool for mysql binary backup.
Ref: http://dev.mysql.com/doc/refman/5.7/en/programs-overview.html
 GUI tools
• MySQL enterprise monitor
• MySQL workbench/ SQLyog etc.
20

21. MySQL Administration and Monitoring
 Monitoring
• MySQL enterprise monitor : MySQL Enterprise Monitor provides real-time visibility into the performance and
availability of all your MySQL databases.
• Metadata and : From Information schema and mysql database.
• DB statistic: performance_schema databases we can get ongoing as well as historical database activities.
• SHOW PROCESSLIST
• Mysql utilities
• Audit logs & General logs, slow query logs
• Third-party tools : Percona tool kit , Nagios , cacti etc
21

22. MySQL Replication
Master Slave
● server_id
● log_bin
● binlog_format
● binlog-do-db
● binlog-ignore-db
● GTID (optional)
• server_id
• log_bin
• binlog_format
• log_slave_updates
• relay_log=file_name
• replicate-do-db=db_name
• replicate-ignore-db=db_name
ON MASTER:
Created replication user and grant privileges:
mysql> CREATE USER 'repl'@'%.mydomain.com' IDENTIFIED BY 'slavepass';
mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%.mydomain.com';
Mysql> SHOW MASTER STATUS;
ON SLAVE:
mysql> CHANGE MASTER TO MASTER_HOST='master_host_name',MASTER_USER='replication_user_name',
MASTER_PASSWORD='replication_password',MASTER_LOG_FILE=‘master_log_file_name',
MASTER_LOG_POS=master_log_position;
22

23. MySQL Replication
23

24. MySQL Replication
 Best practices
− Use GTID enabled replication
◦ log_bin
◦ log-slave-update
◦ expire_log_days
◦ gtid_mode
◦ Enforce-gtid-consistency
− Crash-safe slaves
◦ master-info-repository=TABLE
◦ Relay-log-info-repository=TABLE
◦ Relay-log-recovery
− Secure slave
◦ read-only
◦ Skip-slave-start
− Avoid replication Lag
◦ slave_compressed_protocol
◦ MTR replication
24

25. MySQL Replication
 Without GTID
◦ Need Master_Log_File and Master_Log_Pos
 GTID
− What problems GTID solves?
◦ It is possible to identify a transaction uniquely across the replication servers. Make the automation of failover process
much easier. There is no need to do calculations, inspect the binary log and so on. Just
MASTER_AUTO_POSITION=1.
− Enable GTID replication
◦ gtid_mode: It can be ON or OFF (not 1 or 0). It enables the GTID on the server.
◦ log_bin: Enable binary logs. Mandatory to create a replication environment.
◦ log-slave-updates: Slave servers must log the changes that comes from the master in its own binary log.
◦ Enforce-gtid-consistency : Statements that can’t be logged in a transactionally safe manner are denied by the server.
25

26. Backup and Restore
 Backup
Binary Backup
− MySQL Enterprise Backup
Logical Backup
− mysqldump / mysqlpump
eg. mysqldump [options] > dump.sql
Hot Backup
− Replication slave
Customized scripts
26

27. Backup and Restore
 Restore
Logical backup restore using mysql client
eg. mysql [options] < dump.sql
Binary backup restore : Replace old binaries or package and run mysql_upgrade script.
27

28. MySQL Upgrade and Downgrade
MySQL Upgrade
Supported upgrade methods include:
In-Place Upgrade: Involves shutting down the old MySQL version, replacing the old MySQL binaries or packages with the new
ones, restarting MySQL on the existing data directory, and running mysql_upgrade.
Logical Upgrade: Involves exporting existing data from the old MySQL version using mysqldump, installing the new MySQL
version, loading the dump file into the new MySQL version, and running mysql_upgrade.
MySQL Downgrade
Supported downgrade methods include:
In-Place Downgrade: Involves shutting down the new MySQL version, replacing the new MySQL binaries or packages with the
old ones, and restarting the old MySQL version on the existing data directory. In-place downgrades are supported for
downgrades between GA versions within the same release series. For example, in-place downgrades are supported for
downgrades from 5.7.10 to 5.7.9.
Logical Downgrade: Involves using mysqldump to dump all tables from the new MySQL version, and then loading the dump
file into the old MySQL version. Logical downgrades are supported for downgrades between GA versions within the same
release series and for downgrades between release levels. For example, logical downgrades are supported for downgrades
from 5.7.10 to 5.7.9 and for downgrades from 5.7 to 5.6.
28

29. Additional Resources
MySQL Tutorial: http://dev.mysql.com/doc/refman/5.7/en/tutorial.html
MySQL Administration: http://dev.mysql.com/doc/refman/5.7/en/server-administration.html
Blogs: https://planet.mysql.com/
29

30. THANK YOU
30