[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)

Title

EffectiveMySQL.com - Its all about Performance and Scalability

Wednesday, October 19, 2011

Title

Mastering MySQL
Performance Tuning
Ronald Bradford
http://ronaldbradford.com
Insight Out DB Showcase
October 2011



OBJECTIVE

Identify key MySQL instrumentation
Detail the MySQL terms to know and use
Provide references to further materials



ABOUT AUTHOR
Ronald BRADFORD

12 years with MySQL / 22 years with RDBMS
Senior Consultant at MySQL Inc (06-08)
Consultant for Oracle Corporation (96-99)

7 years presenting MySQL content
All time top MySQL blogger
Available NOW
Published author for consulting

Oracle ACE Director
http://RonaldBradford.com



MySQL
Instrumentation



Key sources

Conﬁguration
my.cnf (e.g. /etc/my.cnf, /etc/mysql/my.cnf)
SHOW [SESSION|GLOBAL] VARIABLES
INFORMATION_SCHEMA tables

Status
SHOW [SESSION|GLOBAL] STATUS
INFORMATION_SCHEMA tables



STATUS
SHOW STATUS EXAMPLE
INFORMATION_SCHEMA.STATUS Example



Key Sources

Running Threads
SHOW FULL PROCESSLIST
INFORMATION_SCHEMA.PROCESSLIST

Storage Engines
SHOW ENGINE INNODB STATUS

Performance Schema
PERFORMANCE_SCHEMA


PROCESSLIST
PROCESSLIST EXAMPLE
INFORMATION_SCHEMA.PROCESSLIST Example



KEY areas

System performance
SQL performance



Know your system
bottleneck



BOTTLENECK

Memory
Disk
CPU
Network



MEMORY

Basic conﬁguration (Global use)
Basic conﬁguration (Thread use)
Memory Tables
Thread Temporary tables

http://dev.mysql.com/doc/refman/5.5/en/memory-use.html



Memory

Global Buffer conﬁguration
innodb_buffer_pool_size
key_buffer_size
query_cache_size



InnoDB Size
EXAMPLE

Was 1 GB, should be 300M+
AWS small instance with 1.7G RAM
mysql> source sql/innodb_size.sql
+-------------------+----------+---------+----------+
| title | total_mb | data_mb | index_mb |
+-------------------+----------+---------+----------+
| Total InnoDB Size | 278 | 171 | 107 |
+-------------------+----------+---------+----------+



InnoDB Size
EXAMPLE

Was 1 GB, should be 300M+
AWS small instance with 1.7G RAM
mysql> source sql/innodb_size.sql
+-------------------+----------+---------+----------+
+-------------------+----------+---------+----------+
| Total InnoDB Size | 278 | 171 | 107 |
+-------------------+----------+---------+----------+

Rule of 75-80% of RAM is WRONG!



MyISAM Size
EXAMPLE

key_buffer_size
For MyISAM Indexes ONLY
Was default in RDS, should be 700M+
Github
mysql> source sql/myisam_size.sql
+-------------------+----------+---------+----------+
+-------------------+----------+---------+----------+
| Total MyISAM Size | 6140 | 5492 | 648 |
+-------------------+----------+---------+----------+



MyISAM Size
EXAMPLE

key_buffer_size
For MyISAM Indexes ONLY
Was default in RDS, should be 700M+
Github
mysql> source sql/myisam_size.sql
+-------------------+----------+---------+----------+
+-------------------+----------+---------+----------+
| Total MyISAM Size | 6140 | 5492 | 648 |
+-------------------+----------+---------+----------+

Needs 1M min for all InnoDB system



memory

Thread buffer conﬁguration
read_buffer_size
read_rnd_buffer_size
sort_buffer_size
join_buffer_size



memory

In Memory Tables
max_heap_table_size
Internal Temporary Tables
MIN(tmp_table_size,max_heap_table_size)



Memory

mysqld memory
Unbounded
Linux process list (ps)
$ $ ps -eopid,fname,vsz,rss,user,command | grep -e "RSS" -e "mysql"
PID COMMAND VSZ RSS USER COMMAND
13628 grep 3352 820 ubuntu grep -e RSS -e mysql
28082 mysqld 631076 405660 mysql /usr/sbin/mysqld

$ ps -eopid,fname,vsz,rss,user,command | grep " mysqld " | grep -v grep
| awk '{print $3,$4}'
631076 405660

http://ronaldbradford.com/blog/are-you-monitoring-rss-vsz-2009-03-08/



MEMORY

PROCESSLIST
Copying to tmp table



Disk

fsync conﬁguration
flush_log_at_trx_commit
sync_binlog
innodb_file_per_table



MySQL DISK

Status Variables
mysql> show global status like 'innodb%write%';
+-----------------------------------+-----------+ Volumes
| Innodb_buffer_pool_write_requests | 821053284 |
| Innodb_data_pending_writes | 0 |
| Innodb_data_writes | 198400090 |
| Innodb_dblwr_writes | 547847 |
| Innodb_log_write_requests | 64860531 |
| Innodb_log_writes | 188609653 |
| Innodb_os_log_pending_writes | 0 |
+-----------------------------------+-----------+

mysql> show global status like '%sync%';
+------------------------------+---------+ True Costs
| Innodb_data_fsyncs | 6835646 |
| Innodb_data_pending_fsyncs | 0 |
| Innodb_os_log_fsyncs | 5681432 |
| Innodb_os_log_pending_fsyncs | 0 |
+------------------------------+---------+
mysql> show global status like 'created_tmp_disk_tables';
+-------------------------+-------+
| Created_tmp_disk_tables | 87 |
+-------------------------+-------+


CPU

Query sorting (ORDER BY/GROUP BY)
PROCESSLIST
Sorting result
EXPLAIN
Using Filesort



NETWORK

STATUS
Bytes_sent
Bytes_received
PROCESSLIST
Sending data



SQL



SQL performance

Throughput
Scalability is about increasing
throughout
Latency
Performance is about improving
latency



THROUgHPUT

Status
Performance Schema
TCP/IP



MySQL STATUS
EXAMPLE
$ status_now.sh
--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Bytes_received | 142721 |
| Bytes_sent | 85170 | Temporary disk table
| Com_admin_commands | 160 | writes per second
| Com_delete | 4 |
| Com_insert | 12 |
| Com_select | 331 |
| Com_show_status | 1 |
| Com_update | 5 |
| Created_tmp_tables | 53 |
...
| Qcache_inserts | 331 |
| Qcache_hits | 382 |
| Qcache_lowmem_prunes | 319 |
| Qcache_free_memory | -8600 |
...
| Uptime | 1 |
+--------------------------+------------+

https://github.com/ronaldbradford/EffectiveMySQL


MySQL STATUS
EXAMPLE
$ status_now.sh What is happening this second?
--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Bytes_received | 142721 |
| Bytes_sent | 85170 | Temporary disk table
| Com_admin_commands | 160 | writes per second
| Com_delete | 4 |
| Com_insert | 12 |
| Com_select | 331 |
| Com_show_status | 1 |
| Com_update | 5 |
| Created_tmp_tables | 53 |
...
| Qcache_inserts | 331 |
| Qcache_hits | 382 |
| Qcache_lowmem_prunes | 319 |
| Qcache_free_memory | -8600 |
...
| Uptime | 1 |
+--------------------------+------------+

https://github.com/ronaldbradford/EffectiveMySQL


TCP/IP
TCP/IP Example



Latency

EXPLAIN
SHOW CREATE TABLE
SHOW INDEXES



EXPLAin
EXPLAIN Example



LATENCY

Optimizer Trace (Since 5.6.3)
Shows all optimizer actions
Rows statistics
Tree pruning



OPTIMIZER TRACE
Optimizer Trace Example



latency

performance_schema
Introduced in 5.5
Much improved 5.6.3



Performance schema

Performance Schema example



throughput

Repeating queries
Redundant queries
Unnecessary complex queries



throughput

MySQL Query Cache
Great for high read
Poor for read/write



The art of doing more
starts with doing less



MYSQL STATUS
EXAMPLE
====================================================================================================
Uptime: 12 hours 17 mins Snapshot Period 1: 1 minute interval
====================================================================================================
Variable Delta/Percentage Per Second Total
Why are you doing crazy
====================================================================================================
Statement Activity
things Mr Framework?
====================================================================================================

SELECT: 16,042 267.37 8,177,050 (46.03%)
INSERT: 5,838 97.30 1,826,616 (10.28%)
UPDATE: 1,109 18.48 738,546 (4.16%)
DELETE: 2,018 33.63 1,374,983 (7.74%)
REPLACE: 0 0.00 0 (0.00%)
INSERT ... SELECT: 0 0.00 27 (0.00%)
REPLACE ... SELECT: 0 0.00 0 (0.00%)
Multi UPDATE: 0 0.00 0 (0.00%)
Multi DELETE: 0 0.00 0 (0.00%)
COMMIT: 5,708 95.13 2,161,232 (12.17%)
ROLLBACK: 5,746 95.77 3,485,828 (19.62%)



Row Processing
EXAMPLE
SELECT name FROM firms WHERE id=727;
SELECT name FROM firms WHERE id=740; SELECT id, name
SELECT name FROM firms WHERE id=849; FROM firms
SELECT name FROM firms WHERE id=839; WHERE id IN (723, 727, 740, 758, 812, 829, 839,
SELECT name FROM firms WHERE id=847; 847, 849, 857, 867, 868);

http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/
http://ronaldbradford.com/blog/simple-lessons-in-improving-scalability-2011-02-16/
http://ronaldbradford.com/blog/the-rat-and-the-cat-2006-08-24/



Row Processing
EXAMPLE
SELECT name FROM firms WHERE id=727; Classic N+1 problem
SELECT name FROM firms WHERE id=740; SELECT id, name
SELECT name FROM firms WHERE id=849; FROM firms
SELECT name FROM firms WHERE id=839; WHERE id IN (723, 727, 740, 758, 812, 829, 839,
SELECT name FROM firms WHERE id=847; 847, 849, 857, 867, 868);

http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/
http://ronaldbradford.com/blog/simple-lessons-in-improving-scalability-2011-02-16/
http://ronaldbradford.com/blog/the-rat-and-the-cat-2006-08-24/



Row Processing
EXAMPLE
SET PROFILING=1;
SELECT … 4X longer processing for
SHOW PROFILES; every page load
+----------+------------+---------------------------------------------------------
| Query_ID | Duration | Query
+----------+------------+---------------------------------------------------------
| 1 | 0.00030400 | SELECT name FROM firms WHERE id=727
SELECT 'Sum Individual Queries' AS txt,SUM(DURATI
INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID BETWE
UNION
| 4 | 0.00014000 | SELECT name FROM firms WHERE id=740Query',SUM(DURATION) FROM INFORM
SELECT 'Combined
| 5 | 0.00012300 | SELECT name FROM firms WHERE 13;
QUERY_ID = id=849
+------------------------+------------+
| txt | total_time |
+------------------------+------------+
| 9 | 0.00013900 | SELECT name FROM firms WHERE id=829Queries |
| Sum Individual 0.001311 |
| 10 | 0.00014000 | SELECT name FROM firms WHERE Query
| Combined id=812 | 0.000311 |
+------------------------+------------+
| 13 | 0.00031100 | SELECT id, name FROM firms WHERE id IN (723 ...
+----------+------------+---------------------------------------------------------



Row Processing
EXAMPLE
SET PROFILING=1;
SELECT … 4X longer processing for
SHOW PROFILES; every page load
+----------+------------+---------------------------------------------------------
| Query_ID | Duration | Query
+----------+------------+---------------------------------------------------------
SELECT 'Sum Individual Queries' AS txt,SUM(DURATI
INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID BETWE
UNION
| 4 | 0.00014000 | SELECT name FROM firms WHERE id=740Query',SUM(DURATION) FROM INFORM
SELECT 'Combined
| 5 | 0.00012300 | SELECT name FROM firms WHERE 13;
QUERY_ID = id=849
+------------------------+------------+
| txt | total_time |
+------------------------+------------+
| 9 | 0.00013900 | SELECT name FROM firms WHERE id=829Queries |
| Sum Individual 0.001311 |
| 10 | 0.00014000 | SELECT name FROM firms WHERE Query
| Combined id=812 | 0.000311 |
+------------------------+------------+
| 13 | 0.00031100 | SELECT id, name FROM firms WHERE id IN (723 ...
+----------+------------+---------------------------------------------------------
Instant DB Scalability



resources

Many principles are straight forward
for experienced Oracle resoures
Open Source projects
Less skilled workforce



Architecture



ARCHITECTure

Storage Engines
Reducing SQL
Monitoring
Replication



replication
Weakest Link



conclusion

MySQL is a relational database
Internals are very different
Instrumentation is improving in new
versions 5.5 GA 5.6 DMR
Planet MySQL http://planet.mysql.com



Available NOW

http://effectivemysql.com/book/optimizing-sql-statements/


Ronald Bradford
http://effectiveMySQL.com


[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie [INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)

Ähnlich wie [INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford) (20)

Mehr von Insight Technology, Inc.

Mehr von Insight Technology, Inc. (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

[INSIGHT OUT 2011] A17 mastering my sql performance tuning(ronald bradford)