SlideShare ist ein Scribd-Unternehmen logo
1 von 67
Downloaden Sie, um offline zu lesen
Advanced MySQL Query Tuning
Alexander Rubin
Principal Architect, Percona
May 13, 2015
www.percona.com
About Me
•  Working with MySQL for over 10 years
–  Started at MySQL AB, then Sun Microsystems,
–  then Oracle (MySQL Consulting)
–  Joined Percona 2 years ago
My name is Alexander Rubin
https://www.linkedin.com/in/alexanderrubin
www.percona.com
§  Queries
– Temporary Tables and Filesort
GROUP BY Optimizations
– ORDER BY Optimizations
– DISTINCT Queries
– “Calculated” fields – MySQL 5.7
Agenda
www.percona.com
§  Removed basic staff to focus on advanced
§  MySQL 5.6 by default
– Will not talk about subqueries issues
§  Improvements in MySQL 5.7
Additional Topics
www.percona.com
Query Tuning: basics
Indexes and
Explain examples
www.percona.com
MySQL Index: B-Tree
•  Scan thru the tree and go directly to 1 leaf
•  Stop
Equality search: select * from table where id = 12
http://en.wikipedia.org/wiki/B-tree
www.percona.com
MySQL Index: B-Tree
•  Scan thru the tree and visit many leafs/nodes
Range: select * from table where id in (6, 12, 18)
http://en.wikipedia.org/wiki/B-tree
www.percona.com
CREATE TABLE City (
ID int(11) NOT NULL AUTO_INCREMENT,
Name char(35) NOT NULL DEFAULT '',
CountryCode char(3) NOT NULL DEFAULT '',
District char(20) NOT NULL DEFAULT '',
Population int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (ID),
KEY CountryCode (CountryCode)
) Engine=InnoDB;
www.percona.com
• MySQL will use 1 (best) index
mysql> explain select * from City where ID = 1;
+-------+-------+---------------+---------+---------+-------+------+-------+
| table | type | possible_keys | key | key_len | ref | rows | Extra |
+-------+-------+---------------+---------+---------+-------+------+-------+
| City | const | PRIMARY | PRIMARY | 4 | const | 1 | |
+-------+-------+---------------+---------+---------+-------+------+-------+
mysql> explain select * from City where CountryCode = 'USA';
+-------+------+---------------+-------------+---------+-------+------+------------+
| table | type | possible_keys | key | key_len | ref | rows | Extra |
+-------+------+---------------+-------------+---------+-------+------+------------+
| City | ref | CountryCode | CountryCode | 3 | const | 274 | Using where|
+-------+------+---------------+-------------+---------+-------+------+------------+
www.percona.com
•  Leftmost part of combined index
mysql> alter table City add key
comb(CountryCode, District, Population);
www.percona.com
• Leftmost part of combined index
mysql> explain select * from City
where CountryCode = 'USA'G
********************** 1. row ******************
table: City
type: ref
possible_keys: comb
key: comb
key_len: 3
ref: const
rows: 273
Uses first field from the comb key
www.percona.com
•  Key_len = total size (in bytes) of index parts used
Index: comb(CountryCode, District, Population)
Explain:
key: comb
key_len: 3
Fields:
CountryCode char(3)
District char(20)
Population int(11)
3 -> Char(3) -> First field is used
www.percona.com
• 2 Leftmost Fields
mysql> explain select * from City
where CountryCode = 'USA' and District = 'California'G
********************** 1. row ******************
table: City
type: ref
possible_keys: comb
key: comb
key_len: 23
ref: const,const
rows: 68
Uses 2 first fields from the comb key
CountryCode = 3 chars
District = 20 chars
Total = 23
www.percona.com
• 3 Leftmost Fields
mysql> explain select * from City
where CountryCode = 'USA' and District = 'California’
and population > 10000G
********************** 1. row ******************
table: City
type: range
possible_keys: comb
key: comb
key_len: 27
ref: NULL
rows: 68
Uses all fields from the comb key
CountryCode = 3 chars/bytes
District = 20 chars/bytes
Population = 4 bytes (INT)
Total = 27
www.percona.com
mysql>	
  explain	
  format=JSON	
  select	
  *	
  from	
  City	
  	
  where	
  CountryCode	
  =	
  'USA'	
  and	
  District	
  =	
  'California'G	
  
***************************	
  1.	
  row	
  ***************************	
  
EXPLAIN:	
  {	
  
	
  	
  "query_block":	
  {	
  
	
  	
  	
  	
  "select_id":	
  1,	
  
	
  	
  	
  	
  "table":	
  {	
  
	
  	
  	
  	
  	
  	
  "table_name":	
  "City",	
  
	
  	
  	
  	
  	
  	
  "access_type":	
  "ref",	
  
	
  	
  	
  	
  	
  	
  "possible_keys":	
  [	
  
	
  	
  	
  	
  	
  	
  	
  	
  "comb"	
  
	
  	
  	
  	
  	
  	
  ],	
  
	
  	
  	
  	
  	
  	
  "key":	
  "comb",	
  
	
  	
  	
  	
  	
  	
  "used_key_parts":	
  [	
  
	
  	
  	
  	
  	
  	
  	
  	
  "CountryCode",	
  
	
  	
  	
  	
  	
  	
  	
  	
  "District"	
  
	
  	
  	
  	
  	
  	
  ],	
  
	
  	
  	
  	
  	
  	
  "key_length":	
  "23",	
  
	
  	
  	
  	
  	
  	
  "ref":	
  [	
  
	
  	
  	
  	
  	
  	
  	
  	
  "const",	
  
	
  	
  	
  	
  	
  	
  	
  	
  "const"	
  
	
  	
  	
  	
  	
  	
  ],	
  
	
  	
  	
  	
  	
  	
  "rows":	
  84,	
  
	
  	
  	
  	
  ...	
  
www.percona.com
• Can’t use combined index – not a leftmost part
mysql> explain select * from City where
District = 'California' and population > 10000G
********************** 1. row ******************
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 3868
Does not have the CountryCode
in the where clause
= can’t use comb index
www.percona.com
• Covered index = cover all fields in query
select name from City where CountryCode = 'USA'
and District = 'Alaska' and population > 10000
mysql> alter table City add key
cov1(CountryCode, District, population, name);
Uses all fields in the query in particular
order:
1.  Where part
2.  Group By/Order (not used now)
3.  Select part (here: name)
www.percona.com
• Explain
mysql> explain select name from City where CountryCode =
'USA' and District = 'Alaska' and population > 10000G
*************************** 1. row ***********
table: City
type: range
possible_keys: cov1
key: cov1
key_len: 27
ref: NULL
rows: 1
Extra: Using where; Using index
Using index = covered index is
used
MySQL will only use index
Will not go to the data file
www.percona.com
Query tuning example
Calculated Expressions
In MySQL Queries
www.percona.com
Air Traffic Statistics table
	
  
CREATE	
  TABLE	
  `ontime`	
  (	
  
	
  	
  `id`	
  int(11)	
  NOT	
  NULL	
  AUTO_INCREMENT,	
  
	
  	
  `YearD`	
  year(4)	
  NOT	
  NULL,	
  
	
  	
  `FlightDate`	
  date	
  DEFAULT	
  NULL,	
  
	
  	
  `Carrier`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginAirportID`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginCityName`	
  varchar(100)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginState`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestAirportID`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestCityName`	
  varchar(100)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestState`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `DepDelayMinutes`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `ArrDelayMinutes`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `Cancelled`	
  tinyint(4)	
  DEFAULT	
  NULL,	
  
	
  	
  `CancellationCode`	
  char(1)	
  DEFAULT	
  NULL,	
  
	
  	
  `Diverted`	
  tinyint(4)	
  DEFAULT	
  NULL,	
  
	
  	
  PRIMARY	
  KEY	
  (`id`),	
  
	
  	
  KEY	
  `FlightDate`	
  (`FlightDate`)	
  
)	
  ENGINE=InnoDB	
  	
  
	
  
www.percona.com
All flights in 2013, group by airline
	
  
mysql>	
  EXPLAIN	
  SELECT	
  carrier,	
  count(*)	
  FROM	
  ontime_sm	
  	
  
	
  	
  	
  	
  	
  WHERE	
  year(FlightDate)	
  =	
  2013	
  group	
  by	
  carrierG	
  
***************************	
  1.	
  row	
  ***************************	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  id:	
  1	
  
	
  	
  select_type:	
  SIMPLE	
  
	
  	
  	
  	
  	
  	
  	
  	
  table:	
  ontime_sm	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  type:	
  ALL	
  
possible_keys:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  key:	
  NULL	
  
	
  	
  	
  	
  	
  	
  key_len:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  ref:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  rows:	
  151253427	
  
	
  	
  	
  	
  	
  	
  	
  	
  Extra:	
  Using	
  where;	
  Using	
  temporary;	
  Using	
  filesort	
  
	
  
Results:	
  
16	
  rows	
  in	
  set	
  (1	
  min	
  49.48	
  sec)	
  
	
  
	
  
Year() = “calculated expression”
MySQL can’t use index
www.percona.com
Rewritten: flights in 2013,
group by airline
	
  
mysql>	
  EXPLAIN	
  SELECT	
  carrier,	
  count(*)	
  FROM	
  ontime_sm	
  	
  
	
  	
  	
  	
  	
  WHERE	
  	
  FlightDate	
  between	
  '2013-­‐01-­‐01'	
  and	
  '2013-­‐12-­‐31'	
  
	
  	
  	
  	
  	
  	
  	
  GROUP	
  BY	
  carrierG	
  
	
  
***************************	
  1.	
  row	
  ***************************	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  id:	
  1	
  
	
  	
  select_type:	
  SIMPLE	
  
	
  	
  	
  	
  	
  	
  	
  	
  table:	
  ontime_sm	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  type:	
  range	
  
possible_keys:	
  FlightDate	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  key:	
  FlightDate	
  
	
  	
  	
  	
  	
  	
  key_len:	
  4	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  ref:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  rows:	
  10434762	
  
	
  	
  	
  	
  	
  	
  	
  	
  Extra:	
  Using	
  index	
  condition;	
  Using	
  temporary;	
  Using	
  filesort	
  
	
  
Results:	
  	
  
16	
  rows	
  in	
  set	
  (11.98	
  sec)	
  
	
  
	
  
RANGE condition
MySQL can use index
Almost 10x faster
www.percona.com
All flights on Sunday example
	
  
	
  
	
  
	
  
mysql>	
  EXPLAIN	
  SELECT	
  carrier,	
  count(*)	
  FROM	
  ontime_sm	
  	
  
	
  	
  	
  	
  	
  WHERE	
  dayofweek(FlightDate)	
  =	
  7	
  group	
  by	
  carrierG	
  
***************************	
  1.	
  row	
  ***************************	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  id:	
  1	
  
	
  	
  select_type:	
  SIMPLE	
  
	
  	
  	
  	
  	
  	
  	
  	
  table:	
  ontime_sm	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  type:	
  ALL	
  
possible_keys:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  key:	
  NULL	
  
	
  	
  	
  	
  	
  	
  key_len:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  ref:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  rows:	
  151253427	
  
	
  	
  	
  	
  	
  	
  	
  	
  Extra:	
  Using	
  where;	
  Using	
  temporary;	
  Using	
  filesort	
  
	
  
Results:	
  
32	
  rows	
  in	
  set	
  (1	
  min	
  57.93	
  sec)	
  
	
  
Dayofweek() = “calculated expression”
MySQL can’t use index
www.percona.com
Storing additional field
	
  
	
  
	
  
CREATE	
  TABLE	
  `ontime_sm`	
  (	
  
	
  	
  `id`	
  int(11)	
  NOT	
  NULL	
  AUTO_INCREMENT,	
  
	
  	
  `YearD`	
  year(4)	
  NOT	
  NULL,	
  
	
  	
  `FlightDate`	
  date	
  DEFAULT	
  NULL,	
  
	
  	
  `Carrier`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginAirportID`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginCityName`	
  varchar(100)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginState`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestAirportID`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestCityName`	
  varchar(100)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestState`	
  char(2)	
  DEFAULT	
  NULL,	
  DEFAULT	
  	
  
	
  	
  `DepDelayMinutes`	
  int(11)	
  NULL,	
  
	
  	
  `ArrDelayMinutes`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `Cancelled`	
  tinyint(4)	
  DEFAULT	
  NULL,	
  
	
  	
  …	
  
	
  	
  Flight_dayofweek	
  tinyint	
  NOT	
  NULL,	
  
	
  	
  PRIMARY	
  KEY	
  (`id`),	
  
	
  	
  KEY	
  `Flight_dayofweek`	
  (`Flight_dayofweek`)	
  
)	
  ENGINE=InnoDB	
  DEFAULT	
  CHARSET=latin1;	
  
Materialize the field…
www.percona.com
Storing additional field
… and populating it with trigger
CREATE	
  DEFINER	
  =	
  CURRENT_USER	
  	
  
TRIGGER	
  ontime_insert	
  
BEFORE	
  INSERT	
  ON	
  ontime_sm_triggers	
  	
  
FOR	
  EACH	
  ROW	
  
SET	
  	
  
NEW.Flight_dayofweek	
  =	
  dayofweek(NEW.FlightDate);	
  
May be slower on insert
www.percona.com
Virtual Columns in MySQL 5.7.7
(labs release)
CREATE	
  TABLE	
  `ontime_sm_virtual`	
  (	
  
	
  	
  `id`	
  int(11)	
  NOT	
  NULL	
  AUTO_INCREMENT,	
  
	
  	
  `YearD`	
  year(4)	
  NOT	
  NULL,	
  
	
  	
  `FlightDate`	
  date	
  DEFAULT	
  NULL,	
  
	
  	
  `Carrier`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginAirportID`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginCityName`	
  varchar(100)	
  DEFAULT	
  NULL,	
  
	
  	
  `OriginState`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestAirportID`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestCityName`	
  varchar(100)	
  DEFAULT	
  NULL,	
  
	
  	
  `DestState`	
  char(2)	
  DEFAULT	
  NULL,	
  
	
  	
  `DepDelayMinutes`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `ArrDelayMinutes`	
  int(11)	
  DEFAULT	
  NULL,	
  
	
  	
  `Cancelled`	
  tinyint(4)	
  DEFAULT	
  NULL,	
  
	
  	
  ...	
  
`Flight_dayofweek`	
  tinyint(4)	
  	
  
GENERATED	
  ALWAYS	
  AS	
  (dayofweek(FlightDate))	
  VIRTUAL,	
  	
  	
  
PRIMARY	
  KEY	
  (`id`),	
  	
  
KEY	
  `Flight_dayofweek`	
  (`Flight_dayofweek	
  `)	
  
)	
  ENGINE=InnoDB;	
  
	
  
http://mysqlserverteam.com/generated-­‐columns-­‐in-­‐mysql-­‐5-­‐7-­‐5/	
  
https://dev.mysql.com/worklog/task/?id=8114	
  	
  
http://labs.mysql.com/	
  
	
  
Does not store the column
But INDEX it
www.percona.com
Virtual Columns in MySQL 5.7.7
(labs release)
	
  
mysql>	
  insert	
  into	
  ontime_sm_triggers	
  (id,	
  YearD,	
  FlightDate,	
  Carrier,	
  
OriginAirportID,	
  OriginCityName,	
  	
  OriginState,	
  DestAirportID,	
  DestCityName,	
  
DestState,	
  DepDelayMinutes,	
  ArrDelayMinutes,	
  Cancelled,	
  
CancellationCode,Diverted,	
  CRSElapsedTime,	
  ActualElapsedTime,	
  AirTime,	
  
Flights,	
  Distance)	
  select	
  *	
  from	
  ontime_sm;	
  
	
  
Query	
  OK,	
  999999	
  rows	
  affected	
  (27.86	
  sec)	
  
Records:	
  999999	
  	
  Duplicates:	
  0	
  	
  Warnings:	
  0	
  
	
  
mysql>	
  insert	
  into	
  ontime_sm_virtual	
  (id,	
  YearD,	
  FlightDate,	
  Carrier,	
  
OriginAirportID,	
  OriginCityName,	
  	
  OriginState,	
  DestAirportID,	
  DestCityName,	
  
DestState,	
  DepDelayMinutes,	
  ArrDelayMinutes,	
  Cancelled,	
  
CancellationCode,Diverted,	
  CRSElapsedTime,	
  ActualElapsedTime,	
  AirTime,	
  
Flights,	
  Distance)	
  select	
  *	
  from	
  ontime_sm;	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  
	
  
Query	
  OK,	
  999999	
  rows	
  affected	
  (16.29	
  sec)	
  
Records:	
  999999	
  	
  Duplicates:	
  0	
  	
  Warnings:	
  0	
  
Much faster on load
Compared to triggers
www.percona.com
Virtual Columns in MySQL 5.7.7
(labs release)
mysql>	
  EXPLAIN	
  SELECT	
  carrier,	
  count(*)	
  FROM	
  ontime_sm_virtual	
  	
  
WHERE	
  Flight_dayofweek	
  =	
  7	
  group	
  by	
  carrierG	
  
***************************	
  1.	
  row	
  ***************************	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  id:	
  1	
  
	
  	
  select_type:	
  SIMPLE	
  
	
  	
  	
  	
  	
  	
  	
  	
  table:	
  ontime_sm_virtual	
  
	
  	
  	
  partitions:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  type:	
  ref	
  
possible_keys:	
  Flight_dayofweek	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  key:	
  Flight_dayofweek	
  
	
  	
  	
  	
  	
  	
  key_len:	
  2	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  ref:	
  const	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  rows:	
  165409	
  
	
  	
  	
  	
  	
  filtered:	
  100.00	
  
	
  	
  	
  	
  	
  	
  	
  	
  Extra:	
  Using	
  where;	
  Using	
  temporary;	
  Using	
  filesort	
  
1	
  row	
  in	
  set,	
  1	
  warning	
  (0.00	
  sec)	
  
	
  
Using index
www.percona.com
Virtual Columns in MySQL 5.7.7
Limitations
mysql>	
  alter	
  table	
  ontime_sm_virtual	
  	
  
	
  	
  	
  	
  	
  	
  	
  add	
  key	
  comb(Flight_dayofweek,	
  carrier);	
  
	
  
ERROR	
  3105	
  (HY000):	
  'Virtual	
  generated	
  column	
  combines	
  with	
  
other	
  columns	
  to	
  be	
  indexed	
  together'	
  is	
  not	
  supported	
  for	
  
generated	
  columns.	
  
Can’t create a combined index
on virtual and “normal” columns
www.percona.com
Virtual Columns in MySQL 5.7.7
Limitations Workaround
mysql>	
  alter	
  table	
  ontime_sm_virtual	
  add	
  Carrier_virtual	
  char(2)	
  
GENERATED	
  ALWAYS	
  AS	
  (Carrier)	
  VIRTUAL;	
  
	
  
mysql>	
  alter	
  table	
  ontime_sm_virtual	
  add	
  key	
  comb(Flight_dayofweek,	
  
Carrier_virtual);	
  
	
  
mysql>	
  EXPLAIN	
  SELECT	
  Carrier_virtual,	
  count(*)	
  FROM	
  ontime_sm_virtual	
  
WHERE	
  Flight_dayofweek	
  =	
  7	
  group	
  by	
  Carrier_virtual	
  
***************************	
  1.	
  row	
  ***************************	
  
	
   	
   	
  	
   	
  ...	
  
	
  	
  	
  	
  	
  	
  	
  	
  table:	
  ontime_sm_virtual	
  
	
  	
  	
  partitions:	
  NULL	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  type:	
  ref	
  
possible_keys:	
  Flight_dayofweek,comb	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  key:	
  comb	
  
	
  	
  	
  	
  	
  	
  key_len:	
  2	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  ref:	
  const	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  rows:	
  141223	
  
	
  	
  	
  	
  	
  	
  	
  	
  Extra:	
  Using	
  where;	
  Using	
  index
www.percona.com
Complex Slow Queries
… Group By …
… Order By …
Select distinct …
Filesort
Temporary
tables
www.percona.com
GROUP BY Queries
www.percona.com
mysql> explain select CountryCode, count(*) from City
group by CountryCodeG
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using temporary; Using filesort
Temporary tables are slow!
How many cities in each country?
www.percona.com
Temporary Tables: Theory
www.percona.com
•  MySQL can create temporary tables when query uses:
•  GROUP BY
•  Range + ORDER BY
•  Some other expressions
Main performance issues
•  2 types of temporary tables
•  MEMORY
•  On-disk
www.percona.com
•  First, MySQL tries to create temporary table in memory
•  MySQL configuration variables:
•  tmp_table_size
•  maximum size for in Memory temporary tables
•  max_heap_table_size
•  Sets the maximum size for MEMORY tables
www.percona.com
MySQL
temp table
>
tmp_table_
size
OR
MySQL
temp table
>
max_heap_
table_size
convert to
MyISAM
temporary
table on
disk
MySQL 5.7:
Temp tables ON DISK are in
InnoDB format by default
www.percona.com
•  MEMORY engine does not support BLOB/TEXT
•  select blob_field from table group by field1
•  select concat(...string>512 chars) group by field1
• Create on-disk temporary table right away
www.percona.com
Temporary Tables: Practice
www.percona.com
6M rows, ~2G in size
CREATE TABLE ontime_2012 (
YearD int(11) DEFAULT NULL,
MonthD tinyint(4) DEFAULT NULL,
DayofMonth tinyint(4) DEFAULT NULL,
DayOfWeek tinyint(4) DEFAULT NULL,
Carrier char(2) DEFAULT NULL,
Origin char(5) DEFAULT NULL,
DepDelayMinutes int(11) DEFAULT NULL,
...
) ENGINE=InnoDB DEFAULT CHARSET=latin1
http://www.transtats.bts.gov/DL_SelectFields.asp?
Table_ID=236&DB_Short_Name=On-Time
www.percona.com
SELECT max(DepDelayMinutes),
carrier, dayofweek
FROM ontime_2012
WHERE dayofweek = 7
GROUP BY Carrier
•  Find maximum delay for flights on Sunday
•  Group by airline
www.percona.com
select max(DepDelayMinutes), carrier, dayofweek
from ontime_2012
where dayofweek = 7
group by Carrier
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4833086
Extra: Using where; Using temporary; Using
filesort
Full table scan!
Temporary table!
www.percona.com
mysql> alter table ontime_2012 add key (dayofweek);
explain select max(DepDelayMinutes), Carrier,
dayofweek from ontime_2012 where dayofweek =7
group by CarrierG
type: ref
possible_keys: DayOfWeek
key: DayOfWeek
key_len: 2
ref: const
rows: 817258
Extra: Using where; Using temporary; Using filesort
Index is used = better
BUT: Large temporary table!
www.percona.com
mysql> alter table ontime_2012
add key covered(dayofweek, Carrier, DepDelayMinutes);
explain select max(DepDelayMinutes), Carrier, dayofweek from
ontime_2012 where dayofweek =7 group by CarrierG
...
possible_keys: DayOfWeek,covered
key: covered
key_len: 2
ref: const
rows: 905138
Extra: Using where; Using index
•  Called “tight index scan”
No temporary table!
MySQL will only use index
www.percona.com
mysql> explain select max(DepDelayMinutes), Carrier,
dayofweek from ontime_2012
where dayofweek > 3 group by Carrier, dayofweekG
...
type: range
possible_keys: covered
key: covered
key_len: 2
ref: NULL
rows: 2441781
Extra: Using where; Using index; Using temporary;
Using filesort
Range scan
www.percona.com
(select max(DepDelayMinutes), Carrier, dayofweek
from ontime_2012
where dayofweek = 3
group by Carrier, dayofweek)
union
(select max(DepDelayMinutes), Carrier, dayofweek
from ontime_2012
where dayofweek = 4
group by Carrier, dayofweek)
www.percona.com
*************************** 1. row ***************************
table: ontime_2012
key: covered
... Removed ...
Extra: Using where; Using index
*************************** 2. row ***************************
table: ontime_2012
key: covered
... Removed ...
Extra: Using where; Using index
*************************** 3. row ***************************
id: NULL
select_type: UNION RESULT
table: <union1,2>
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
Extra: Using temporary
UNION ALL is an option
www.percona.com
GROUP BY: Loose index scan
•  Loose index scan:
•  considers only a fraction of the keys in an
index
•  Following rules apply:
•  The query is over a single table.
•  The GROUP BY columns should form a leftmost
prefix of the index
•  The only aggregate functions = MIN() and
MAX(), same column
www.percona.com
Loose index scan example
mysql> alter table ontime_2012 add key loose_index_scan
(Carrier, dayofweek, DepDelayMinutes);
mysql> explain select max(DepDelayMinutes), Carrier,
dayofweek from ontime_2012 where dayofweek > 5 group
by Carrier, dayofweekG
...
table: ontime_2012
type: range
possible_keys: NULL
key: loose_index_scan
key_len: 5
ref: NULL
rows: 201
Extra: Using where; Using index for group-by
Loose index scan
Very fast
“Range” works!
www.percona.com
Loose index scan vs. tight index scan
Table: ontime_2012, 6M rows, data: 2G, index: 210M
CREATE TABLE ontime_2012 (
YearD int(11) DEFAULT NULL,
MonthD tinyint(4) DEFAULT NULL,
DayofMonth tinyint(4) DEFAULT NULL,
DayOfWeek tinyint(4) DEFAULT NULL,
Carrier char(2) DEFAULT NULL,
Origin char(5) DEFAULT NULL,
DepDelayMinutes int(11) DEFAULT NULL,
...
KEY loose_index_scan (Carrier,DayOfWeek,DepDelayMinutes),
KEY covered (DayOfWeek,Carrier,DepDelayMinutes)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
www.percona.com
Loose index scan vs. tight index scan
Loose index scan
select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012
where dayofweek = 5 group by Carrier, dayofweek
table: ontime_2012
type: range
possible_keys: covered
key: loose_index_scan
key_len: 5
ref: NULL
rows: 201
Extra: Using where; Using index for group-by
mysql> select ...
+------+---------+-----------+
| ddm | Carrier | dayofweek |
+------+---------+-----------+
| 1606 | AA | 7 |
..
30 rows in set (0.00 sec)
Carrier,
DayOfWeek,
DepDelayMinutes
www.percona.com
Loose index scan vs. tight index scan
Results
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Loose index scan Tight Index Scan Temporary table
www.percona.com
Where loose index scan is not supported
•  AVG() + Group By – loose index scan is not supported
mysql> explain select avg(DepDelayMinutes) as ddm, Carrier, dayofweek
from ontime_2012 where dayofweek >5 group by Carrier, dayofweek G
table: ontime_2012
type: range
key: covered
key_len: 2
ref: NULL
rows: 2961617
Extra: Using where; Using index; Using temporary; Using filesort
mysql> select ...
+---------+---------+-----------+
| ddm | Carrier | dayofweek |
+---------+---------+-----------+
| 10.8564 | AA | 6 |
...
30 rows in set (1.39 sec)
1.  No loose index scan
2.  Filter by key
3.  Group by filesort
www.percona.com
ORDER BY and filesort
www.percona.com
mysql> explain select district, name, population from City
where CountryCode = 'USA' order by population desc limit 10G
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using where; Using filesort
www.percona.com
mysql> alter table City
add key my_sort2 (CountryCode, population);
mysql> explain select district, name, population from City
where CountryCode = 'USA' order by population desc limit 10G
table: City
type: ref
key: my_sort2
key_len: 3
ref: const
rows: 207
Extra: Using where
No filesort
www.percona.com
mysql> alter table ontime_2012 add key (DepDelayMinutes);
Query OK, 0 rows affected (38.68 sec)
mysql> explain select * from ontime_2012
where dayofweek in (6,7) order by DepDelayMinutes desc
limit 10G
type: index
possible_keys: DayOfWeek,covered
key: DepDelayMinutes
key_len: 5
ref: NULL
rows: 24
Extra: Using where
10 rows in set (0.00 sec)
1.  Index is sorted
2.  Scan the whole table in the
order of the index
3.  Filter results
4.  Stop after finding 10 rows
matching the “where” condition
www.percona.com
If Index points to the beginning of the table (physically) = fast
As it stops after 10 rows (LIMIT 10)
www.percona.com
If Index points to the end of table (physically) or random = slower
Much more rows to scan (and skip)
www.percona.com
SELECT DISTINCT
www.percona.com
select t.*, c.name from title t
join movie_companies m on t.id = m.movie_id
join company_name c on m.company_id = c.id
join company_type ct on m.company_type_id = ct.id
where
production_year > 1960 and ct.kind <> 'distributors’
order by production_year desc limit 10G
...
10 rows in set (0.00 sec)
JOINs are for filtering
Result set may contain duplicates
DISTINCT?
www.percona.com
select DISTINCT t.*, c.name from title t
join movie_companies m on t.id = m.movie_id
join company_name c on m.company_id = c.id
join company_type ct on m.company_type_id = ct.id
where
production_year > 1960 and ct.kind <> 'distributors’
order by production_year desc limit 10G
...
10 rows in set (37.27 sec)
www.percona.com
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t
type: range
possible_keys: PRIMARY,production_year
key: production_year
key_len: 5
ref: NULL
rows: 1163888
Extra: Using index condition; Using temporary
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: m
...
www.percona.com
mysql> show status like 'created%';
+-------------------------+-------+
| Variable_name | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 1 |
| Created_tmp_files | 0 |
| Created_tmp_tables | 1 |
+-------------------------+-------+
3 rows in set (0.00 sec)
www.percona.com
select distinct * from
(select t.*, c.name from title t
join movie_companies m on t.id = m.movie_id
join company_name c on m.company_id = c.id
join company_type ct on m.company_type_id = ct.id
where production_year > 1960
and ct.kind <> 'distributors’
order by production_year desc limit 1000)
as a limit 10;
...
10 rows in set (0.02 sec)
www.percona.com
Percona Cloud Tools
www.percona.com
Thank you!
https://www.linkedin.com/in/alexanderrubin
Alexander Rubin

Weitere ähnliche Inhalte

Was ist angesagt?

The MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer TraceThe MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer Traceoysteing
 
Boost Performance With My S Q L 51 Partitions
Boost Performance With  My S Q L 51 PartitionsBoost Performance With  My S Q L 51 Partitions
Boost Performance With My S Q L 51 PartitionsPerconaPerformance
 
MySQL8.0_performance_schema.pptx
MySQL8.0_performance_schema.pptxMySQL8.0_performance_schema.pptx
MySQL8.0_performance_schema.pptxNeoClova
 
Using Optimizer Hints to Improve MySQL Query Performance
Using Optimizer Hints to Improve MySQL Query PerformanceUsing Optimizer Hints to Improve MySQL Query Performance
Using Optimizer Hints to Improve MySQL Query Performanceoysteing
 
PostgreSQL Performance Tuning
PostgreSQL Performance TuningPostgreSQL Performance Tuning
PostgreSQL Performance Tuningelliando dias
 
MySQL 8.0 EXPLAIN ANALYZE
MySQL 8.0 EXPLAIN ANALYZEMySQL 8.0 EXPLAIN ANALYZE
MySQL 8.0 EXPLAIN ANALYZENorvald Ryeng
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
 
What is new in PostgreSQL 14?
What is new in PostgreSQL 14?What is new in PostgreSQL 14?
What is new in PostgreSQL 14?Mydbops
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
 
PostgreSQL Deep Internal
PostgreSQL Deep InternalPostgreSQL Deep Internal
PostgreSQL Deep InternalEXEM
 
Mysql Explain Explained
Mysql Explain ExplainedMysql Explain Explained
Mysql Explain ExplainedJeremy Coates
 
Innodb에서의 Purge 메커니즘 deep internal (by 이근오)
Innodb에서의 Purge 메커니즘 deep internal (by  이근오)Innodb에서의 Purge 메커니즘 deep internal (by  이근오)
Innodb에서의 Purge 메커니즘 deep internal (by 이근오)I Goo Lee.
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PGConf APAC
 
Mysql query optimization
Mysql query optimizationMysql query optimization
Mysql query optimizationBaohua Cai
 
Optimizing queries MySQL
Optimizing queries MySQLOptimizing queries MySQL
Optimizing queries MySQLGeorgi Sotirov
 
MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0Mayank Prasad
 
The InnoDB Storage Engine for MySQL
The InnoDB Storage Engine for MySQLThe InnoDB Storage Engine for MySQL
The InnoDB Storage Engine for MySQLMorgan Tocker
 

Was ist angesagt? (20)

The MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer TraceThe MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer Trace
 
The PostgreSQL Query Planner
The PostgreSQL Query PlannerThe PostgreSQL Query Planner
The PostgreSQL Query Planner
 
Load Data Fast!
Load Data Fast!Load Data Fast!
Load Data Fast!
 
Boost Performance With My S Q L 51 Partitions
Boost Performance With  My S Q L 51 PartitionsBoost Performance With  My S Q L 51 Partitions
Boost Performance With My S Q L 51 Partitions
 
MySQL8.0_performance_schema.pptx
MySQL8.0_performance_schema.pptxMySQL8.0_performance_schema.pptx
MySQL8.0_performance_schema.pptx
 
How to Use JSON in MySQL Wrong
How to Use JSON in MySQL WrongHow to Use JSON in MySQL Wrong
How to Use JSON in MySQL Wrong
 
Using Optimizer Hints to Improve MySQL Query Performance
Using Optimizer Hints to Improve MySQL Query PerformanceUsing Optimizer Hints to Improve MySQL Query Performance
Using Optimizer Hints to Improve MySQL Query Performance
 
PostgreSQL Performance Tuning
PostgreSQL Performance TuningPostgreSQL Performance Tuning
PostgreSQL Performance Tuning
 
MySQL 8.0 EXPLAIN ANALYZE
MySQL 8.0 EXPLAIN ANALYZEMySQL 8.0 EXPLAIN ANALYZE
MySQL 8.0 EXPLAIN ANALYZE
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
 
What is new in PostgreSQL 14?
What is new in PostgreSQL 14?What is new in PostgreSQL 14?
What is new in PostgreSQL 14?
 
How to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better PerformanceHow to Analyze and Tune MySQL Queries for Better Performance
How to Analyze and Tune MySQL Queries for Better Performance
 
PostgreSQL Deep Internal
PostgreSQL Deep InternalPostgreSQL Deep Internal
PostgreSQL Deep Internal
 
Mysql Explain Explained
Mysql Explain ExplainedMysql Explain Explained
Mysql Explain Explained
 
Innodb에서의 Purge 메커니즘 deep internal (by 이근오)
Innodb에서의 Purge 메커니즘 deep internal (by  이근오)Innodb에서의 Purge 메커니즘 deep internal (by  이근오)
Innodb에서의 Purge 메커니즘 deep internal (by 이근오)
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs
 
Mysql query optimization
Mysql query optimizationMysql query optimization
Mysql query optimization
 
Optimizing queries MySQL
Optimizing queries MySQLOptimizing queries MySQL
Optimizing queries MySQL
 
MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0MySQL Performance Schema in MySQL 8.0
MySQL Performance Schema in MySQL 8.0
 
The InnoDB Storage Engine for MySQL
The InnoDB Storage Engine for MySQLThe InnoDB Storage Engine for MySQL
The InnoDB Storage Engine for MySQL
 

Andere mochten auch

Zurich2007 MySQL Query Optimization
Zurich2007 MySQL Query OptimizationZurich2007 MySQL Query Optimization
Zurich2007 MySQL Query OptimizationHiệp Lê Tuấn
 
Building High Performance MySql Query Systems And Analytic Applications
Building High Performance MySql Query Systems And Analytic ApplicationsBuilding High Performance MySql Query Systems And Analytic Applications
Building High Performance MySql Query Systems And Analytic Applicationsguest40cda0b
 
MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014
MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014
MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014Dave Stokes
 
56 Query Optimization
56 Query Optimization56 Query Optimization
56 Query OptimizationMYXPLAIN
 
MYSQL Query Anti-Patterns That Can Be Moved to Sphinx
MYSQL Query Anti-Patterns That Can Be Moved to SphinxMYSQL Query Anti-Patterns That Can Be Moved to Sphinx
MYSQL Query Anti-Patterns That Can Be Moved to SphinxPythian
 
Query Optimization with MySQL 5.6: Old and New Tricks
Query Optimization with MySQL 5.6: Old and New TricksQuery Optimization with MySQL 5.6: Old and New Tricks
Query Optimization with MySQL 5.6: Old and New TricksMYXPLAIN
 
Tunning sql query
Tunning sql queryTunning sql query
Tunning sql queryvuhaininh88
 
MySQL Query tuning 101
MySQL Query tuning 101MySQL Query tuning 101
MySQL Query tuning 101Sveta Smirnova
 
Advanced MySQL Query and Schema Tuning
Advanced MySQL Query and Schema TuningAdvanced MySQL Query and Schema Tuning
Advanced MySQL Query and Schema TuningMYXPLAIN
 
MySQL Query Optimization
MySQL Query OptimizationMySQL Query Optimization
MySQL Query OptimizationMorgan Tocker
 
Webinar 2013 advanced_query_tuning
Webinar 2013 advanced_query_tuningWebinar 2013 advanced_query_tuning
Webinar 2013 advanced_query_tuning晓 周
 
MySQL Query Optimization (Basics)
MySQL Query Optimization (Basics)MySQL Query Optimization (Basics)
MySQL Query Optimization (Basics)Karthik .P.R
 
Percona Live 2012PPT: MySQL Query optimization
Percona Live 2012PPT: MySQL Query optimizationPercona Live 2012PPT: MySQL Query optimization
Percona Live 2012PPT: MySQL Query optimizationmysqlops
 

Andere mochten auch (16)

Zurich2007 MySQL Query Optimization
Zurich2007 MySQL Query OptimizationZurich2007 MySQL Query Optimization
Zurich2007 MySQL Query Optimization
 
Building High Performance MySql Query Systems And Analytic Applications
Building High Performance MySql Query Systems And Analytic ApplicationsBuilding High Performance MySql Query Systems And Analytic Applications
Building High Performance MySql Query Systems And Analytic Applications
 
MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014
MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014
MySQL Query Tuning for the Squeemish -- Fossetcon Orlando Sep 2014
 
56 Query Optimization
56 Query Optimization56 Query Optimization
56 Query Optimization
 
MYSQL Query Anti-Patterns That Can Be Moved to Sphinx
MYSQL Query Anti-Patterns That Can Be Moved to SphinxMYSQL Query Anti-Patterns That Can Be Moved to Sphinx
MYSQL Query Anti-Patterns That Can Be Moved to Sphinx
 
Query Optimization with MySQL 5.6: Old and New Tricks
Query Optimization with MySQL 5.6: Old and New TricksQuery Optimization with MySQL 5.6: Old and New Tricks
Query Optimization with MySQL 5.6: Old and New Tricks
 
Tunning sql query
Tunning sql queryTunning sql query
Tunning sql query
 
MySQL Query tuning 101
MySQL Query tuning 101MySQL Query tuning 101
MySQL Query tuning 101
 
Advanced MySQL Query and Schema Tuning
Advanced MySQL Query and Schema TuningAdvanced MySQL Query and Schema Tuning
Advanced MySQL Query and Schema Tuning
 
MySQL Query Optimization
MySQL Query OptimizationMySQL Query Optimization
MySQL Query Optimization
 
My sql optimization
My sql optimizationMy sql optimization
My sql optimization
 
Webinar 2013 advanced_query_tuning
Webinar 2013 advanced_query_tuningWebinar 2013 advanced_query_tuning
Webinar 2013 advanced_query_tuning
 
MySQL Query Optimization.
MySQL Query Optimization.MySQL Query Optimization.
MySQL Query Optimization.
 
MySQL Query Optimization (Basics)
MySQL Query Optimization (Basics)MySQL Query Optimization (Basics)
MySQL Query Optimization (Basics)
 
Percona Live 2012PPT: MySQL Query optimization
Percona Live 2012PPT: MySQL Query optimizationPercona Live 2012PPT: MySQL Query optimization
Percona Live 2012PPT: MySQL Query optimization
 
Mastering MySQL/Query
Mastering MySQL/QueryMastering MySQL/Query
Mastering MySQL/Query
 

Ähnlich wie Advanced MySQL Query Tuning

Troubleshooting MySQL Performance
Troubleshooting MySQL PerformanceTroubleshooting MySQL Performance
Troubleshooting MySQL PerformanceSveta Smirnova
 
Introduction into MySQL Query Tuning
Introduction into MySQL Query TuningIntroduction into MySQL Query Tuning
Introduction into MySQL Query TuningSveta Smirnova
 
Migration from mysql to elasticsearch
Migration from mysql to elasticsearchMigration from mysql to elasticsearch
Migration from mysql to elasticsearchRyosuke Nakamura
 
Covering indexes
Covering indexesCovering indexes
Covering indexesMYXPLAIN
 
Why Use EXPLAIN FORMAT=JSON?
 Why Use EXPLAIN FORMAT=JSON?  Why Use EXPLAIN FORMAT=JSON?
Why Use EXPLAIN FORMAT=JSON? Sveta Smirnova
 
mysql 高级优化之 理解索引使用
mysql 高级优化之 理解索引使用mysql 高级优化之 理解索引使用
mysql 高级优化之 理解索引使用nigel889
 
Optimizing Queries with Explain
Optimizing Queries with ExplainOptimizing Queries with Explain
Optimizing Queries with ExplainMYXPLAIN
 
SQL Tuning and VST
SQL Tuning and VST SQL Tuning and VST
SQL Tuning and VST Kyle Hailey
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQLEvan Weaver
 
PPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_paginationPPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_paginationmysqlops
 
A few things about the Oracle optimizer - 2013
A few things about the Oracle optimizer - 2013A few things about the Oracle optimizer - 2013
A few things about the Oracle optimizer - 2013Connor McDonald
 
Postgres Conference (PgCon) New York 2019
Postgres Conference (PgCon) New York 2019Postgres Conference (PgCon) New York 2019
Postgres Conference (PgCon) New York 2019Ibrar Ahmed
 
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)Ontico
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQLSurat Singh Bhati
 
What's New In MySQL 5.6
What's New In MySQL 5.6What's New In MySQL 5.6
What's New In MySQL 5.6Abdul Manaf
 
Algorithms devised for a google interview
Algorithms devised for a google interviewAlgorithms devised for a google interview
Algorithms devised for a google interviewRussell Childs
 
Cassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE SearchCassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE SearchCaleb Rackliffe
 
DataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise SearchDataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise SearchDataStax Academy
 

Ähnlich wie Advanced MySQL Query Tuning (20)

Troubleshooting MySQL Performance
Troubleshooting MySQL PerformanceTroubleshooting MySQL Performance
Troubleshooting MySQL Performance
 
Optimizando MySQL
Optimizando MySQLOptimizando MySQL
Optimizando MySQL
 
Introduction into MySQL Query Tuning
Introduction into MySQL Query TuningIntroduction into MySQL Query Tuning
Introduction into MySQL Query Tuning
 
Migration from mysql to elasticsearch
Migration from mysql to elasticsearchMigration from mysql to elasticsearch
Migration from mysql to elasticsearch
 
Covering indexes
Covering indexesCovering indexes
Covering indexes
 
Why Use EXPLAIN FORMAT=JSON?
 Why Use EXPLAIN FORMAT=JSON?  Why Use EXPLAIN FORMAT=JSON?
Why Use EXPLAIN FORMAT=JSON?
 
mysql 高级优化之 理解索引使用
mysql 高级优化之 理解索引使用mysql 高级优化之 理解索引使用
mysql 高级优化之 理解索引使用
 
Optimizing Queries with Explain
Optimizing Queries with ExplainOptimizing Queries with Explain
Optimizing Queries with Explain
 
SQL Tuning and VST
SQL Tuning and VST SQL Tuning and VST
SQL Tuning and VST
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQL
 
PPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_paginationPPC2009_yahoo_mysql_pagination
PPC2009_yahoo_mysql_pagination
 
A few things about the Oracle optimizer - 2013
A few things about the Oracle optimizer - 2013A few things about the Oracle optimizer - 2013
A few things about the Oracle optimizer - 2013
 
Postgres Conference (PgCon) New York 2019
Postgres Conference (PgCon) New York 2019Postgres Conference (PgCon) New York 2019
Postgres Conference (PgCon) New York 2019
 
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
 
Efficient Pagination Using MySQL
Efficient Pagination Using MySQLEfficient Pagination Using MySQL
Efficient Pagination Using MySQL
 
What's New In MySQL 5.6
What's New In MySQL 5.6What's New In MySQL 5.6
What's New In MySQL 5.6
 
Algorithms devised for a google interview
Algorithms devised for a google interviewAlgorithms devised for a google interview
Algorithms devised for a google interview
 
Cassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE SearchCassandra Summit 2015: Intro to DSE Search
Cassandra Summit 2015: Intro to DSE Search
 
DataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise SearchDataStax: An Introduction to DataStax Enterprise Search
DataStax: An Introduction to DataStax Enterprise Search
 
Sql queries
Sql queriesSql queries
Sql queries
 

Kürzlich hochgeladen

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️Delhi Call girls
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
Active Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdfActive Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdfCionsystems
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
Test Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and BackendTest Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and BackendArshad QA
 
Software Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsSoftware Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsArshad QA
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software DevelopersVinodh Ram
 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionSolGuruz
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Steffen Staab
 
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AISyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AIABDERRAOUF MEHENNI
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️anilsa9823
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq
 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Modelsaagamshah0812
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about usDynamic Netsoft
 
A Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxA Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxComplianceQuest1
 

Kürzlich hochgeladen (20)

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
Active Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdfActive Directory Penetration Testing, cionsystems.com.pdf
Active Directory Penetration Testing, cionsystems.com.pdf
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
Test Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and BackendTest Automation Strategy for Frontend and Backend
Test Automation Strategy for Frontend and Backend
 
Software Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsSoftware Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software Developers
 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
 
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AISyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
SyndBuddy AI 2k Review 2024: Revolutionizing Content Syndication with AI
 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS LiveVip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online  ☂️
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
 
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about us
 
A Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxA Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docx
 

Advanced MySQL Query Tuning

  • 1. Advanced MySQL Query Tuning Alexander Rubin Principal Architect, Percona May 13, 2015
  • 2. www.percona.com About Me •  Working with MySQL for over 10 years –  Started at MySQL AB, then Sun Microsystems, –  then Oracle (MySQL Consulting) –  Joined Percona 2 years ago My name is Alexander Rubin https://www.linkedin.com/in/alexanderrubin
  • 3. www.percona.com §  Queries – Temporary Tables and Filesort GROUP BY Optimizations – ORDER BY Optimizations – DISTINCT Queries – “Calculated” fields – MySQL 5.7 Agenda
  • 4. www.percona.com §  Removed basic staff to focus on advanced §  MySQL 5.6 by default – Will not talk about subqueries issues §  Improvements in MySQL 5.7 Additional Topics
  • 6. www.percona.com MySQL Index: B-Tree •  Scan thru the tree and go directly to 1 leaf •  Stop Equality search: select * from table where id = 12 http://en.wikipedia.org/wiki/B-tree
  • 7. www.percona.com MySQL Index: B-Tree •  Scan thru the tree and visit many leafs/nodes Range: select * from table where id in (6, 12, 18) http://en.wikipedia.org/wiki/B-tree
  • 8. www.percona.com CREATE TABLE City ( ID int(11) NOT NULL AUTO_INCREMENT, Name char(35) NOT NULL DEFAULT '', CountryCode char(3) NOT NULL DEFAULT '', District char(20) NOT NULL DEFAULT '', Population int(11) NOT NULL DEFAULT '0', PRIMARY KEY (ID), KEY CountryCode (CountryCode) ) Engine=InnoDB;
  • 9. www.percona.com • MySQL will use 1 (best) index mysql> explain select * from City where ID = 1; +-------+-------+---------------+---------+---------+-------+------+-------+ | table | type | possible_keys | key | key_len | ref | rows | Extra | +-------+-------+---------------+---------+---------+-------+------+-------+ | City | const | PRIMARY | PRIMARY | 4 | const | 1 | | +-------+-------+---------------+---------+---------+-------+------+-------+ mysql> explain select * from City where CountryCode = 'USA'; +-------+------+---------------+-------------+---------+-------+------+------------+ | table | type | possible_keys | key | key_len | ref | rows | Extra | +-------+------+---------------+-------------+---------+-------+------+------------+ | City | ref | CountryCode | CountryCode | 3 | const | 274 | Using where| +-------+------+---------------+-------------+---------+-------+------+------------+
  • 10. www.percona.com •  Leftmost part of combined index mysql> alter table City add key comb(CountryCode, District, Population);
  • 11. www.percona.com • Leftmost part of combined index mysql> explain select * from City where CountryCode = 'USA'G ********************** 1. row ****************** table: City type: ref possible_keys: comb key: comb key_len: 3 ref: const rows: 273 Uses first field from the comb key
  • 12. www.percona.com •  Key_len = total size (in bytes) of index parts used Index: comb(CountryCode, District, Population) Explain: key: comb key_len: 3 Fields: CountryCode char(3) District char(20) Population int(11) 3 -> Char(3) -> First field is used
  • 13. www.percona.com • 2 Leftmost Fields mysql> explain select * from City where CountryCode = 'USA' and District = 'California'G ********************** 1. row ****************** table: City type: ref possible_keys: comb key: comb key_len: 23 ref: const,const rows: 68 Uses 2 first fields from the comb key CountryCode = 3 chars District = 20 chars Total = 23
  • 14. www.percona.com • 3 Leftmost Fields mysql> explain select * from City where CountryCode = 'USA' and District = 'California’ and population > 10000G ********************** 1. row ****************** table: City type: range possible_keys: comb key: comb key_len: 27 ref: NULL rows: 68 Uses all fields from the comb key CountryCode = 3 chars/bytes District = 20 chars/bytes Population = 4 bytes (INT) Total = 27
  • 15. www.percona.com mysql>  explain  format=JSON  select  *  from  City    where  CountryCode  =  'USA'  and  District  =  'California'G   ***************************  1.  row  ***************************   EXPLAIN:  {      "query_block":  {          "select_id":  1,          "table":  {              "table_name":  "City",              "access_type":  "ref",              "possible_keys":  [                  "comb"              ],              "key":  "comb",              "used_key_parts":  [                  "CountryCode",                  "District"              ],              "key_length":  "23",              "ref":  [                  "const",                  "const"              ],              "rows":  84,          ...  
  • 16. www.percona.com • Can’t use combined index – not a leftmost part mysql> explain select * from City where District = 'California' and population > 10000G ********************** 1. row ****************** table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 3868 Does not have the CountryCode in the where clause = can’t use comb index
  • 17. www.percona.com • Covered index = cover all fields in query select name from City where CountryCode = 'USA' and District = 'Alaska' and population > 10000 mysql> alter table City add key cov1(CountryCode, District, population, name); Uses all fields in the query in particular order: 1.  Where part 2.  Group By/Order (not used now) 3.  Select part (here: name)
  • 18. www.percona.com • Explain mysql> explain select name from City where CountryCode = 'USA' and District = 'Alaska' and population > 10000G *************************** 1. row *********** table: City type: range possible_keys: cov1 key: cov1 key_len: 27 ref: NULL rows: 1 Extra: Using where; Using index Using index = covered index is used MySQL will only use index Will not go to the data file
  • 19. www.percona.com Query tuning example Calculated Expressions In MySQL Queries
  • 20. www.percona.com Air Traffic Statistics table   CREATE  TABLE  `ontime`  (      `id`  int(11)  NOT  NULL  AUTO_INCREMENT,      `YearD`  year(4)  NOT  NULL,      `FlightDate`  date  DEFAULT  NULL,      `Carrier`  char(2)  DEFAULT  NULL,      `OriginAirportID`  int(11)  DEFAULT  NULL,      `OriginCityName`  varchar(100)  DEFAULT  NULL,      `OriginState`  char(2)  DEFAULT  NULL,      `DestAirportID`  int(11)  DEFAULT  NULL,      `DestCityName`  varchar(100)  DEFAULT  NULL,      `DestState`  char(2)  DEFAULT  NULL,      `DepDelayMinutes`  int(11)  DEFAULT  NULL,      `ArrDelayMinutes`  int(11)  DEFAULT  NULL,      `Cancelled`  tinyint(4)  DEFAULT  NULL,      `CancellationCode`  char(1)  DEFAULT  NULL,      `Diverted`  tinyint(4)  DEFAULT  NULL,      PRIMARY  KEY  (`id`),      KEY  `FlightDate`  (`FlightDate`)   )  ENGINE=InnoDB      
  • 21. www.percona.com All flights in 2013, group by airline   mysql>  EXPLAIN  SELECT  carrier,  count(*)  FROM  ontime_sm              WHERE  year(FlightDate)  =  2013  group  by  carrierG   ***************************  1.  row  ***************************                        id:  1      select_type:  SIMPLE                  table:  ontime_sm                    type:  ALL   possible_keys:  NULL                      key:  NULL              key_len:  NULL                      ref:  NULL                    rows:  151253427                  Extra:  Using  where;  Using  temporary;  Using  filesort     Results:   16  rows  in  set  (1  min  49.48  sec)       Year() = “calculated expression” MySQL can’t use index
  • 22. www.percona.com Rewritten: flights in 2013, group by airline   mysql>  EXPLAIN  SELECT  carrier,  count(*)  FROM  ontime_sm              WHERE    FlightDate  between  '2013-­‐01-­‐01'  and  '2013-­‐12-­‐31'                GROUP  BY  carrierG     ***************************  1.  row  ***************************                        id:  1      select_type:  SIMPLE                  table:  ontime_sm                    type:  range   possible_keys:  FlightDate                      key:  FlightDate              key_len:  4                      ref:  NULL                    rows:  10434762                  Extra:  Using  index  condition;  Using  temporary;  Using  filesort     Results:     16  rows  in  set  (11.98  sec)       RANGE condition MySQL can use index Almost 10x faster
  • 23. www.percona.com All flights on Sunday example         mysql>  EXPLAIN  SELECT  carrier,  count(*)  FROM  ontime_sm              WHERE  dayofweek(FlightDate)  =  7  group  by  carrierG   ***************************  1.  row  ***************************                        id:  1      select_type:  SIMPLE                  table:  ontime_sm                    type:  ALL   possible_keys:  NULL                      key:  NULL              key_len:  NULL                      ref:  NULL                    rows:  151253427                  Extra:  Using  where;  Using  temporary;  Using  filesort     Results:   32  rows  in  set  (1  min  57.93  sec)     Dayofweek() = “calculated expression” MySQL can’t use index
  • 24. www.percona.com Storing additional field       CREATE  TABLE  `ontime_sm`  (      `id`  int(11)  NOT  NULL  AUTO_INCREMENT,      `YearD`  year(4)  NOT  NULL,      `FlightDate`  date  DEFAULT  NULL,      `Carrier`  char(2)  DEFAULT  NULL,      `OriginAirportID`  int(11)  DEFAULT  NULL,      `OriginCityName`  varchar(100)  DEFAULT  NULL,      `OriginState`  char(2)  DEFAULT  NULL,      `DestAirportID`  int(11)  DEFAULT  NULL,      `DestCityName`  varchar(100)  DEFAULT  NULL,      `DestState`  char(2)  DEFAULT  NULL,  DEFAULT        `DepDelayMinutes`  int(11)  NULL,      `ArrDelayMinutes`  int(11)  DEFAULT  NULL,      `Cancelled`  tinyint(4)  DEFAULT  NULL,      …      Flight_dayofweek  tinyint  NOT  NULL,      PRIMARY  KEY  (`id`),      KEY  `Flight_dayofweek`  (`Flight_dayofweek`)   )  ENGINE=InnoDB  DEFAULT  CHARSET=latin1;   Materialize the field…
  • 25. www.percona.com Storing additional field … and populating it with trigger CREATE  DEFINER  =  CURRENT_USER     TRIGGER  ontime_insert   BEFORE  INSERT  ON  ontime_sm_triggers     FOR  EACH  ROW   SET     NEW.Flight_dayofweek  =  dayofweek(NEW.FlightDate);   May be slower on insert
  • 26. www.percona.com Virtual Columns in MySQL 5.7.7 (labs release) CREATE  TABLE  `ontime_sm_virtual`  (      `id`  int(11)  NOT  NULL  AUTO_INCREMENT,      `YearD`  year(4)  NOT  NULL,      `FlightDate`  date  DEFAULT  NULL,      `Carrier`  char(2)  DEFAULT  NULL,      `OriginAirportID`  int(11)  DEFAULT  NULL,      `OriginCityName`  varchar(100)  DEFAULT  NULL,      `OriginState`  char(2)  DEFAULT  NULL,      `DestAirportID`  int(11)  DEFAULT  NULL,      `DestCityName`  varchar(100)  DEFAULT  NULL,      `DestState`  char(2)  DEFAULT  NULL,      `DepDelayMinutes`  int(11)  DEFAULT  NULL,      `ArrDelayMinutes`  int(11)  DEFAULT  NULL,      `Cancelled`  tinyint(4)  DEFAULT  NULL,      ...   `Flight_dayofweek`  tinyint(4)     GENERATED  ALWAYS  AS  (dayofweek(FlightDate))  VIRTUAL,       PRIMARY  KEY  (`id`),     KEY  `Flight_dayofweek`  (`Flight_dayofweek  `)   )  ENGINE=InnoDB;     http://mysqlserverteam.com/generated-­‐columns-­‐in-­‐mysql-­‐5-­‐7-­‐5/   https://dev.mysql.com/worklog/task/?id=8114     http://labs.mysql.com/     Does not store the column But INDEX it
  • 27. www.percona.com Virtual Columns in MySQL 5.7.7 (labs release)   mysql>  insert  into  ontime_sm_triggers  (id,  YearD,  FlightDate,  Carrier,   OriginAirportID,  OriginCityName,    OriginState,  DestAirportID,  DestCityName,   DestState,  DepDelayMinutes,  ArrDelayMinutes,  Cancelled,   CancellationCode,Diverted,  CRSElapsedTime,  ActualElapsedTime,  AirTime,   Flights,  Distance)  select  *  from  ontime_sm;     Query  OK,  999999  rows  affected  (27.86  sec)   Records:  999999    Duplicates:  0    Warnings:  0     mysql>  insert  into  ontime_sm_virtual  (id,  YearD,  FlightDate,  Carrier,   OriginAirportID,  OriginCityName,    OriginState,  DestAirportID,  DestCityName,   DestState,  DepDelayMinutes,  ArrDelayMinutes,  Cancelled,   CancellationCode,Diverted,  CRSElapsedTime,  ActualElapsedTime,  AirTime,   Flights,  Distance)  select  *  from  ontime_sm;                                                                             Query  OK,  999999  rows  affected  (16.29  sec)   Records:  999999    Duplicates:  0    Warnings:  0   Much faster on load Compared to triggers
  • 28. www.percona.com Virtual Columns in MySQL 5.7.7 (labs release) mysql>  EXPLAIN  SELECT  carrier,  count(*)  FROM  ontime_sm_virtual     WHERE  Flight_dayofweek  =  7  group  by  carrierG   ***************************  1.  row  ***************************                        id:  1      select_type:  SIMPLE                  table:  ontime_sm_virtual        partitions:  NULL                    type:  ref   possible_keys:  Flight_dayofweek                      key:  Flight_dayofweek              key_len:  2                      ref:  const                    rows:  165409            filtered:  100.00                  Extra:  Using  where;  Using  temporary;  Using  filesort   1  row  in  set,  1  warning  (0.00  sec)     Using index
  • 29. www.percona.com Virtual Columns in MySQL 5.7.7 Limitations mysql>  alter  table  ontime_sm_virtual                  add  key  comb(Flight_dayofweek,  carrier);     ERROR  3105  (HY000):  'Virtual  generated  column  combines  with   other  columns  to  be  indexed  together'  is  not  supported  for   generated  columns.   Can’t create a combined index on virtual and “normal” columns
  • 30. www.percona.com Virtual Columns in MySQL 5.7.7 Limitations Workaround mysql>  alter  table  ontime_sm_virtual  add  Carrier_virtual  char(2)   GENERATED  ALWAYS  AS  (Carrier)  VIRTUAL;     mysql>  alter  table  ontime_sm_virtual  add  key  comb(Flight_dayofweek,   Carrier_virtual);     mysql>  EXPLAIN  SELECT  Carrier_virtual,  count(*)  FROM  ontime_sm_virtual   WHERE  Flight_dayofweek  =  7  group  by  Carrier_virtual   ***************************  1.  row  ***************************            ...                  table:  ontime_sm_virtual        partitions:  NULL                    type:  ref   possible_keys:  Flight_dayofweek,comb                      key:  comb              key_len:  2                      ref:  const                    rows:  141223                  Extra:  Using  where;  Using  index
  • 31. www.percona.com Complex Slow Queries … Group By … … Order By … Select distinct … Filesort Temporary tables
  • 33. www.percona.com mysql> explain select CountryCode, count(*) from City group by CountryCodeG id: 1 select_type: SIMPLE table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using temporary; Using filesort Temporary tables are slow! How many cities in each country?
  • 35. www.percona.com •  MySQL can create temporary tables when query uses: •  GROUP BY •  Range + ORDER BY •  Some other expressions Main performance issues •  2 types of temporary tables •  MEMORY •  On-disk
  • 36. www.percona.com •  First, MySQL tries to create temporary table in memory •  MySQL configuration variables: •  tmp_table_size •  maximum size for in Memory temporary tables •  max_heap_table_size •  Sets the maximum size for MEMORY tables
  • 37. www.percona.com MySQL temp table > tmp_table_ size OR MySQL temp table > max_heap_ table_size convert to MyISAM temporary table on disk MySQL 5.7: Temp tables ON DISK are in InnoDB format by default
  • 38. www.percona.com •  MEMORY engine does not support BLOB/TEXT •  select blob_field from table group by field1 •  select concat(...string>512 chars) group by field1 • Create on-disk temporary table right away
  • 40. www.percona.com 6M rows, ~2G in size CREATE TABLE ontime_2012 ( YearD int(11) DEFAULT NULL, MonthD tinyint(4) DEFAULT NULL, DayofMonth tinyint(4) DEFAULT NULL, DayOfWeek tinyint(4) DEFAULT NULL, Carrier char(2) DEFAULT NULL, Origin char(5) DEFAULT NULL, DepDelayMinutes int(11) DEFAULT NULL, ... ) ENGINE=InnoDB DEFAULT CHARSET=latin1 http://www.transtats.bts.gov/DL_SelectFields.asp? Table_ID=236&DB_Short_Name=On-Time
  • 41. www.percona.com SELECT max(DepDelayMinutes), carrier, dayofweek FROM ontime_2012 WHERE dayofweek = 7 GROUP BY Carrier •  Find maximum delay for flights on Sunday •  Group by airline
  • 42. www.percona.com select max(DepDelayMinutes), carrier, dayofweek from ontime_2012 where dayofweek = 7 group by Carrier type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4833086 Extra: Using where; Using temporary; Using filesort Full table scan! Temporary table!
  • 43. www.percona.com mysql> alter table ontime_2012 add key (dayofweek); explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek =7 group by CarrierG type: ref possible_keys: DayOfWeek key: DayOfWeek key_len: 2 ref: const rows: 817258 Extra: Using where; Using temporary; Using filesort Index is used = better BUT: Large temporary table!
  • 44. www.percona.com mysql> alter table ontime_2012 add key covered(dayofweek, Carrier, DepDelayMinutes); explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek =7 group by CarrierG ... possible_keys: DayOfWeek,covered key: covered key_len: 2 ref: const rows: 905138 Extra: Using where; Using index •  Called “tight index scan” No temporary table! MySQL will only use index
  • 45. www.percona.com mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek > 3 group by Carrier, dayofweekG ... type: range possible_keys: covered key: covered key_len: 2 ref: NULL rows: 2441781 Extra: Using where; Using index; Using temporary; Using filesort Range scan
  • 46. www.percona.com (select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek = 3 group by Carrier, dayofweek) union (select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek = 4 group by Carrier, dayofweek)
  • 47. www.percona.com *************************** 1. row *************************** table: ontime_2012 key: covered ... Removed ... Extra: Using where; Using index *************************** 2. row *************************** table: ontime_2012 key: covered ... Removed ... Extra: Using where; Using index *************************** 3. row *************************** id: NULL select_type: UNION RESULT table: <union1,2> type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: NULL Extra: Using temporary UNION ALL is an option
  • 48. www.percona.com GROUP BY: Loose index scan •  Loose index scan: •  considers only a fraction of the keys in an index •  Following rules apply: •  The query is over a single table. •  The GROUP BY columns should form a leftmost prefix of the index •  The only aggregate functions = MIN() and MAX(), same column
  • 49. www.percona.com Loose index scan example mysql> alter table ontime_2012 add key loose_index_scan (Carrier, dayofweek, DepDelayMinutes); mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek > 5 group by Carrier, dayofweekG ... table: ontime_2012 type: range possible_keys: NULL key: loose_index_scan key_len: 5 ref: NULL rows: 201 Extra: Using where; Using index for group-by Loose index scan Very fast “Range” works!
  • 50. www.percona.com Loose index scan vs. tight index scan Table: ontime_2012, 6M rows, data: 2G, index: 210M CREATE TABLE ontime_2012 ( YearD int(11) DEFAULT NULL, MonthD tinyint(4) DEFAULT NULL, DayofMonth tinyint(4) DEFAULT NULL, DayOfWeek tinyint(4) DEFAULT NULL, Carrier char(2) DEFAULT NULL, Origin char(5) DEFAULT NULL, DepDelayMinutes int(11) DEFAULT NULL, ... KEY loose_index_scan (Carrier,DayOfWeek,DepDelayMinutes), KEY covered (DayOfWeek,Carrier,DepDelayMinutes) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
  • 51. www.percona.com Loose index scan vs. tight index scan Loose index scan select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 where dayofweek = 5 group by Carrier, dayofweek table: ontime_2012 type: range possible_keys: covered key: loose_index_scan key_len: 5 ref: NULL rows: 201 Extra: Using where; Using index for group-by mysql> select ... +------+---------+-----------+ | ddm | Carrier | dayofweek | +------+---------+-----------+ | 1606 | AA | 7 | .. 30 rows in set (0.00 sec) Carrier, DayOfWeek, DepDelayMinutes
  • 52. www.percona.com Loose index scan vs. tight index scan Results 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Loose index scan Tight Index Scan Temporary table
  • 53. www.percona.com Where loose index scan is not supported •  AVG() + Group By – loose index scan is not supported mysql> explain select avg(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 where dayofweek >5 group by Carrier, dayofweek G table: ontime_2012 type: range key: covered key_len: 2 ref: NULL rows: 2961617 Extra: Using where; Using index; Using temporary; Using filesort mysql> select ... +---------+---------+-----------+ | ddm | Carrier | dayofweek | +---------+---------+-----------+ | 10.8564 | AA | 6 | ... 30 rows in set (1.39 sec) 1.  No loose index scan 2.  Filter by key 3.  Group by filesort
  • 55. www.percona.com mysql> explain select district, name, population from City where CountryCode = 'USA' order by population desc limit 10G table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using where; Using filesort
  • 56. www.percona.com mysql> alter table City add key my_sort2 (CountryCode, population); mysql> explain select district, name, population from City where CountryCode = 'USA' order by population desc limit 10G table: City type: ref key: my_sort2 key_len: 3 ref: const rows: 207 Extra: Using where No filesort
  • 57. www.percona.com mysql> alter table ontime_2012 add key (DepDelayMinutes); Query OK, 0 rows affected (38.68 sec) mysql> explain select * from ontime_2012 where dayofweek in (6,7) order by DepDelayMinutes desc limit 10G type: index possible_keys: DayOfWeek,covered key: DepDelayMinutes key_len: 5 ref: NULL rows: 24 Extra: Using where 10 rows in set (0.00 sec) 1.  Index is sorted 2.  Scan the whole table in the order of the index 3.  Filter results 4.  Stop after finding 10 rows matching the “where” condition
  • 58. www.percona.com If Index points to the beginning of the table (physically) = fast As it stops after 10 rows (LIMIT 10)
  • 59. www.percona.com If Index points to the end of table (physically) or random = slower Much more rows to scan (and skip)
  • 61. www.percona.com select t.*, c.name from title t join movie_companies m on t.id = m.movie_id join company_name c on m.company_id = c.id join company_type ct on m.company_type_id = ct.id where production_year > 1960 and ct.kind <> 'distributors’ order by production_year desc limit 10G ... 10 rows in set (0.00 sec) JOINs are for filtering Result set may contain duplicates DISTINCT?
  • 62. www.percona.com select DISTINCT t.*, c.name from title t join movie_companies m on t.id = m.movie_id join company_name c on m.company_id = c.id join company_type ct on m.company_type_id = ct.id where production_year > 1960 and ct.kind <> 'distributors’ order by production_year desc limit 10G ... 10 rows in set (37.27 sec)
  • 63. www.percona.com *************************** 1. row *************************** id: 1 select_type: SIMPLE table: t type: range possible_keys: PRIMARY,production_year key: production_year key_len: 5 ref: NULL rows: 1163888 Extra: Using index condition; Using temporary *************************** 2. row *************************** id: 1 select_type: SIMPLE table: m ...
  • 64. www.percona.com mysql> show status like 'created%'; +-------------------------+-------+ | Variable_name | Value | +-------------------------+-------+ | Created_tmp_disk_tables | 1 | | Created_tmp_files | 0 | | Created_tmp_tables | 1 | +-------------------------+-------+ 3 rows in set (0.00 sec)
  • 65. www.percona.com select distinct * from (select t.*, c.name from title t join movie_companies m on t.id = m.movie_id join company_name c on m.company_id = c.id join company_type ct on m.company_type_id = ct.id where production_year > 1960 and ct.kind <> 'distributors’ order by production_year desc limit 1000) as a limit 10; ... 10 rows in set (0.02 sec)