An introduction to three of the new index features in MySQL 8

1. Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
New index features in MySQL 8
Erik Frøseth
Software Developer
MySQL Optimizer Team
February 1st, 2019

2. 2Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

3. 3Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Program Agenda
Functional indexes
Index skip scan
Invisible indexes
1
2
3
4
5
6
7

4. 4Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Program Agenda
Functional indexes
Index skip scan
Invisible indexes
1
2
3
4
5
6
7

5. 5Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Terminology
# This creates a single index/key with two index/key parts
CREATE INDEX my_index ON my_table (column_a, column_b);

6. 6Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – what is it?
● A functional index is an index where at least one of the index
parts is a function instead of a column.
1) CREATE INDEX my_index ON my_table ((ABS(column_a)));
2) CREATE INDEX my_index ON my_table
((ABS(column_a)), column_b);
3) CREATE TABLE my_table (
column_a VARCHAR(255),
INDEX my_index ((UPPER(column_a)))
);

7. 7Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – how do you use it?
● A functional index is created, used and removed exactly like any other index.
mysql> CREATE TABLE my_table (
-> column_a VARCHAR(255),
-> INDEX my_index ((UPPER(column_a)))
-> );
Query OK, 0 rows affected (0.04 sec)
mysql> EXPLAIN SELECT * FROM my_table WHERE UPPER(column_a) = "FOOBAR";
+-------------+----------+------+----------+---------+-------+------+----------+
| select_type | table | type | key | key_len | ref | rows | filtered |
+-------------+----------+------+----------+---------+-------+------+----------+
| SIMPLE | my_table | ref | my_index | 1023 | const | 1 | 100.00 |
+-------------+----------+------+----------+---------+-------+------+----------+
1 row in set, 1 warning (0.00 sec)

8. 8Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – how do you use it?
mysql> CREATE TABLE my_table (
-> column_a VARCHAR(255),
-> column_b INT,
-> INDEX my_index ((UPPER(column_a)), column_b)
-> );
Query OK, 0 rows affected (0.04 sec)
mysql> EXPLAIN SELECT * FROM my_table WHERE UPPER(column_a) = "FOOBAR" AND column_b < 20;
+-------------+----------+-------+---------------+----------+---------+------+----------+
| select_type | table | type | possible_keys | key | key_len | rows | filtered |
+-------------+----------+-------+---------------+----------+---------+------+----------+
| SIMPLE | my_table | range | my_index | my_index | 1028 | 1 | 100.00 |
+-------------+----------+-------+---------------+----------+---------+------+----------+
1 row in set, 1 warning (0.00 sec)
● You can mix functional and non-functional index parts

9. 9Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – observability
mysql> SHOW INDEX FROM t1;
+-------+------------+----------+--------------+-------------+...+-------------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name |...| Expression |
+-------+------------+----------+--------------+-------------+...+-------------------+
| t1 | 1 | idx | 1 | NULL |...| (`col1` + `col1`) |
+-------+------------+----------+--------------+-------------+...+-------------------+
1 row in set (0.01 sec)
mysql> SELECT * FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_NAME = "t1";
+...+------------+--------------+-------------+...+-------------------+
|...| INDEX_NAME | SEQ_IN_INDEX | COLUMN_NAME |...| EXPRESSION |
+...+------------+--------------+-------------+...+-------------------+
|...| idx | 1 | NULL |...| (`col1` + `col1`) |
+...+------------+--------------+-------------+...+-------------------+
1 row in set (0.00 sec)

10. 10Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – observability
mysql> SHOW CREATE TABLE t1;
+-------+-----------------------------------------------------------+
| Table | Create Table |
+-------+-----------------------------------------------------------+
| t1 | CREATE TABLE `t1` (
`col1` int(11) DEFAULT NULL,
KEY `idx` (((`col1` + `col1`)))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci |
+-------+----------------------------------------------------------+
1 row in set (0.00 sec)

11. 11Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – use cases
The main use case is indexing complex type such as
geometries and (especially) JSON.
CREATE INDEX
json_extract_idx
ON my_table
((CAST(data->>'$.person.id' AS CHAR(100))));
{
“person”:
{
“first_name”: “Erik”,
“last_name”: “Frøseth”,
“id”: 8
}
}

12. 12Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Functional index – limitations and other notes
● The primary key cannot be a functional index
● You cannot index a function that returns a BLOB or TEXT
● You can not index non-deterministic functions (RAND(),
NOW(), UUID() etc...)
● Each functional index part “uses” one column
● It’s more costly to maintain a functional index than a regular
index. Do not overuse it!

13. 13Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Program Agenda
Functional indexes
Index skip scan
Invisible indexes
1
2
3
4
5
6
7

14. 14Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
● Based on a contribution from Facebook
● An index can only be used if the first/leading column(s) is
referenced in the WHERE-condition
CREATE INDEX my_index ON my_table (column1, column2);
# Works!
SELECT * FROM my_table WHERE column1 = 33;
SELECT * FROM my_table WHERE column1 = 33 AND column2 > 3;
# Does not work...
SELECT * FROM my_table WHERE column2 > 3;

15. 15Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
● With index skip scan, the first index part can be omitted in the
WHERE condition
CREATE INDEX my_index ON my_table (column1, column2);
# Works!
SELECT * FROM my_table WHERE column1 = 33;
SELECT * FROM my_table WHERE column1 = 33 AND column2 > 3;
# This works too!
SELECT * FROM my_table WHERE column2 > 3;

16. 16Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
column1 column2
1 1
1 2
1 3
1 4
2 1
2 2
2 3
2 4
2 5
SELECT * FROM my_table WHERE column2 > 3;
column1 = 1 AND
column2 > 3
column1 = 2 AND
column2 > 3

17. 17Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
mysql> EXPLAIN SELECT * FROM my_table WHERE column2 > 3;
+----------+-------+------+------+----------+----------------------------------------+
| table | type | key | rows | filtered | Extra |
+----------+-------+------+------+----------+----------------------------------------+
| my_table | range | idx1 | 85 | 100.00 | Using where; Using index for skip scan |
+----------+-------+------+------+----------+----------------------------------------+
1 row in set, 1 warning (0.00 sec)

18. 18Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
●
Usage criterias:
– The query must only reference columns that exists in the in the index.
– The estimated cost must be the lowest of any alternative (many
distinct values in the first key part will most likely results in a high
cost).

19. 19Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
Few distinct values
"skip_scan_range": {
"potential_skip_scan_indexes": [{
"index": "idx2",
"tree_travel_cost": 0.45,
"num_groups": 12,
"rows": 138,
"cost": 31.817
}
]
}
Many distinct values
"skip_scan_range": {
"potential_skip_scan_indexes": [{
"index": "idx2",
"tree_travel_cost": 0.5,
"num_groups": 421,
"rows": 832,
"cost": 546.45
}
]
}

20. 20Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Index skip scan
●
Can be controlled by optimizer switch and query hints
1) SET optimizer_switch='skip_scan=off';
2) SELECT /*+ SKIP_SCAN(t3) */ col1 FROM t3 WHERE col2 < 10;
3) SELECT /*+ SKIP_SCAN(t3 idx1) */ col1 FROM t3 WHERE col2 < 10;
4) SELECT /*+ NO_SKIP_SCAN(t3) */ col1 FROM t3 WHERE col2 < 10;

21. 21Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Program Agenda
Functional indexes
Index skip scan
Invisible indexes
1
2
3
4
5
6
7

22. 22Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Invisible indexes – What is it?
● Lets you turn off any index
● Global setting
● Invisible to the optimizer, but it’s still maintained by the
storage engine
● Can be turned on/off completely by setting the optimizer
switch «use_invisible_indexes»
● Primary keys cannot be made invisible

23. 23Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Invisible indexes – How is it used?
1) ALTER TABLE t1 ALTER INDEX idx INVISIBLE;
2) CREATE INDEX idx ON my_table (a_column) INVISIBLE;
3) CREATE TABLE t1 (
col1 INT,
INDEX idx (col1) INVISIBLE
);

24. 24Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Invisible indexes - Example
mysql> EXPLAIN SELECT COUNT(*) FROM lineitem WHERE l_suppkey = 7706;
...+------+---------------+-------------+---------+-------+------+----------+-------------+
...| type | possible_keys | key | key_len | ref | rows | filtered | Extra |
...+------+---------------+-------------+---------+-------+------+----------+-------------+
...| ref | i_l_suppkey | i_l_suppkey | 5 | const | 604 | 100.00 | Using index |
...+------+---------------+-------------+---------+-------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
mysql> ALTER TABLE lineitem ALTER INDEX i_l_suppkey INVISIBLE;
Query OK, 0 rows affected (0.18 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> EXPLAIN SELECT COUNT(*) FROM lineitem WHERE l_suppkey = 7706;
...+------+---------------+------+---------+------+---------+----------+-------------+
...| type | possible_keys | key | key_len | ref | rows | filtered | Extra |
...+------+---------------+------+---------+------+---------+----------+-------------+
...| ALL | NULL | NULL | NULL | NULL | 5941264 | 0.01 | Using where |
...+------+---------------+------+---------+------+---------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

25. 25Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Invisible indexes - Observability
● See the visibility using INFORMATION_SCHEMA or SHOW INDEXES
mysql> SHOW INDEXES FROM my_table;
+----------+------------+---------------------+...+---------+
| Table | Non_unique | Key_name |...| Visible |
+----------+------------+---------------------+...+---------+
| my_table | 1 | just_to_be_safe_idx |...| NO |
+----------+------------+---------------------+...+---------+
mysql> SELECT INDEX_NAME, IS_VISIBLE FROM INFORMATION_SCHEMA.STATISTICS
> WHERE TABLE_NAME = "my_table";
+---------------------+------------+
| INDEX_NAME | IS_VISIBLE |
+---------------------+------------+
| just_to_be_safe_idx | NO |
+---------------------+------------+
1 row in set (0.01 sec)

26. 26Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Invisible indexes - Observability
mysql> SHOW CREATE TABLE t1;
+-------+----------------------------------------------------------+
| Table | Create Table |
+-------+----------------------------------------------------------+
| t1 | CREATE TABLE `t1` (
`col1` int(11) DEFAULT NULL,
KEY `just_to_be_safe_index` (((`col1` * 2))) /*!80000 INVISIBLE */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci |
+-------+----------------------------------------------------------+

27. 27Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Invisible indexes – what you should remember
●
See if you get any errors due to index hints referring to the
index
●
Slow query log
●
Performance schema
●
Application performance
●
Check sys.schema_unused_indexes

28. 28Copyright © 2019 Oracle and/or its affiliates. All rights reserved.

29. 29Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.