We all know how to define database indexes, but which indexes to define remains a mysterious art for most software developers. This talk will use general principles and specific scenarios to give you practical, step-by-step knowledge to turn a performance bottleneck into an epic win!

1. MENTOR
Your Indexes
Bill Karwin
Independent Oracle Users Group • 2010-9-21

2. Me
• Software developer
• C, Java, Perl, PHP, Ruby
• SQL maven
• Author of new book
SQL Antipatterns

3. “Whenever any result is sought, the
question will then arise—by what
course of calculation can these results
be arrived at by the machine in the
shortest time?”
— Charles Babbage, Passages from the
Life of a Philosopher (1864)

4. Indexes

5. Common blunders:
• Creating indexes naively
• Executing non-indexable queries
• Rejecting indexes because of overhead

6. CREATE TABLE Posts (
PostId
SERIAL PRIMARY KEY,
CreationDate
DATE NOT NULL,
Title
VARCHAR(80) NOT NULL,
Body
TEXT NOT NULL,
Score
INT
);

7. CREATE TABLE Posts (
PostId
SERIAL PRIMARY KEY,
CreationDate
DATE NOT NULL,
Title
VARCHAR(80) NOT NULL,
Body
TEXT NOT NULL,
Score
INT,
INDEX (PostId)
);
redundant index,
already in PK

8. CREATE TABLE Posts (
PostId
SERIAL PRIMARY KEY,
CreationDate
DATE NOT NULL,
Title
VARCHAR(80) NOT NULL,
Body
TEXT NOT NULL,
Score
INT,
INDEX (Title)
);
bulky index

9. CREATE TABLE Posts (
PostId
SERIAL PRIMARY KEY,
CreationDate
DATE NOT NULL,
Title
VARCHAR(80) NOT NULL,
Body
TEXT NOT NULL,
Score
INT,
INDEX (Score)
); unnecessary index,
we may never query on score

10. CREATE TABLE Posts (
PostId
SERIAL PRIMARY KEY,
CreationDate
DATE NOT NULL,
Title
VARCHAR(80) NOT NULL,
Body
TEXT NOT NULL,
Score
INT,
INDEX (Score, CreationDate, Title)
);
unnecessary
composite index

11. SELECT * FROM Posts
WHERE Title LIKE ‘%crash%’
non-leftmost
string match

12. Telephone book analogy:
• Easy to search for Dean Thomas:
uses index
to match
SELECT * FROM TelephoneBook
WHERE full_name LIKE ‘Thomas, %’
• Hard to search for Thomas Riddle: requires full
table scan
SELECT * FROM TelephoneBook
WHERE full_name LIKE ‘%, Thomas’

13. SELECT * FROM Posts
WHERE MONTH(CreationDate) = 4
function applied
to column

14. SELECT * FROM Users
WHERE LastName = ‘Thomas’
OR FirstName = ‘Thomas’
just like searching
for first_name

15. SELECT * FROM Users
ORDER BY FirstName, LastName
non-leftmost
composite key match

16. the benefit quickly
justifies the overhead
O(n) table scan
O(log n) index scan

17. Relational Index
data modeling optimization
is derived is derived
from data from queries

18. MENTOR Your
Indexes

19. Measure
Explain
Nominate
Test
Optimize
Repair

20. Measure
Explain
Nominate
Test
Optimize
Repair

21. • Profile your code to identify your biggest
performance costs.
• MySQL: PROFILER
• Oracle: TKPROF or Trace Analyzer
• Application-level profiling

22. Measure
Explain
Nominate
Test
Optimize
Repair

23. • Analyze the database’s optimization plan
for costly queries.
• Identify queries that don’t use indexes.

24. • MySQL: EXPLAIN Query
• “Explain Output Format”
http://dev.mysql.com/doc/refman/5.5/en/
explain-output.html

25. • Oracle: EXPLAIN PLAN Query
• “Understanding Explain Plan”
http://www.orafaq.com/node/1420

26. Measure
Explain
Nominate
Test
Optimize
Repair

27. • Which queries need optimization?
• Frequently used queries
• Very slow queries
• Reports

28. • Which column(s) need indexes?
• WHERE conditions
• JOIN conditions
• ORDER BY criteria
• MIN() / MAX()

29. • Automatic tools for nominating indexes:
• MySQL Enterprise Query Analyzer
• Oracle Automatic SQL Tuning Advisor

30. Measure
Explain
Nominate
Test
Optimize
Repair

31. • After creating index, measure your high-
priority queries again.
• Confirm that the new index made a
difference to these queries.
• Impress your boss/client!
“The new index gave us a 127%
performance improvement!”

32. Measure
Explain
Nominate
Test
Optimize
Repair

33. • Indexes are compact, frequently-used data
structures.
• Try to cache indexes in memory.

34. • Cache indexes in MySQL/InnoDB:
• Increase innodb_buffer_pool_size
• Used for both data and indexes
• Cache indexes in MySQL/MyISAM:
• Increase key_buffer_size
• LOAD INDEX INTO CACHE TableName
[INDEX IndexName];

35. • Cache indexes in Oracle:
ALTER SYSTEM SET DB_32K_CACHE_SIZE = 100m;
CREATE TABLESPACE INDEX_TS_32K
BLOCKSIZE 32K;
ALTER INDEX IndexName REBUILD ONLINE
TABLESPACE INDEX_TS_32K;
http://www.dba-oracle.com/art_so_optimizer_index_caching.htm

36. Measure
Explain
Nominate
Test
Optimize
Repair

37. • Indexes require periodic maintenance.
• Like a filesystem requires periodic
defragmentation.

38. • Analyze / rebuild indexes in MySQL:
• ANALYZE TABLE TableName
• OPTIMIZE TABLE TableName

39. • Analyze / rebuild indexes in Oracle:
• ANALYZE INDEX IndexName
• ALTER INDEX IndexName REBUILD ...

40. 1. Know Your Data.
2. Know Your Queries.
3. MENTOR Your Indexes.

41. SQL Antipatterns:
Avoiding the Pitfalls of
Database Programming
http://www.pragprog.com/titles/bksqla/

42. Copyright 2010 Bill Karwin
www.slideshare.net/billkarwin
Released under a Creative Commons 3.0 License:
http://creativecommons.org/licenses/by-nc-nd/3.0/
You are free to share - to copy, distribute and
transmit this work, under the following conditions:
Attribution. Noncommercial. No Derivative Works.
You must attribute this You may not use this work You may not alter,
work to Bill Karwin. for commercial purposes. transform, or build
upon this work.