Indexes: The neglected performance all rounder

Markus Winand
Markus WinandTrainer, Coach, Author um http://winand.at/
© 2013 by Markus Winand
Indexes
The Neglected Performance
All-Rounder
Not
always that obvious,
unfortunately!
iStockPhoto
wildpixel
Takeaway #1: Pandemic Scale
It affects you!
(Symbolic image; not real data)
http://upload.wikimedia.org/wikipedia/commons/c/c7/2009_world_subdivisions_flu_pandemic.png
Takeaway #2: Caused by Success
Copyright © 2013 Telerik, Inc. All rights reserved
Takeaway #3: It’s Not Your Fault
http://simpsonswiki.com/wiki/File:I_Didn%27t_Do_It!_Volume_III.png
© 2013 by Markus Winand
The Problem
Improper Index Use
The Problem: Improper Index Use
“A very common cause of performance
problems is lack of proper indexes or the
use of queries that are not using
existing indexes.”
—Buda Consulting
http://www.budaconsulting.com/Portals/52677/docs/top_5_tech_brief.pdf
The Problem: Improper Index Use
“A very common cause of performance
problems is lack of proper indexes or the
use of queries that are not using
existing indexes.”
—Buda Consulting
http://www.budaconsulting.com/Portals/52677/docs/top_5_tech_brief.pdf
Quantifying the Problem
Percona White Paper:
Reasons of performance problems
that caused production downtime:
38% bad SQL
15% schema and indexing
http://www.percona.com/files/white-papers/causes-of-downtime-in-mysql.pdf
Quantifying the Problem
Survey by sqlskills.com:
Root causes of the last few SQL
Server performance problems:
27% T-SQL
19% Poor indexing
http://www.sqlskills.com/blogs/paul/survey-what-are-the-most-common-causes-of-performance-problems/
Quantifying the Problem
Craig S. Mullins (strategist and researcher):
„As much as 75% of poor relational performance
is caused by "bad" SQL and application code.”
Noel Yuhanna (Forrester Research):
„The key difficulties surrounding performance
continue to be poorly written SQL statements,
improper DBMS configuration and a lack of clear
understanding of how to tune databases to solve
performance issues.”
Quantifying the Problem
My observation:
~50% of SQL performance problems
are caused by improper index use
© 2013 by Markus Winand
The Root Cause
© 2013 by Markus Winand
The Root Cause
Admins are Indexing
The Root Cause: DBAs are Indexing
How did databases
work before SQL?
The Root Cause: DBAs are Indexing
Index use was intrinsically
tied to the queries.
The Root Cause: DBAs are Indexing
Example: dBase
Developers had to...
...use indexes explicitly when searching:
!"#$%&'"($#)$*+!#,&+-"
$$$.%&'$/%&+&'
...take care of index maintenance:
!"#$%&'"($#)$*+!#,&+-"0$%'(1
$$$$+22"&'
The Root Cause: DBAs are Indexing
SQL is an abstraction that only
defines the logical view.
The actual SQL implementation
takes care of everything else.
The Root Cause: DBAs are Indexing
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
SQL (language)
has:
SQL Databases (software)
have:
The Root Cause: DBAs are Indexing
Backup
& recovery
Storage
management
Bugs &
patches
Tuning
parameters
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
SQL (language)
has:
SQL Databases (software)
have:
High
Availability
The Root Cause: DBAs are Indexing
Indexes
Backup
& recovery
Storage
management
Bugs &
patches
Tuning
parameters
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
SQL (language)
has:
SQL Databases (software)
have:
High
Availability
The Root Cause: DBAs are Indexing
Indexes
Backup
& recovery
Storage
management
Bugs &
patches
Tuning
parameters
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
SQL Databases (software)
have:
Developers
High
Availability
The Root Cause: DBAs are Indexing
Indexes
Backup
& recovery
Storage
management
Bugs &
patches
Tuning
parameters
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
Developers Administrators
High
Availability
The Root Cause: DBAs are Indexing
Indexing is considered a system
tuning task that belongs to the
administrators responsibilities.
The Root Cause: DBAs are Indexing
A misconception that causes new problems:
The Root Cause: DBAs are Indexing
A misconception that causes new problems:
DBAs don’t know
the queries
Have to “investigate”
to find the queries.
It is time consuming and
almost always incomplete.
by G-10gian82
deviantart.com
The Root Cause: DBAs are Indexing
A misconception that causes new problems:
DBAs don’t know
the queries
Have to “investigate”
to find the queries.
It is time consuming and
almost always incomplete.
DBAs can’t change
the queries
Can make the index
match the query.
Can’t make the query
match the index!
© 2013 by Markus Winand
The Solution
© 2013 by Markus Winand
The Solution
Indexing is a
Development Task
The Solution: It’s a Dev Task
Indexes
Backup
& recovery
Storage
management
Tuning
parameters
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
Developers Administrators
High
Availability
Bugs &
patches
The Solution: It’s a Dev Task
Indexes
Backup
& recovery
Storage
management
Tuning
parameters
Transactions
Constraints
Views
Tables
Data
manipulation
Queries
Developers Administrators
Must match!
High
Availability
Bugs &
patches
Another Problem: It’s not Taught
Indexes are not part of the pure SQL (language) literature
because indexes are not part of the SQL standard.
11 SQL books analyzed: only 1.0% of the pages are
about indexes (70 out of 7330 pages).
Examples:
Oracle SQL by Example: 2.0% (19/960)
Beginning DBs with PostgreSQL: 0.8% (5/664)
Learning SQL: 3.3% (11/336 — highest rate in class)
Another Problem: It’s not Taught
Proper index usage is sometimes covered in database
tuning books but is always buried between hundreds of
pages of HW, OS and DB parameterization topics.
14 database administration books analyzed: 5.1% of the
pages are about indexes (307 out of 6069 pages).
Examples:
Oracle Performance Survival Guide: 5.2% (38/730)
High Performance MySQL: 8% (55/684)
PostgreSQL 9 High Performance: 5.8% (27/468)
Another Problem: It’s not Taught
Consequence:
Developers don’t know how to use
indexes properly.
Another Problem: It’s not Taught
Consequence:
Developers don’t know how to use
indexes properly.
Results of the 3-minute online quiz:
http://use-the-index-luke.com/3-minute-test
5 questions: each about a specific index
usage pattern.
Non-representative!
Q1: Good or Bad? (Function use)
345675$89:5;$#<*,%'($=9$#<*$>!"#$%&'()*+?@
A5B537$#"(#0$'+#",C)*D-&
$$E4=F$#<*
$/G545$,-%./012!"#$%&'()*+0$HIIIIH?$J$H1KLMH@
3-Minute Quiz: Indexing Skills
Q1: Good or Bad? (Function use)
345675$89:5;$#<*,%'($=9$#<*$>!"#$%&'()*+?@
A5B537$#"(#0$'+#",C)*D-&
$$E4=F$#<*
$/G545$,-%./012!"#$%&'()*+0$HIIIIH?$J$H1KLMH@
 


3-Minute Quiz: Indexing Skills
Q1: Good or Bad? (Function use)
345675$89:5;$#*,%'($=9$#*$!#$%'()*+?@
A5B537$#(#0$'+#,C)*D-
$$E4=F$#*
$/G545$,-%./012!#$%'()*+0$HIIIIH?$J$H1KLMH@
3-Minute Quiz: Indexing Skills
3-Minute Quiz: Indexing Skills
Q2: Good or Bad? (Indexed Top-N, no IOS)
345675$89:5;$#*,%'($=9$#*$+0$'+#,C)*?@
A5B537$%'0$+0$'+#,C)*
$$E4=F$#*
$/G545$+$J$NL
$=4:54$OI$'+#,C)*$:5A3
$3454,67@
3-Minute Quiz: Indexing Skills
Q2: Good or Bad? (Indexed Top-N, no IOS)
345675$89:5;$#*,%'($=9$#*$+0$'+#,C)*?@
A5B537$%'0$+0$'+#,C)*
$$E4=F$#*
$/G545$+$J$NL
$=4:54$OI$'+#,C)*$:5A3
$3454,67@

 


Understandable
controversy!
3-Minute Quiz: Indexing Skills
Q3: Good or Bad? (Column order)
CREATE INDEX tbl_idx ON tbl (a, b);
SELECT id, a, b FROM tbl
WHERE a = $1 AND b = $2;
SELECT id, a, b FROM tbl
WHERE b = $1;
3-Minute Quiz: Indexing Skills
Q3: Good or Bad? (Column order)
CREATE INDEX tbl_idx ON tbl (a, b);
SELECT id, a, b FROM tbl
WHERE a = $1 AND b = $2;
SELECT id, a, b FROM tbl
WHERE b = $1;
3-Minute Quiz: Indexing Skills
Q4: Good or Bad? (Indexing LIKE)
CREATE INDEX tbl_idx
ON tbl (text varchar_pattern_ops);
SELECT id, text
FROM tbl
WHERE text LIKE '%TERM%';
3-Minute Quiz: Indexing Skills
Q4: Good or Bad? (Indexing LIKE)
CREATE INDEX tbl_idx
ON tbl (text varchar_pattern_ops);
SELECT id, text
FROM tbl
WHERE text LIKE '%TERM%';
3-Minute Quiz: Indexing Skills
Q5: Good or Bad? (equality vs. ranges)
CREATE INDEX tbl_idx
ON tbl (date_col, state);
SELECT id, date_col, state FROM tbl
WHERE date_col =
CURRENT_DATE - INTERVAL '5' YEAR
AND state = 'X';
3-Minute Quiz: Indexing Skills
Q5: Good or Bad? (equality vs. ranges)
CREATE INDEX tbl_idx
ON tbl (date_col, state);
SELECT id, date_col, state FROM tbl
WHERE date_col =
CURRENT_DATE - INTERVAL '5' YEAR
AND state = 'X';
Indexes: The Neglected All-Rounder
Everybody knows indexing is
important for performance,
yet nobody takes the time to
learn and apply is properly.
Indexes: The Neglected All-Rounder
Index details are hardly known.
! “Details” like column-order or equality vs. range
conditions must be learned and understood.
Only one index capability is used: finding data quickly
! Indexes have three capabilities (powers):
finding data, clustering data, and sorting data.
Indexing is done from single query perspective.
! Should be done from application perspective
(considering all queries). It’s a design task!
Indexes: The Neglected All-Rounder
Are you just adding indexes
or
are you designing indexes?
About Markus Winand
Tuning developers for
high SQL performance
Training  co (one-man show):
winand.at
Geeky blog:
use-the-index-luke.com
Author of:
SQL Performance Explained
1 von 49

Recomendados

How a Developer can Troubleshoot a SQL performing poorly on a Production DB von
How a Developer can Troubleshoot a SQL performing poorly on a Production DBHow a Developer can Troubleshoot a SQL performing poorly on a Production DB
How a Developer can Troubleshoot a SQL performing poorly on a Production DBCarlos Sierra
12.8K views61 Folien
Exploring Oracle Database Performance Tuning Best Practices for DBAs and Deve... von
Exploring Oracle Database Performance Tuning Best Practices for DBAs and Deve...Exploring Oracle Database Performance Tuning Best Practices for DBAs and Deve...
Exploring Oracle Database Performance Tuning Best Practices for DBAs and Deve...Aaron Shilo
2.1K views120 Folien
OrientDB for real & Web App development von
OrientDB for real & Web App developmentOrientDB for real & Web App development
OrientDB for real & Web App developmentLuca Garulli
8.4K views55 Folien
How to upgrade like a boss to MySQL 8.0 - PLE19 von
How to upgrade like a boss to MySQL 8.0 -  PLE19How to upgrade like a boss to MySQL 8.0 -  PLE19
How to upgrade like a boss to MySQL 8.0 - PLE19Alkin Tezuysal
402 views48 Folien
Using Apache Hive with High Performance von
Using Apache Hive with High PerformanceUsing Apache Hive with High Performance
Using Apache Hive with High PerformanceInderaj (Raj) Bains
5.1K views50 Folien
Advanced Sql Training von
Advanced Sql TrainingAdvanced Sql Training
Advanced Sql Trainingbixxman
10.8K views70 Folien

Más contenido relacionado

Was ist angesagt?

PHP mysql Mysql joins von
PHP mysql  Mysql joinsPHP mysql  Mysql joins
PHP mysql Mysql joinsMudasir Syed
2.6K views14 Folien
Amazon Redshift: Performance Tuning and Optimization von
Amazon Redshift: Performance Tuning and OptimizationAmazon Redshift: Performance Tuning and Optimization
Amazon Redshift: Performance Tuning and OptimizationAmazon Web Services
15.6K views46 Folien
PostgreSQL HA von
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HAharoonm
1.3K views28 Folien
PostgreSQL Tutorial For Beginners | Edureka von
PostgreSQL Tutorial For Beginners | EdurekaPostgreSQL Tutorial For Beginners | Edureka
PostgreSQL Tutorial For Beginners | EdurekaEdureka!
1.1K views56 Folien
Microsoft Azure Databricks von
Microsoft Azure DatabricksMicrosoft Azure Databricks
Microsoft Azure DatabricksSascha Dittmann
1.2K views17 Folien
PostgreSql query planning and tuning von
PostgreSql query planning and tuningPostgreSql query planning and tuning
PostgreSql query planning and tuningFederico Campoli
3K views49 Folien

Was ist angesagt?(20)

PHP mysql Mysql joins von Mudasir Syed
PHP mysql  Mysql joinsPHP mysql  Mysql joins
PHP mysql Mysql joins
Mudasir Syed2.6K views
Amazon Redshift: Performance Tuning and Optimization von Amazon Web Services
Amazon Redshift: Performance Tuning and OptimizationAmazon Redshift: Performance Tuning and Optimization
Amazon Redshift: Performance Tuning and Optimization
Amazon Web Services15.6K views
PostgreSQL HA von haroonm
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
haroonm1.3K views
PostgreSQL Tutorial For Beginners | Edureka von Edureka!
PostgreSQL Tutorial For Beginners | EdurekaPostgreSQL Tutorial For Beginners | Edureka
PostgreSQL Tutorial For Beginners | Edureka
Edureka!1.1K views
State transfer With Galera von Mydbops
State transfer With GaleraState transfer With Galera
State transfer With Galera
Mydbops216 views
Postgresql Database Administration- Day3 von PoguttuezhiniVP
Postgresql Database Administration- Day3Postgresql Database Administration- Day3
Postgresql Database Administration- Day3
PoguttuezhiniVP73 views
Big Data Processing With Spark von Edureka!
Big Data Processing With SparkBig Data Processing With Spark
Big Data Processing With Spark
Edureka!1.6K views
Oracle database 12c sql worshop 1 student guide vol 2 von Otto Paiz
Oracle database 12c sql worshop 1 student guide vol 2Oracle database 12c sql worshop 1 student guide vol 2
Oracle database 12c sql worshop 1 student guide vol 2
Otto Paiz12.7K views
SQL window functions for MySQL von Dag H. Wanvik
SQL window functions for MySQLSQL window functions for MySQL
SQL window functions for MySQL
Dag H. Wanvik12.4K views
MySQL Query And Index Tuning von Manikanda kumar
MySQL Query And Index TuningMySQL Query And Index Tuning
MySQL Query And Index Tuning
Manikanda kumar17.9K views
Top 10 Oracle SQL tuning tips von Nirav Shah
Top 10 Oracle SQL tuning tipsTop 10 Oracle SQL tuning tips
Top 10 Oracle SQL tuning tips
Nirav Shah3.8K views
Understanding the architecture of MariaDB ColumnStore von MariaDB plc
Understanding the architecture of MariaDB ColumnStoreUnderstanding the architecture of MariaDB ColumnStore
Understanding the architecture of MariaDB ColumnStore
MariaDB plc2.8K views
Your tuning arsenal: AWR, ADDM, ASH, Metrics and Advisors von John Kanagaraj
Your tuning arsenal: AWR, ADDM, ASH, Metrics and AdvisorsYour tuning arsenal: AWR, ADDM, ASH, Metrics and Advisors
Your tuning arsenal: AWR, ADDM, ASH, Metrics and Advisors
John Kanagaraj6K views
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang von Databricks
Apache Spark Data Source V2 with Wenchen Fan and Gengliang WangApache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Databricks5.9K views

Similar a Indexes: The neglected performance all rounder

Top-10-Java-Performance-Problems.pdf von
Top-10-Java-Performance-Problems.pdfTop-10-Java-Performance-Problems.pdf
Top-10-Java-Performance-Problems.pdfKiranChinnagangannag
11 views29 Folien
08 von
0808
08Traci Webb
3 views49 Folien
Chapter 5 data processing von
Chapter 5 data processingChapter 5 data processing
Chapter 5 data processingUMaine
6.7K views14 Folien
Designing for Performance: Database Related Worst Practices von
Designing for Performance: Database Related Worst PracticesDesigning for Performance: Database Related Worst Practices
Designing for Performance: Database Related Worst PracticesChristian Antognini
1.8K views24 Folien
IT 655 Milestone One Guidelines and Rubric Presenta.docx von
 IT 655 Milestone One Guidelines and Rubric   Presenta.docx IT 655 Milestone One Guidelines and Rubric   Presenta.docx
IT 655 Milestone One Guidelines and Rubric Presenta.docxaryan532920
3 views14 Folien
Best Laid Plans: Saving Time, Money and Trouble with Optimal Forecasting von
Best Laid Plans: Saving Time, Money and Trouble with Optimal ForecastingBest Laid Plans: Saving Time, Money and Trouble with Optimal Forecasting
Best Laid Plans: Saving Time, Money and Trouble with Optimal ForecastingEric Kavanagh
118 views33 Folien

Similar a Indexes: The neglected performance all rounder(20)

Chapter 5 data processing von UMaine
Chapter 5 data processingChapter 5 data processing
Chapter 5 data processing
UMaine6.7K views
Designing for Performance: Database Related Worst Practices von Christian Antognini
Designing for Performance: Database Related Worst PracticesDesigning for Performance: Database Related Worst Practices
Designing for Performance: Database Related Worst Practices
Christian Antognini1.8K views
IT 655 Milestone One Guidelines and Rubric Presenta.docx von aryan532920
 IT 655 Milestone One Guidelines and Rubric   Presenta.docx IT 655 Milestone One Guidelines and Rubric   Presenta.docx
IT 655 Milestone One Guidelines and Rubric Presenta.docx
aryan5329203 views
Best Laid Plans: Saving Time, Money and Trouble with Optimal Forecasting von Eric Kavanagh
Best Laid Plans: Saving Time, Money and Trouble with Optimal ForecastingBest Laid Plans: Saving Time, Money and Trouble with Optimal Forecasting
Best Laid Plans: Saving Time, Money and Trouble with Optimal Forecasting
Eric Kavanagh118 views
Advanced Database Management SystemsQuestion # 1It is normally.docx von nettletondevon
Advanced Database Management SystemsQuestion # 1It is normally.docxAdvanced Database Management SystemsQuestion # 1It is normally.docx
Advanced Database Management SystemsQuestion # 1It is normally.docx
nettletondevon4 views
Software Modeling And Analysis Project von Pam Fenno
Software Modeling And Analysis ProjectSoftware Modeling And Analysis Project
Software Modeling And Analysis Project
Pam Fenno4 views
Database Performance Improvements Environment Document Requirement Planning A... von SlideTeam
Database Performance Improvements Environment Document Requirement Planning A...Database Performance Improvements Environment Document Requirement Planning A...
Database Performance Improvements Environment Document Requirement Planning A...
SlideTeam16 views
Sybase job interview_preparation_guide von NV Suresh Kumar
Sybase job interview_preparation_guideSybase job interview_preparation_guide
Sybase job interview_preparation_guide
NV Suresh Kumar3.2K views
Ibm web sphere_job_interview_preparation_guide von Khemnath Chauhan
Ibm web sphere_job_interview_preparation_guideIbm web sphere_job_interview_preparation_guide
Ibm web sphere_job_interview_preparation_guide
Khemnath Chauhan6.6K views
Government and Education Webinar: Simplify Your Database Performance Manageme... von SolarWinds
Government and Education Webinar: Simplify Your Database Performance Manageme...Government and Education Webinar: Simplify Your Database Performance Manageme...
Government and Education Webinar: Simplify Your Database Performance Manageme...
SolarWinds373 views
Design Process And Generating Model Of A Knuckle Joint von Suzanne Horn
Design Process And Generating Model Of A Knuckle JointDesign Process And Generating Model Of A Knuckle Joint
Design Process And Generating Model Of A Knuckle Joint
Suzanne Horn3 views
Application Software And Relational Databases von Jessica Oatis
Application Software And Relational DatabasesApplication Software And Relational Databases
Application Software And Relational Databases
Jessica Oatis2 views

Más de Markus Winand

Standard SQL features where PostgreSQL beats its competitors von
Standard SQL features where PostgreSQL beats its competitorsStandard SQL features where PostgreSQL beats its competitors
Standard SQL features where PostgreSQL beats its competitorsMarkus Winand
58.9K views77 Folien
Four* Major Database Releases of 2017 in Review von
Four* Major Database Releases of 2017 in ReviewFour* Major Database Releases of 2017 in Review
Four* Major Database Releases of 2017 in ReviewMarkus Winand
1.5K views29 Folien
Row Pattern Matching in SQL:2016 von
Row Pattern Matching in SQL:2016Row Pattern Matching in SQL:2016
Row Pattern Matching in SQL:2016Markus Winand
109.4K views92 Folien
SQL Transactions - What they are good for and how they work von
SQL Transactions - What they are good for and how they workSQL Transactions - What they are good for and how they work
SQL Transactions - What they are good for and how they workMarkus Winand
33.9K views72 Folien
Backend to Frontend: When database optimization affects the full stack von
Backend to Frontend: When database optimization affects the full stackBackend to Frontend: When database optimization affects the full stack
Backend to Frontend: When database optimization affects the full stackMarkus Winand
3.7K views52 Folien
Modern SQL in Open Source and Commercial Databases von
Modern SQL in Open Source and Commercial DatabasesModern SQL in Open Source and Commercial Databases
Modern SQL in Open Source and Commercial DatabasesMarkus Winand
450.8K views174 Folien

Más de Markus Winand(9)

Standard SQL features where PostgreSQL beats its competitors von Markus Winand
Standard SQL features where PostgreSQL beats its competitorsStandard SQL features where PostgreSQL beats its competitors
Standard SQL features where PostgreSQL beats its competitors
Markus Winand58.9K views
Four* Major Database Releases of 2017 in Review von Markus Winand
Four* Major Database Releases of 2017 in ReviewFour* Major Database Releases of 2017 in Review
Four* Major Database Releases of 2017 in Review
Markus Winand1.5K views
Row Pattern Matching in SQL:2016 von Markus Winand
Row Pattern Matching in SQL:2016Row Pattern Matching in SQL:2016
Row Pattern Matching in SQL:2016
Markus Winand109.4K views
SQL Transactions - What they are good for and how they work von Markus Winand
SQL Transactions - What they are good for and how they workSQL Transactions - What they are good for and how they work
SQL Transactions - What they are good for and how they work
Markus Winand33.9K views
Backend to Frontend: When database optimization affects the full stack von Markus Winand
Backend to Frontend: When database optimization affects the full stackBackend to Frontend: When database optimization affects the full stack
Backend to Frontend: When database optimization affects the full stack
Markus Winand3.7K views
Modern SQL in Open Source and Commercial Databases von Markus Winand
Modern SQL in Open Source and Commercial DatabasesModern SQL in Open Source and Commercial Databases
Modern SQL in Open Source and Commercial Databases
Markus Winand450.8K views
Volkskrankheit "Stiefmuetterliche Indizierung" von Markus Winand
Volkskrankheit "Stiefmuetterliche Indizierung"Volkskrankheit "Stiefmuetterliche Indizierung"
Volkskrankheit "Stiefmuetterliche Indizierung"
Markus Winand16K views
SQL Performance - Vienna System Architects Meetup 20131202 von Markus Winand
SQL Performance - Vienna System Architects Meetup 20131202SQL Performance - Vienna System Architects Meetup 20131202
SQL Performance - Vienna System Architects Meetup 20131202
Markus Winand1.8K views
Pagination Done the Right Way von Markus Winand
Pagination Done the Right WayPagination Done the Right Way
Pagination Done the Right Way
Markus Winand415.4K views

Último

Setting Up Your First CloudStack Environment with Beginners Challenges - MD R... von
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...ShapeBlue
173 views15 Folien
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue von
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlueElevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlueShapeBlue
222 views7 Folien
Why and How CloudStack at weSystems - Stephan Bienek - weSystems von
Why and How CloudStack at weSystems - Stephan Bienek - weSystemsWhy and How CloudStack at weSystems - Stephan Bienek - weSystems
Why and How CloudStack at weSystems - Stephan Bienek - weSystemsShapeBlue
238 views13 Folien
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit... von
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...ShapeBlue
159 views25 Folien
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading... von
Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading...The Digital Insurer
90 views52 Folien
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue von
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueWhat’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueShapeBlue
263 views23 Folien

Último(20)

Setting Up Your First CloudStack Environment with Beginners Challenges - MD R... von ShapeBlue
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
ShapeBlue173 views
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue von ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlueElevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
ShapeBlue222 views
Why and How CloudStack at weSystems - Stephan Bienek - weSystems von ShapeBlue
Why and How CloudStack at weSystems - Stephan Bienek - weSystemsWhy and How CloudStack at weSystems - Stephan Bienek - weSystems
Why and How CloudStack at weSystems - Stephan Bienek - weSystems
ShapeBlue238 views
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit... von ShapeBlue
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
ShapeBlue159 views
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading... von The Digital Insurer
Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading...
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue von ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueWhat’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
ShapeBlue263 views
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T von ShapeBlue
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
ShapeBlue152 views
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N... von James Anderson
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
James Anderson160 views
State of the Union - Rohit Yadav - Apache CloudStack von ShapeBlue
State of the Union - Rohit Yadav - Apache CloudStackState of the Union - Rohit Yadav - Apache CloudStack
State of the Union - Rohit Yadav - Apache CloudStack
ShapeBlue297 views
KVM Security Groups Under the Hood - Wido den Hollander - Your.Online von ShapeBlue
KVM Security Groups Under the Hood - Wido den Hollander - Your.OnlineKVM Security Groups Under the Hood - Wido den Hollander - Your.Online
KVM Security Groups Under the Hood - Wido den Hollander - Your.Online
ShapeBlue221 views
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT von ShapeBlue
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBITUpdates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT
ShapeBlue206 views
Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P... von ShapeBlue
Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P...Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P...
Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P...
ShapeBlue194 views
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue von ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlueCloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
ShapeBlue138 views
Digital Personal Data Protection (DPDP) Practical Approach For CISOs von Priyanka Aash
Digital Personal Data Protection (DPDP) Practical Approach For CISOsDigital Personal Data Protection (DPDP) Practical Approach For CISOs
Digital Personal Data Protection (DPDP) Practical Approach For CISOs
Priyanka Aash158 views
Extending KVM Host HA for Non-NFS Storage - Alex Ivanov - StorPool von ShapeBlue
Extending KVM Host HA for Non-NFS Storage -  Alex Ivanov - StorPoolExtending KVM Host HA for Non-NFS Storage -  Alex Ivanov - StorPool
Extending KVM Host HA for Non-NFS Storage - Alex Ivanov - StorPool
ShapeBlue123 views
DRBD Deep Dive - Philipp Reisner - LINBIT von ShapeBlue
DRBD Deep Dive - Philipp Reisner - LINBITDRBD Deep Dive - Philipp Reisner - LINBIT
DRBD Deep Dive - Philipp Reisner - LINBIT
ShapeBlue180 views
Future of AR - Facebook Presentation von Rob McCarty
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentation
Rob McCarty64 views
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ... von ShapeBlue
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...
ShapeBlue126 views
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue von ShapeBlue
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlueMigrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
ShapeBlue218 views
Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And... von ShapeBlue
Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And...Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And...
Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And...
ShapeBlue106 views

Indexes: The neglected performance all rounder

  • 1. © 2013 by Markus Winand Indexes The Neglected Performance All-Rounder Not always that obvious, unfortunately! iStockPhoto wildpixel
  • 2. Takeaway #1: Pandemic Scale It affects you! (Symbolic image; not real data) http://upload.wikimedia.org/wikipedia/commons/c/c7/2009_world_subdivisions_flu_pandemic.png
  • 3. Takeaway #2: Caused by Success Copyright © 2013 Telerik, Inc. All rights reserved
  • 4. Takeaway #3: It’s Not Your Fault http://simpsonswiki.com/wiki/File:I_Didn%27t_Do_It!_Volume_III.png
  • 5. © 2013 by Markus Winand The Problem Improper Index Use
  • 6. The Problem: Improper Index Use “A very common cause of performance problems is lack of proper indexes or the use of queries that are not using existing indexes.” —Buda Consulting http://www.budaconsulting.com/Portals/52677/docs/top_5_tech_brief.pdf
  • 7. The Problem: Improper Index Use “A very common cause of performance problems is lack of proper indexes or the use of queries that are not using existing indexes.” —Buda Consulting http://www.budaconsulting.com/Portals/52677/docs/top_5_tech_brief.pdf
  • 8. Quantifying the Problem Percona White Paper: Reasons of performance problems that caused production downtime: 38% bad SQL 15% schema and indexing http://www.percona.com/files/white-papers/causes-of-downtime-in-mysql.pdf
  • 9. Quantifying the Problem Survey by sqlskills.com: Root causes of the last few SQL Server performance problems: 27% T-SQL 19% Poor indexing http://www.sqlskills.com/blogs/paul/survey-what-are-the-most-common-causes-of-performance-problems/
  • 10. Quantifying the Problem Craig S. Mullins (strategist and researcher): „As much as 75% of poor relational performance is caused by "bad" SQL and application code.” Noel Yuhanna (Forrester Research): „The key difficulties surrounding performance continue to be poorly written SQL statements, improper DBMS configuration and a lack of clear understanding of how to tune databases to solve performance issues.”
  • 11. Quantifying the Problem My observation: ~50% of SQL performance problems are caused by improper index use
  • 12. © 2013 by Markus Winand The Root Cause
  • 13. © 2013 by Markus Winand The Root Cause Admins are Indexing
  • 14. The Root Cause: DBAs are Indexing How did databases work before SQL?
  • 15. The Root Cause: DBAs are Indexing Index use was intrinsically tied to the queries.
  • 16. The Root Cause: DBAs are Indexing Example: dBase Developers had to... ...use indexes explicitly when searching: !"#$%&'"($#)$*+!#,&+-" $$$.%&'$/%&+&' ...take care of index maintenance: !"#$%&'"($#)$*+!#,&+-"0$%'(1 $$$$+22"&'
  • 17. The Root Cause: DBAs are Indexing SQL is an abstraction that only defines the logical view. The actual SQL implementation takes care of everything else.
  • 18. The Root Cause: DBAs are Indexing Transactions Constraints Views Tables Data manipulation Queries SQL (language) has: SQL Databases (software) have:
  • 19. The Root Cause: DBAs are Indexing Backup & recovery Storage management Bugs & patches Tuning parameters Transactions Constraints Views Tables Data manipulation Queries SQL (language) has: SQL Databases (software) have: High Availability
  • 20. The Root Cause: DBAs are Indexing Indexes Backup & recovery Storage management Bugs & patches Tuning parameters Transactions Constraints Views Tables Data manipulation Queries SQL (language) has: SQL Databases (software) have: High Availability
  • 21. The Root Cause: DBAs are Indexing Indexes Backup & recovery Storage management Bugs & patches Tuning parameters Transactions Constraints Views Tables Data manipulation Queries SQL Databases (software) have: Developers High Availability
  • 22. The Root Cause: DBAs are Indexing Indexes Backup & recovery Storage management Bugs & patches Tuning parameters Transactions Constraints Views Tables Data manipulation Queries Developers Administrators High Availability
  • 23. The Root Cause: DBAs are Indexing Indexing is considered a system tuning task that belongs to the administrators responsibilities.
  • 24. The Root Cause: DBAs are Indexing A misconception that causes new problems:
  • 25. The Root Cause: DBAs are Indexing A misconception that causes new problems: DBAs don’t know the queries Have to “investigate” to find the queries. It is time consuming and almost always incomplete. by G-10gian82 deviantart.com
  • 26. The Root Cause: DBAs are Indexing A misconception that causes new problems: DBAs don’t know the queries Have to “investigate” to find the queries. It is time consuming and almost always incomplete. DBAs can’t change the queries Can make the index match the query. Can’t make the query match the index!
  • 27. © 2013 by Markus Winand The Solution
  • 28. © 2013 by Markus Winand The Solution Indexing is a Development Task
  • 29. The Solution: It’s a Dev Task Indexes Backup & recovery Storage management Tuning parameters Transactions Constraints Views Tables Data manipulation Queries Developers Administrators High Availability Bugs & patches
  • 30. The Solution: It’s a Dev Task Indexes Backup & recovery Storage management Tuning parameters Transactions Constraints Views Tables Data manipulation Queries Developers Administrators Must match! High Availability Bugs & patches
  • 31. Another Problem: It’s not Taught Indexes are not part of the pure SQL (language) literature because indexes are not part of the SQL standard. 11 SQL books analyzed: only 1.0% of the pages are about indexes (70 out of 7330 pages). Examples: Oracle SQL by Example: 2.0% (19/960) Beginning DBs with PostgreSQL: 0.8% (5/664) Learning SQL: 3.3% (11/336 — highest rate in class)
  • 32. Another Problem: It’s not Taught Proper index usage is sometimes covered in database tuning books but is always buried between hundreds of pages of HW, OS and DB parameterization topics. 14 database administration books analyzed: 5.1% of the pages are about indexes (307 out of 6069 pages). Examples: Oracle Performance Survival Guide: 5.2% (38/730) High Performance MySQL: 8% (55/684) PostgreSQL 9 High Performance: 5.8% (27/468)
  • 33. Another Problem: It’s not Taught Consequence: Developers don’t know how to use indexes properly.
  • 34. Another Problem: It’s not Taught Consequence: Developers don’t know how to use indexes properly. Results of the 3-minute online quiz: http://use-the-index-luke.com/3-minute-test 5 questions: each about a specific index usage pattern. Non-representative!
  • 35. Q1: Good or Bad? (Function use) 345675$89:5;$#<*,%'($=9$#<*$>!"#$%&'()*+?@ A5B537$#"(#0$'+#",C)*D-& $$E4=F$#<* $/G545$,-%./012!"#$%&'()*+0$HIIIIH?$J$H1KLMH@ 3-Minute Quiz: Indexing Skills
  • 36. Q1: Good or Bad? (Function use) 345675$89:5;$#<*,%'($=9$#<*$>!"#$%&'()*+?@ A5B537$#"(#0$'+#",C)*D-& $$E4=F$#<* $/G545$,-%./012!"#$%&'()*+0$HIIIIH?$J$H1KLMH@ 3-Minute Quiz: Indexing Skills
  • 37. Q1: Good or Bad? (Function use) 345675$89:5;$#*,%'($=9$#*$!#$%'()*+?@ A5B537$#(#0$'+#,C)*D- $$E4=F$#* $/G545$,-%./012!#$%'()*+0$HIIIIH?$J$H1KLMH@ 3-Minute Quiz: Indexing Skills
  • 38. 3-Minute Quiz: Indexing Skills Q2: Good or Bad? (Indexed Top-N, no IOS) 345675$89:5;$#*,%'($=9$#*$+0$'+#,C)*?@ A5B537$%'0$+0$'+#,C)* $$E4=F$#* $/G545$+$J$NL $=4:54$OI$'+#,C)*$:5A3 $3454,67@
  • 39. 3-Minute Quiz: Indexing Skills Q2: Good or Bad? (Indexed Top-N, no IOS) 345675$89:5;$#*,%'($=9$#*$+0$'+#,C)*?@ A5B537$%'0$+0$'+#,C)* $$E4=F$#* $/G545$+$J$NL $=4:54$OI$'+#,C)*$:5A3 $3454,67@ Understandable controversy!
  • 40. 3-Minute Quiz: Indexing Skills Q3: Good or Bad? (Column order) CREATE INDEX tbl_idx ON tbl (a, b); SELECT id, a, b FROM tbl WHERE a = $1 AND b = $2; SELECT id, a, b FROM tbl WHERE b = $1;
  • 41. 3-Minute Quiz: Indexing Skills Q3: Good or Bad? (Column order) CREATE INDEX tbl_idx ON tbl (a, b); SELECT id, a, b FROM tbl WHERE a = $1 AND b = $2; SELECT id, a, b FROM tbl WHERE b = $1;
  • 42. 3-Minute Quiz: Indexing Skills Q4: Good or Bad? (Indexing LIKE) CREATE INDEX tbl_idx ON tbl (text varchar_pattern_ops); SELECT id, text FROM tbl WHERE text LIKE '%TERM%';
  • 43. 3-Minute Quiz: Indexing Skills Q4: Good or Bad? (Indexing LIKE) CREATE INDEX tbl_idx ON tbl (text varchar_pattern_ops); SELECT id, text FROM tbl WHERE text LIKE '%TERM%';
  • 44. 3-Minute Quiz: Indexing Skills Q5: Good or Bad? (equality vs. ranges) CREATE INDEX tbl_idx ON tbl (date_col, state); SELECT id, date_col, state FROM tbl WHERE date_col = CURRENT_DATE - INTERVAL '5' YEAR AND state = 'X';
  • 45. 3-Minute Quiz: Indexing Skills Q5: Good or Bad? (equality vs. ranges) CREATE INDEX tbl_idx ON tbl (date_col, state); SELECT id, date_col, state FROM tbl WHERE date_col = CURRENT_DATE - INTERVAL '5' YEAR AND state = 'X';
  • 46. Indexes: The Neglected All-Rounder Everybody knows indexing is important for performance, yet nobody takes the time to learn and apply is properly.
  • 47. Indexes: The Neglected All-Rounder Index details are hardly known. ! “Details” like column-order or equality vs. range conditions must be learned and understood. Only one index capability is used: finding data quickly ! Indexes have three capabilities (powers): finding data, clustering data, and sorting data. Indexing is done from single query perspective. ! Should be done from application perspective (considering all queries). It’s a design task!
  • 48. Indexes: The Neglected All-Rounder Are you just adding indexes or are you designing indexes?
  • 49. About Markus Winand Tuning developers for high SQL performance Training co (one-man show): winand.at Geeky blog: use-the-index-luke.com Author of: SQL Performance Explained