There are any number of tricks and traps around getting the query optimizer to provide you with an optimal execution plan that gets you your data quickly and efficiently. But, at the end of the day, the principal driving factor of the optimizer, and therefore of your queries, are the statistics that define your data. This session teaches you how those statistics are put together and maintained by SQL Server. Different types of maintenance results in different levels of accuracy within statistics so we detail what the structures and information looks like after this maintenance. Your understanding of how the optimizer works with statistics will better enable you to understand why you’re getting the performance and types of execution plans that you are getting. Understanding enables you to write better t-sql statements and deal with performance problems such as bad parameter sniffing.
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Statistics And the Query Optimizer
1. Statistics and
the Query Optimizer
Grant Fritchey
Product Evangelist
Red Gate Software
www.ScaryDBA.com
Grant Fritchey | www.ScaryDBA.com
2. Goals
Learn how SQL Server creates, stores and maintains
statistics
Understand how the optimizer consumes statistics
to arrive at an execution plan
Learn various methods for controlling statistics to
take more direct control of your queries
Grant Fritchey | www.ScaryDBA.com
5. CREATE PROC dbo.spAddressByCity @City NVARCHAR(30)
AS
SELECT a.AddressID,
a.AddressLine1,
a.AddressLine2,
a.City,
sp.[Name] AS StateProvinceName,
a.PostalCode
FROM
Person.Address AS a
JOIN Person.StateProvince AS sp
ON a.StateProvinceID = sp.StateProvinceID
WHERE
a.City = @City;
Grant Fritchey | www.ScaryDBA.com
12. Cardinality
Selectivity
» Returned values / Total Values
Histogram
Density
» 1/distinct values
Compound Columns
» Selectivity * Selectivity
» Density * Density
+ Trace Flags which we’ll talk about
Grant Fritchey | www.ScaryDBA.com
12
13. Estimates vs. Cardinality
Comparing variables
Using != and NOT
Predicates comparing columns within a table
Functions w/o constant value
Joins using arithmetic or string operations
No statistics!
Skewed distribution (eh)
Grant Fritchey | www.ScaryDBA.com
13
14. Statistics are used to…
Determine cardinality which is used to…
» Determine number of rows processed which is
used to…
— Determine cost of the operation in the plan which is
used to…
– Pick the operations used in the plan which is used to
» Return your data in an efficient manner which is used for
whatever the heck the business wants
Grant Fritchey | www.ScaryDBA.com
14
29. What?
The US State of Washington… yes I’m serious
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID
Table Column Number
System
Grant Fritchey | www.ScaryDBA.com
29
30. YOU CAN CREATE STATS
Grant Fritchey | www.ScaryDBA.com
30
32. CREATE STATISTICS MyJrStats
ON Person.Person (Suffix)
WHERE Suffix = 'Jr.'
WITH FULLSCAN;
Grant Fritchey | www.ScaryDBA.com
32
33. CREATE STATISTICS MyComboStats
ON Person.Person (Title,Suffix)
WHERE Suffix = 'PhD'
WITH FULLSCAN;
Grant Fritchey | www.ScaryDBA.com
33
34. …Or Drop Them
DROP STATISTICS Person.Person.MyStats;
DROP STATISTICS Person.Person.MyJrStats;
DROP STATISTICS Person.Person.MyComboStats;
Grant Fritchey | www.ScaryDBA.com
34
40. Advanced D&D
Trace Flag 2371
» SQL Sever 2008 R2 Sp1 and above
» After 25,000 rows
— Percentage changed based on number of rows
— Reducing as the number grows
Trace Flag 4137
» Minimum selectivity instead of multiplication on
AND predicates
Trace Flag 9471 (SQL Server 2014)
» Minimum selectivity on AND and OR predicates
Trace Flag 9472 (SQL Server 2014)
» Independence (pre-2014 behavior)
Grant Fritchey | www.ScaryDBA.com
40
53. Recommendations
Compatibility setting
Automatic creation
Automatic update
Update asynchronous where necessary
Use appropriate sample rate
Grant Fritchey | www.ScaryDBA.com
53
54. Goals
Learn how SQL Server creates, stores and maintains
statistics
Understand how the optimizer consumes statistics
to arrive at an execution plan
Learn various methods for controlling statistics to
take more direct control of your queries
Grant Fritchey | www.ScaryDBA.com
55. Resources
Understanding SQL Server Cardinality Estimations
Fixing Cardinality Estimation Errors
Statistics Used by the Optimizer
First look at the
query_optimizer_estimate_cardinality XE Event
Changes to automatic update statistics inSQL
Server – traceflag 2371
Cardinality Estimation for Multiple Predicates
Grant Fritchey | www.ScaryDBA.com
55