Understanding Optimizer-Statistics-for-Developers

Understanding
Op.mizer

Sta.s.cs
for
Developers

May
the
Odds
Be
Ever
in
Your
Favor

www.enkitec.com

www.facebook.com/enkitec

@enkitec

©2013 Enkitec 3

karen.morton@enkitec.com

karenmorton.blogspot.com

@karen_morton

©2013 Enkitec 4

Statistics?

Odds?

It's just math.

SQL>desc tributes
Name Type
----------- -------------
PANEM_ID NUMBER
NAME VARCHAR2(20)
DISTRICT NUMBER
GENDER VARCHAR2(1)
AGE NUMBER
QUAL_SCORE NUMBER
WEAPON VARCHAR2(60)
©2013 Enkitec 6

Cardinality

The
es.mated
number
of
rows

a
query
is
expected
to
return.

number of rows in table
x
predicate selectivity
©2013 Enkitec 8

select *
from tributes
order by district, age;

Cardinality
24 x 1 = 24

©2013 Enkitec 9

select *
from tributes
where gender = 'F' ;

Cardinality
24 x 1/2 = 12

©2013 Enkitec 11

select *
from tributes
where district = 12
and gender = 'F' ;

Cardinality
24 x 1/12 x 1/2 = 1

©2013 Enkitec 13

select *
from tributes
where age > 15 ;

Cardinality
(17 – 15) High
Value
-‐
Predicate
Value

24 x
(17 – 12) High
Value
-‐
Low
Value

©2013 Enkitec = 9.6 15

select *
from tributes
where weapon = :b1 ;

Cardinality
24 x 1/11 = 2.18

©2013 Enkitec 17

:b1 = 'Bow and Arrow'

©2013 Enkitec 18

No tricks

just

math

Sta.s.cs
that
don't
reasonably

describe
your
data

…lead
to
poor

cardinality
es.mates

…which
leads
to
poor

access
path
selec.on

…which
leads
to
poor

join
method
selec.on

…which
leads
to
poor

join
order
selec.on

…which
leads
to
poor

SQL
execu.on
.mes.

Goal

Collect
sta.s.cs
that
are

"good
enough"
to
meet

most
needs
most
of
the
.me.

Say
you
were
standing
with
one

foot
in
the
oven
and
one
foot
in

an
ice
bucket.
According
to
the

percentage
people,
you
would

be
perfectly
comfortable.

–
Bobby
Bragan

The
op.mizer
assumes

uniform
distribu.on

of
column
values.

Color
column
-‐
uniform
distribu.on

Color
column
–
skewed
distribu.on

Data
skew
must
be

iden.ﬁed
with

a
histogram.

Table: obj_tab
100%
Sta.s.cs

Statistic Current value FOR
ALL
COLUMNS
SIZE
1

--------------- --------------
# rows 1601874
Blocks 22321
Avg Row Len 94
Sample Size 1601874
Monitoring YES

Column: object_type

OBJECT_TYPE PCT_TOTAL
------------------------------- ---------
WINDOW GROUP - PROGRAM .00-.02
EVALUATION CONTEXT - XML SCHEMA .03-.05
OPERATOR - PROCEDURE .11-.17
LIBRARY - TYPE BODY .30-.35
FUNCTION - INDEX PARTITION .54-.64
JAVA RESOURCE - PACKAGE 1.54-1.69
TABLE - VIEW 3.44-7.35
JAVA CLASS 32.80
SYNONYM 40.01

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
SQL_ID 16yy3p8sstr28, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ owner, object_name, object_type, object_id,
status from obj_tab where object_type = 'PROCEDURE'

Plan hash value: 2862749165
--------------------------------------------------------------------------------
| Id | Operation | Name | E-Rows | A-Rows | Buffers |
--------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| OBJ_TAB | 44497 | 2720 | 1237 |
|* 2 | INDEX RANGE SCAN | OBJ_TYPE_IDX | 44497 | 2720 | 193 |
--------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("OBJECT_TYPE"='PROCEDURE')

R
=
.06
seconds

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
SQL_ID 9u6ppkh5mhr8v, child number 0
-------------------------------------
status from obj_tab where object_type = 'SYNONYM'


--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| OBJ_TAB | 44497 | 640K| 104K|
|* 2 | INDEX RANGE SCAN | OBJ_TYPE_IDX | 44497 | 640K| 44082 |
--------------------------------------------------------------------------------

---------------------------------------------------

2 - access("OBJECT_TYPE"='SYNONYM')

R
=
14.25
seconds

100%
Sta.s.cs

FOR
ALL
COLUMNS
SIZE
AUTO

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
SQL_ID 16yy3p8sstr28, child number 0
-------------------------------------
status from obj_tab where object_type = 'PROCEDURE'

--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| OBJ_TAB | 2720 | 2720 | 1237 |
|* 2 | INDEX RANGE SCAN | OBJ_TYPE_IDX | 2720 | 2720 | 193 |
--------------------------------------------------------------------------------

---------------------------------------------------
2 - access("OBJECT_TYPE"='PROCEDURE')

R
=
.07
seconds

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
SQL_ID 9u6ppkh5mhr8v, child number 0
-------------------------------------
status from obj_tab where object_type = 'SYNONYM'


-----------------------------------------------------------------
-----------------------------------------------------------------
|* 1 | TABLE ACCESS FULL| OBJ_TAB | 640K| 640K| 64263 |
-----------------------------------------------------------------

---------------------------------------------------
1 - filter("OBJECT_TYPE"='SYNONYM')

R
=
3.36
seconds

op.mizer_dynamic_sampling

parameter

dynamic_sampling

hint

SQL>create table ds_test as
2 select mod(num, 100) c1,
3 mod(num, 100) c2,
4 mod(num, 75) c3,
5 mod(num, 30) c4
6 from (select level num from dual
7 connect by level <= 10001);

Table created.

SQL>exec dbms_stats.gather_table_stats( user, 'ds_test',
estimate_percent => null, method_opt => 'for all columns size
1');

PL/SQL procedure successfully completed.

Statistic Current value
--------------- --------------
# rows 10001
Blocks 28
Avg Row Len 11
Sample Size 10001
Monitoring YES

Column NDV Density AvgLen Histogram LowVal HighVal
------- --- ------- ------ --------- ------ -------
C1 100 .010000 3 NONE (1) 0 99
C2 100 .010000 3 NONE (1) 0 99
C3 75 .013333 3 NONE (1) 0 74
C4 30 .033333 3 NONE (1) 0 29

SQL>set autotrace traceonly explain
SQL>select count(*) from ds_test where c1 = 10;

Execution Plan
----------------------------------------------------------
----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 3 | 8 (0)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 3 | | |
|* 2 | TABLE ACCESS FULL| DS_TEST | 100 | 300 | 8 (0)| 00:00:01 |
----------------------------------------------------------------------------------

---------------------------------------------------
2 - filter("C1"=10)

SQL>select count(*) from ds_test where c1 = 10 and c2 = 10;

Execution Plan
----------------------------------------------------------

----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 6 | | |
----------------------------------------------------------------------------------

---------------------------------------------------

2 - filter("C1"=10 AND "C2"=10)

SQL>select /*+ dynamic_sampling (4) */ count(*)
2 from ds_test where c1 = 10 and c2 = 10;

Execution Plan
----------------------------------------------------------

----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 6 | | |
----------------------------------------------------------------------------------

---------------------------------------------------

2 - filter("C1"=10 AND "C2"=10)

Note
-----
- dynamic sampling used for this statement

SQL> select dbms_stats.create_extended_stats(ownname=>user,
2 tabname => 'DS_TEST',
3 extension => '(c1, c2)' ) AS c1_c2_correlation
4 from dual ;

C1_C2_CORRELATION
-------------------------------------------------------------
SYS_STUF3GLKIOP5F4B0BTTCFTMX0W

SQL> exec dbms_stats.gather_table_stats( user, 'ds_test');

PL/SQL procedure successfully completed.

SQL> set autotrace traceonly explain
SQL> select count(*) from ds_test where c1 = 10 and c2 = 10;

Execution Plan
----------------------------------------------------------
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 6 | | |
----------------------------------------------------------------------------------

---------------------------------------------------
2 - filter("C1"=10 AND "C2"=10)

Understanding
how
the

op.mizer
uses
sta.s.cs

helps
you
know
when

"special"
sta.s.cs

can
help.

©2013 Enkitec 50

Query
Transforma.on

©2013 Enkitec 51

SQL
and
the
Op.mizer

Parsed
SQL

iterative

Query transformations
Logical

op.miza.ons

Access & Join methods
Join order
Physical

stats Dic.onary

op.miza.ons

Cost
Es.mator

Execution Plan

©2013 Enkitec 52

Query
Transforma.ons

•  Goal
is
to
enhance
query
performance
by

making
more
choices
available
to
the
op6mizer

•  Creates
a
seman.cally
equivalent
statement

that
will
produce
the
same
results

•  Remove
extraneous
condi.ons

•  Add
inferred
condi.ons

Think
algebra…

©2013 Enkitec 53

Types
of
Transforma.ons

•  Automa.c

–  Always
produce
a
faster
plan

•  Heuris.c-‐based
(pre-‐10gR1)

–  Should
produce
a
faster
plan
most
of
the
.me

•  Cost-‐based

–  Does
not
always
produce
a
faster
plan

©2013 Enkitec 54

So,
if
the
op.mizer
will

transform
my
SQL
for
me,

why
should
I
write
it

any
diﬀerently?

Why
You
Should
Refactor

•  You
know
your
stuff
best
(or
you
should)

•  Always
filter
early

•  Defines
your
expecta.ons

•  K.I.S.S.

•  The
op.mizer
might
not
be
able
to

©2013 Enkitec 56

Common
Transforma.ons

•  FPD
–
ﬁlter
push-‐down

•  PM
–
predicate
move-‐around

•  SU
–
subquery
unnes.ng

•  CVM
–
complex
view
merging

•  SPJ
–
select-‐project-‐join
(simple
view
merging)

•  JF
–
join
factoriza.on

©2013 Enkitec Numerous additional transformations exist – see 10053 trace 57

SPJ
–
Simple
View
Merging

is transformed into

Merged automatically as it is deemed “always better” for the optimizer to work with direct joins. 58

Complex
View
Merging

is transformed into

“Complex” due to GROUP BY. CVM can also be done when using DISTINCT or outer join. 59

Filter
Push-‐Down

is transformed into

Purpose: To push outer query predicates into view to perform earlier filtering. 60

Predicate
Move-‐Around

is transformed into

Purpose: To move inexpensive predicates into view query blocks to perform earlier filtering. 61
Can generate filter predicates based on transitivity or functional dependencies.

Join
Factoriza.on

is transformed into

Combines branches of UNION / UNION ALL that join a common table in order to reduce # of accesses to that table.
62

Understanding
how
the

op.mizer
transforms
queries

helps
you
write
beTer
SQL
and

understand
execu.on
plans.

©2013 Enkitec 63

What
is
Cardinality
Feedback?

•  Automa.cally
improve
plans
for
repeatedly

executed
queries
where
the
op.mizer
does

not
es.mate
cardinali.es
properly

•  Mises.mates
may
be
due
to

–  Missing
or
inaccurate
sta.s.cs

–  Complex
predicates

…and
more

•  Es.mates
and
actuals
are
compared

•  New
plan
generated
using
adjustment
factors

•  Es.mates
lost
if
plan
ages
out
of
cache

©2013 Enkitec 65

Es.mates
Diﬀerences

Plan
Opera.on
Object
Rows
es.mate

Original
Plan
23835

TABLE
ACCESS
FULL
VS_PRODUCT
67

INDEX
FAST
FULL
SCAN
IDX_VS_ORDER_027
514K

INDEX
FAST
FULL
SCAN
IDX_VS_ORDER_TRACK_017
2665K

INDEX
FAST
FULL
SCAN
PK_VS_ORDER
8673K

New
Plan
8371
✔
INDEX
FULL
SCAN
IDX_VS_PRODUCT_002
84

INDEX
SKIP
SCAN
IDX_VS_ORDER_027
941K

INDEX
SKIP
SCAN
IDX_VS_ORDER_TRACK_017
2899K

INDEX
FAST
FULL
SCAN
PK_VS_ORDER
10M

©2013 Enkitec 69

Cardinality
feedback
is
an
automated

way
the
op.mizer
tries
to

"self-‐correct"
and
produce
plans

that
perform
op.mally.

©2013 Enkitec 70

Dynamic
Sta.s.cs

•  Replaces
dynamic
sampling

•  Compensates
for
missing
or
incomplete
stats

•  Time
to
do
collec.ons
is
limited
by
op.mizer

based
on
es.mated
query
run
.me

•  When
collected,
stats
will
be
stored
in
cache

and
therefore
can
be
shared
among
queries

•  Diﬀerent
queries
using
same
predicates
can

used
dynamic
stats
stored
in
cache

©2013 Enkitec 72

Adap.ve
Sta.s.cs

•  Was
cardinality
feedback
in
11gR2

•  Es.mates
compared
to
actuals

•  Signiﬁcant
varia.ons
cause
new
plan
choice

•  New
plan
uses
stats
from
previous
execu.ons

•  Previous
stats
retained
for
later
use
as
SQL

Plan
Direc.ves

•  Statements
can
be
re-‐op.mized
over
and
over

–  New
column
V$SQL.IS_REOPTIMIZABLE
©2013 Enkitec 73

Adap.ve
Execu.on
Plans

•  More
than
1
plan
pre-‐computed
and
stored

•  Sta.s.cs
collectors
used
to
capture
execu.on

info

•  If
NL
chosen
and
exceeds
certain
threshold,

will
switch
to
HJ

•  DBMS_XPLAN.DISPLAY_CURSOR
has
new

format
parameter
to
show
ac.ons

format=>'+all_dyn_plan +adaptive'

©2013 Enkitec 74

Sta.s.cs
must
reasonably

represent
your
actual
data.

76

Understanding
basic

op.mizer
sta.s.cs

computa.ons
is
key.

77

Write
SQL
to

u.lize
your
knowledge

of
how
sta.s.cs
are

used
by
the
op.mizer.

78

The
more
you
know,

the
more
likely
your

sta.s.cs
strategy

and
the
way
you
write
SQL

will
support
op.mally

performing
execu.on
plans.

79

Q

&

Q U E S T I O N S
A N S W E R S

Understanding Optimizer-Statistics-for-Developers

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (8)

Ähnlich wie Understanding Optimizer-Statistics-for-Developers

Ähnlich wie Understanding Optimizer-Statistics-for-Developers (20)

Mehr von Enkitec

Mehr von Enkitec (20)

Understanding Optimizer-Statistics-for-Developers