Oracle optimizer bootcamp

Oracle Optimizer Bootcamp
10 Optimizer tips you can't
do without
Maria Colgan & Jonathan Lewis
@SQLMaria @JLOracle

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
Who are we ?
30+ years in IT
25+ using Oracle
Strategy, Design, Review, Briefings,
Educational, Trouble-shooting
Jonathan Lewis Independent Consultant
Member of the Oak Table Network
Oracle ACE Director
Oracle author of the year 2006
Select Editor’s choice 2007
UKOUG Inspiring Presenter 2011
UKOUG Council member 2012
ODTUG 2012 Best Presenter (d/b)
O1 visa for USA

Program Agenda
 Using the right tools
 Finding the right sample size
 Tell the optimizer everything
 Functions friends or foe
 Most useful hints to know

Using the right tools
 Query – How many customers do we have in a specific zipcode
 Customers2 table has b-tree index on zipcode
 There is a histogram on the zipcode column due to data skew
Expected index range scan but got full table scan?
SELECT count(cust_first_name)
FROM customers2
WHERE zipcode = :n;

Explain plan command
shows a full table scan
plan, why?
Even though the bind
variable is defined
Explain plan command is
not bind aware

Where does the
cardinality estimate of
10065 come from?
No bind peeking occurs
so histogram can’t be
used
Cardinality estimates is
NUM_ROWS 20129
NDV 2
10064.5
Rounded up to 10065

When query is executed
we get the right plan
Bind variable value is
peeked and histogram is
used to determine the
correct cardinality estimate
Use DISPLAY_CURSOR
to display bind values by
add additional format
parameters +peeked_binds

Take a look at V$SQL after we execute each approach 3 times
The cursors for an
explain plan command
are not shared

Autotrace is a popular tool
as it allows you to see
performance statistics for
SQL stmts
Autotrace traceonly
explain uses the explain
plan infrastructure
Not good for statements
with binds as no peeking
occurs

Use the right tools
Take a look at V$SQL after we execute each approach 3 times
Only one cursor now BUT plan hash value is the full table scan and not index
access
Cursor generated by autotrace explain will be shared by other and you
can’t tell that the bad plan comes from autotrace explain!

Another Example
select gp.small_vc_gp, p.small_vc_p, c.small_vc_c
from grandparentgp,
parent p,
child c
where gp.small_num_gp between 100 and 125
and p.id_gp = gp.id
and p.small_num_p between 110 and 135
and c.id_p = p.id
and c.id_gp = p.id_gp
and c.small_num_c between 120 and 145;

Serial plan
| Id | Operation | Name | Rows |
| 0 | SELECT STATEMENT | | |
|* 1 | HASH JOIN | | 134 |
|* 2 | TABLE ACCESS FULL | GRANDPARENT | 136 |
|* 3 | HASH JOIN | | 134 |
|* 4 | TABLE ACCESS FULL| PARENT | 135 |
|* 5 | TABLE ACCESS FULL| CHILD | 135 |

Parallel Plan
| Id | Operation | Name | Rows | TQ |IN-OUT| PQ Distrib |
| 0 | SELECT STATEMENT | | | | | |
| 1 | PX COORDINATOR | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10004 | 134 | Q1,04 | P->S | QC (RAND) |
|* 3 | HASH JOIN BUFFERED | | 134 | Q1,04 | PCWP | |
| 4 | PX RECEIVE | | 136 | Q1,04 | PCWP | |
| 5 | PX SEND HASH | :TQ10002 | 136 | Q1,02 | P->P | HASH |
| 6 | PX BLOCK ITERATOR | | 136 | Q1,02 | PCWC | |
|* 7 | TABLE ACCESS FULL | GRANDPARENT | 136 | Q1,02 | PCWP | |
| 8 | PX RECEIVE | | 134 | Q1,04 | PCWP | |
| 11 | PX RECEIVE | | 135 | Q1,03 | PCWP | |
|* 14 | TABLE ACCESS FULL| PARENT | 135 | Q1,00 | PCWP | |
| 15 | PX RECEIVE | | 135 | Q1,03 | PCWP | |
|* 18 | TABLE ACCESS FULL| CHILD | 135 | Q1,01 | PCWP | |

Parallel Plan
| 4 | PX RECEIVE | | 136 | Q1,04 | PCWP | |
| 8 | PX RECEIVE | | 134 | Q1,04 | PCWP | |
| 11 | PX RECEIVE | | 135 | Q1,03 | PCWP | |
| 15 | PX RECEIVE | | 135 | Q1,03 | PCWP | |

V$pq_tqstat
TQ_ID SERVER_TYPE PROCESS NUM_ROWS BYTES
0 Producer P000 40 928 Parent
P001 90 2026
Consumer P002 70 1587
P003 60 1367
1 Producer P000 0 48 Child
P001 130 2906
P003 67 1521
2 Producer P002 80 1407 Grandparent
P003 50 898
P001 62 1101
…….

V$pq_tqstat
TQ_ID SERVER_TYPE PROCESS NUM_ROWS BYTES
3 Producer P002 41 1237 Join P & C
P003 39 1179
P001 42 1266
4 Producer P000 16 600 Join G &(P/C)
P001 13 492
Consumer QC 29 1092
…….

Program Agenda

Finding the right sample size
 # 1 most commonly asked question
– “What sample size should I use?”
 Controlled by ESTIMATE_PRECENT parameter
 From 11g onwards use default value AUTO_SAMPLE_SIZE
– New hash based algorithm
– Speed of a 10% sample
– Accuracy of 100% sample
AUTO_SAMPLE_SIZE
More info in the following paper http://dl.acm.org/citation.cfm?id=1376721

Results of statistics gathering experiment
AUTO_SAMPLE_SIZE
Lineitem table from TPC-H benchmark scale factor 300GB

Why AUTO_SAMPLE_SIZE is the right sample
size
 The number of distinct values in a column is the most critical statistics
 Optimizer uses this statistics in all cardinality calculations
 Providing an accurate NDV is critical

Why AUTO_SAMPLE_SIZE is the right sample
size

Client Data (modelled)
select
status, count(*)
from
orders
group by
status
order by
status;
STATUS COUNT(*)
C 529,100
P 300
R 300
S 300
X 500,000

Important Client Queries
select * from orders where status = 'P
select * from orders where status = 'R'
select * from orders where status = 'S'
So we gather a histogram - four times !

Bad Luck
ENDPOINT_NUMBER FREQUENCY STATUS
2848 2848 C
2849 1 P
5629 2780 X
2741 2741 C
2742 1 P
2743 1 R
5331 2588 X
2706 2706 C
2708 2 P
5355 2647 X
2852 2852 C
2854 2 P
2856 2 R
2859 3 S
5472 2613 X
Results 11.2.0.3 - four attempts (auto-sample)
We may miss ALL the rare values
And maybe the rare values are only there during the day anyway

Solution (part 2)
c_array := dbms_stats.chararray('C', 'P', 'R', 'S', 'X' );
srec.bkvals := dbms_stats.numarray ( 5000, 3, 3, 3, 5000);
srec.epc := 5;
dbms_stats.prepare_column_values(srec, c_array);
dbms_stats.set_column_stats(
ownname => user,
tabname => 'ORDERS',
colname => 'STATUS',
distcnt => m_distcnt,
density => m_density,
nullcnt => m_nullcnt,
srec => srec,
avgclen => m_avgclen
);
end;

Solution (part 1)
declare
srec dbms_stats.statrec;
c_array dbms_stats.chararray;
http://jonathanlewis.wordpress.com/2009/05/28/frequency-histograms/
m_distcnt number;
m_density number;
m_nullcnt number;
m_avgclen number;
begin
m_distcnt := 5;
m_density := 0.00001;
m_nullcnt := 0;
m_avgclen := 1;

Precision (12c)
2741 2741 C
2742 1 P
2743 1 R
5331 2588 X
2706 2706 C
2708 2 P
5355 2647 X
2852 2852 C
2854 2 P
2856 2 R
2859 3 S
5472 2613 X
529100 529100 C
529400 300 P
529700 300 R
530000 300 S
1030000 500000 X
11.2.0.3 12.1.0.0
2848 2848 C
2849 1 P
5629 2780 X
12c has enhanced the code for the calculation
of "approximate NDV" so for a small number of
distinct values it can produce an accurate
frequency histogram at virtually no extra cost

Program Agenda

Tell the Optimizer everything
 The Optimizer makes assumptions in certain case
– Each additional where clause predicates reduces the number of rows
– Each additional group by column will increase number of rows returned
– Applying a function to column will alter the NDV of the column
 Sometimes you know more about your data than basic statistics show
– Best way to get a good plan is to tell the Optimizer everything
– Otherwise it assumes making an ass out of u and me

Correlated column example
SELECT * FROM vehicles WHERE model = ‘530xi’
AND color = 'RED’;
MAKE MODEL COLOR Year
BMW 530xi RED 2013
Cardinality #ROWS * 1 * 1 12 * 1 * 1 1
NDV c1 NDV c2 4 3
= => =

Correlated column example
AND make = ‘BMW’;
BMW 530xi RED 2013
BMW 530xi BLACK 2011
BMW 530xi BLUE 2012
Cardinality #ROWS * 1 * 1 12 * 1 * 1 1
NDV c1 NDV c2 4 3
=> =

Create extended statistics on column group
DBMS_STATS.CREATE_EXTENDED_STATS
New Column with
system generated
name

Correlated column example with extended statistics
AND make = ‘BMW’;
BMW 530xi RED 2013
BMW 530xi BLACK 2011
BMW 530xi BLUE 2012
Cardinality calculated using column group statistics

Tell the optimizer everything
 Use SEED_COL_USAGE to have Oracle monitor workload
 Can monitor STS or live system
 Recommends what column groups are needed based on workload
Automatic column group detection

Bad Defaults (a)
create table t1 as
select
decode(
mod(rownum - 1,1000),
0,to_date('31-Dec-4000'),
to_date('01-Jan-2010') +
trunc((rownum-1)/100)
) date_closed
from
large_data_source
where
rownum <= 1827 * 100
;
-- 5 years in days * 100
-- every 1000th row
-- is a 'null' date

Bad Defaults (b)
select * from t1 where date_closed
between to_date('01-Jan-2012','dd-mon-yyyy')
and to_date('31-Dec-2012','dd-mon-yyyy');
-- 36,500 rows
With simple statistics.
TABLE ACCESS (FULL) OF 'T1' (Cost=46 Card=291 Bytes=2328)
With 11 column histogram (two per year and one for the garbage)
With nulls and simple statistics.

Graphical Interpretation
The real data
Oracle's first impression
Impression with histogram
Data for 2012

Bad Data Types (a)
create table t1 as
select
d1,
to_number(to_char(d1,'yyyymmdd')) n1,
to_char(d1,'yyyymmdd') v1
from (
select
to_date('31-Dec-2009') + rownum d1
from all_objects
where
rownum <= 1827 -- 5 years, daily
);
-- date
-- numeric
-- char

Bad Data Types (b)
select * from t1 where d1 -- 7 rows
between to_date('30-Dec-2012','dd-mon-yyyy')
and to_date('05-Jan-2013','dd-mon-yyyy');
On the date type.
N1 between 20121230 and 20030105
V1 between ' 20121230' and '20030105'
10g+ gets Card=397 - everything looks like numbers)

Program Agenda

Functions friends or foe?
 Query – How many packages of bounce did we sell?
 Sales 2 has a b-tree index on the prod_id column
Expected index range scan but got fast full index scan
SELECT count(*)
FROM sales2
WHERE to_char(prod_id)=‘139’;

Cardinality estimate is in the
right ballpark so not a
problem with statistics
Why is an equality predicate
being evaluated as a filter
and not an access
predicate?
Could it have something to do
with the TO_CHAR function

 INTERNAL_FUNCTION indicates Oracle needs to modify predicate
– In this case a data type conversion occurred
 Predicate is TO_CHAR(prod_id)
 Optimizer has no idea how function effects values in prod_id column
 Optimizer can’t determine which rows will be accessed now

 What data type is the prod_id column ?
 But literal value is a character string ‘139’
Better to apply inverse function on other side of predicate

Query rewrite SELECT count(*)
FROM sales2
WHERE prod_id=to_number(‘139’);
Solution - Use inverse function on other side of predicate

 Keep the following in mind when deciding where to place the function
– Try to place functions on top of constants (literals, binds) rather than on columns
– Avoid functions on index columns or partition keys as it prevents index use or
partition pruning
– For function-based index to be considered, use that exact function as specified in
index
– If multiple predicates involve the same columns, write predicates such that they
share common expressions For example,
WHERE f(a) = b WHERE a = inv_f(b)
AND a = c AND a = c
Using inverse function on other side of predicate
Should be
rewritten as
This will allow transitive
predicate c=inv_f(b) to be
added by the optimizer

Virtual Columns (a)
create table t1 as
Select
cast(rownum as number(8,0)) id1,
cast(rownum as number(8,0)) id2
from all_objects
where rownum <= 10000
;
execute dbms_stats.gather_table_stats(user,'t1');
alter table t1 add
m2 generated always as ( mod(id2,3)) virtual
;

Virtual Columns (b)
set autotrace traceonly explain
select * from t1 where mod(id1,2) = 0;
.
|* 1 | TABLE ACCESS FULL| T1 | 100 | 1100 | 4 |
Predicate Information (identified by operation id):
1 - filter(MOD("ID1",3)=0)
select * from t1 where mod(id1,2) != 0;
.
1 - filter(MOD("ID1",3)<>0)

Virtual Columns (c)
set autotrace traceonly explain
select * from t1 where mod(id2,3) = 0;
.
1 - filter("T1"."M2"=0)
select * from t1 where mod(id2,3) != 0;
.
1 - filter("T1"."M2"<>0)

Program Agenda

Most useful hints to know
Understanding Optimizer hints
 Hints allow you to influence the Optimizer when it has to choose
between several possibilities
 A hint is a directive that will be followed when applicable
 Can influence everything from the Optimizer mode used to each
operation in the execution
 The hint mechanism is not exclusively used by the Optimizer
 Some of the most useful hint influence SQL execution

 GATHER_PLAN_STATISTICS triggers execution time statistics to be
preserved
 Records actual number of rows seen for each operation (step) in plan
 Execution statistis displayed in plan when format parameter of
DMBS_XPLAN set to ALLSTATS LAST
– Means display all execution statistics for last execution of this cursor
GATHER_PLAN_STATISTICS hint

GATHER_PLAN_STATISTICS hint
 SELECT * FROM table (
 DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>'ALLSTATS
LAST'));
 Compare estimated rows returned for each operation to actual rows

GATHER_PLAN_STATISTICS Hint
SELECT * FROM table (
DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>'ALLSTATS LAST'));
Note: a lot of the data is zero in
the A-rows column because we
only show last executed cursor
which is the QC. Need to use
ALLSTATS ALL to see info on all
parallel server cursors

GATHER_PLAN_STATISTICS Hint
SELECT * FROM table (
DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>'ALLSTATS ALL'));

Alternatively using MONITOR hint

Most helpful hints to know
 Allows value for init.ora Optimizer parameters to be changed for a specific
query
 Useful way to prevent setting non-default parameter value system-wide
 Only the following Optimizer influencing init.ora parameters can be set:
OPT_PARAM hint
 OPTIMIZER_DYNAMIC_SAMPLING
 OPTIMIZER_INDEX_CACHING
 OPTIMIZER_INDEX_COST_ADJ
 OPTIMIZER_USE_PENDING_STATISTICS
 Optimizer related underscore parameters
 STAR_TRANSFORMATION_ENABLED
 PARALLEL_DEGREE_POLICY
 PARALLEL_DEGREE_LIMIT

OPT_PARAM hint
Example

 OPTIMIZER_FEATURES_ENABLE parameter allows you to switch
between optimizer versions
 Setting it to previous database version reverts the Optimizer to that
version
– Disables any functionality that was not present in that version
 Easy way to work around unexpected behavior in a new release
 Hint allows you to revert the Optimizer for just a single statement
OPTIMIZER_FEATURES_ENABLE hint

Precision matters (a)
select /*+ parallel(pt1,2) */
{list of columns}
from pt1
wherept1.n1 = 5
and pt1.n2 = 10
and pt1.pt_group in (0,10)
;
The optimizer DID obey the hint
.
|Id|Operation |Name |Cost |Pstart| Pstop|
| 0|SELECT STATEMENT | | 48| | |
| 1| PARTITION LIST INLIST | | 48|KEY(I)|KEY(I)|
|*2| TABLE ACCESS BY LOCAL INDEX ROWID|PT1 | 48|KEY(I)|KEY(I)|
|*3| INDEX RANGE SCAN |PT1_I1| 3|KEY(I)|KEY(I)|

Precision matters (b)
...
NESTED LOOP
TABLE ACCESS (FULL) OF T7
TABLE ACCESS BY INDEX ROWID T8
INDEX RANGE SCAN T8_IND_SECOND -- “wrong” index
...
select /*+ index(t8 t8_ind_first) */ -- hint the “right” one
...
HASH JOIN -- wrong method
TABLE ACCESS (FULL) OF T7
TABLE ACCESS BY INDEX ROWID T8
INDEX FULL SCAN T8_IND_FIRST -- “right” index
...

How much don't we know ?
select
/*+
leading(t4 t1 t2 t3)
full(t4)
use_hash(t1) full(t1)
*/
...
From t4, t1, t2, t3
Where t1.id = t4.id1
and t2.id = t4.id2
and t3.id = t4.id3;

How much don't we know ?
N – 1 simultaneous build (hash) tables
Maximum two simultaneous build (hash) tables
HASH JOIN
TABLE ACCESS (FULL) OF TABLE_3
HASH JOIN
HASH JOIN
HASH JOIN
HASH JOIN
HASH JOIN

Summary
 The Optimizer is not out to get you
– But its knowledge is limited to the information you provide it
 A complete representative set of statistics gets the best plans
 Be mindful of when you capture a representative set of statistics

More Information
 White papers
– Optimizer with Oracle Database 12c
– SQL Plan Management with Oracle Database 12c
– Understanding Optimizer Statistics with Oracle Database 12c
 Blogs
– http://jonathanlewis.wordpress.com/all-postings/
– http://blogs.oracle.com/optimizer
– https://SQLMaria.com/
 Oracle.com
– http://www.oracle.com/technetwork/database/focus-areas/bi-
datawarehousing/dbbi-tech-info-optmztn-092214.html

Graphic Section Divider

Oracle optimizer bootcamp

Recommended

Recommended

More Related Content

More from Maria Colgan

More from Maria Colgan (11)

Recently uploaded

Recently uploaded (20)

Oracle optimizer bootcamp