B25 Reducing redo by Julian Dyke

1
Reducing
Redo
Julian Dyke
Independent Consultant
DB Tech Showcase - Osaka
May 2013
juliandyke.com© 2013 Julian Dyke

juliandyke.com© 2013 Julian Dyke2
Agenda
 Introduction
 Tests
 Indexes
 Number of columns processed
 SELECT FOR UPDATE
 Number of rows processed
 COMMIT
 Batch size
 Global temporary tables
 External tables
 Conclusion

Redo Records
Redo Block
Header
16 bytes
Redo Block
512 or
1024 bytes
Redo Block
Body
496 bytes
Redo Record 1
Redo Record 2
Wastage
Redo Record 3
Wastage
Header
Body
Body
Header
Spare
Header
Body
Spare
STOP

Change Vectors
Redo
Record
Header
Body
Header
Body
Header
Body
Header
Body
Change
Vectors
Change
Vector 3
Change
Vector 1
Change
Vector 2
STOP

Header
Change Vector
Header
Body
STOP

Example
 Examples in this presentation taken from Formula 1 database
 Contains full details of all races from 1961 to 2012
 Updated annually in November (end of season)
 Currently
 22 cars per race
 19 races per season
 Approximately 420 new rows per season

Schema
CLASSIFICATION
SEASON
GRANDPRIX
RACE TEAMDRIVER
COUNTRYCIRCUIT
ENGINE
CAR

Cars
 Each season has up to 20 races (19 in 2005)
 Each race has up to 39 entrants (13 races in 1989)
 Each car has
 driver, team and engine
 laps completed (may be zero)
 optional notes
 Results are classified as follows
C Classified
DNF Did not finish
DNS Did not start
DNQ Did not qualify
DIS Disqualified

Points
 Points awarded to driver and team as follows
1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th
Pre 1991 9 6 4 3 2 1
1991-2002 10 6 4 3 2 1
2003-2009 10 8 6 5 4 3 2 1
2010 onwards 25 18 15 12 10 8 6 4 2 1
 But . . . not always straightforward
 Half points awarded for incomplete races
 Split races (two half point races aggregated)
 Driver and / or team disqualifications e.g. Tyrrell in 1984
 Up to 1980 only best scores counted for each half of season e.g.
 Best 5 results from first 7 races and best 5 results from last 7 races
 1982-1990 only best 11 results counted for drivers
 1961-1978 only first car to finish counted for each team

Input file - car.csv
 Comma separated file
 16181 rows
 Fields are:
 season_key
 race_key
 position
 driver_key
 team_key
 engine_key
 laps_completed
 classification_key
 notes (optional)
2004 17 17 ZBAU MIN FOR 41 DNF Spin
2004 17 18 DCOU MCL MER 38 DNF Accident
2004 17 19 RBAR FER FER 38 DNF Accident
2004 17 20 MWEB JAG FOR 20 DNF Overheated
2004 18 1 JMON WIL BMW 71 C
2004 18 2 KRAI MCL MER 71 C
2004 18 3 RBAR FER FER 71 C
2004 18 4 FALO REN REN 71 C
2004 18 5 RSCH WIL BMW 71 C
2004 18 6 TSAT BAR HON 71 C
2004 18 7 MSCH FER FER 71 C
2004 18 8 FMAS SAU FER 71 C
2004 18 9 GFIS SAU FER 71 C
2004 18 10 JVIL REN REN 70 C
2004 18 11 DCOU MCL MER 70 C
2004 18 12 JTRU TOY TOY 70 C
2004 18 13 RZON TOY TOY 70 C
2004 18 14 CKLI JAG FOR 69 C
2004 18 15 TGLO JOR FOR 69 C
2004 18 16 ZBAU MIN FOR 67 C
2004 18 17 GBRU MIN FOR 67 C
2004 18 18 MWEB JAG FOR 23 DNF Accident
2004 18 19 NHEI JOR FOR 15 DNF Clutch
2004 18 20 JBUT BAR HON 3 DNF Engine

Input file - points.csv
 Comma separated file
 16181 rows
 Fields are:
 season_key
 race_key
 position
 driver_points
 team_points
2004 17 17 0 0
2004 17 18 0 0
2004 17 19 0 0
2004 17 20 0 0
2004 18 1 10 10
2004 18 2 8 8
2004 18 3 6 6
2004 18 4 5 5
2004 18 5 4 4
2004 18 6 3 3
2004 18 7 2 2
2004 18 8 1 1
2004 18 9 0 0
2004 18 10 0 0
2004 18 11 0 0
2004 18 12 0 0
2004 18 13 0 0
2004 18 14 0 0
2004 18 15 0 0
2004 18 16 0 0
2004 18 17 0 0
2004 18 18 0 0
2004 18 19 0 0
2004 18 20 0 0

CAR table
 CAR table and index definitions
CREATE TABLE car
(
season_key NUMBER NOT NULL,
race_key NUMBER NOT NULL,
position NUMBER NOT NULL,
driver_key VARCHAR2(4) NOT NULL,
team_key VARCHAR2(3) NOT NULL,
engine_key VARCHAR2(3) NOT NULL,
laps_completed NUMBER NOT NULL,
classification_key VARCHAR2(4) NOT NULL,
notes VARCHAR2(100),
driver_points NUMBER NOT NULL DEFAULT 0,
team_points NUMBER NOT NULL DEFAULT 0
);
ALTER TABLE car ADD CONSTRAINT car_pk
PRIMARY KEY (season_key,race_key,position);
CREATE INDEX car_driver
ON car (season_key,driver_key,driver_points);

CAR
 CAR table relational integrity definitions
ALTER TABLE car ADD CONSTRAINT car_race
FOREIGN KEY (season_key,race_key)
REFERENCES race (season_key,race_key);
ALTER TABLE car ADD CONSTRAINT car_driver
FOREIGN KEY (driver_key)
REFERENCES driver (driver_key);
ALTER TABLE car ADD CONSTRAINT car_team
FOREIGN KEY (team_key)
REFERENCES team (team_key);
ALTER TABLE car ADD CONSTRAINT car_engine
FOREIGN KEY (engine_key)
REFERENCES engine (engine_key);
ALTER TABLE car ADD CONSTRAINT car_classification
FOREIGN KEY (classification_key)
REFERENCES classification (classification_key);

For each line in car.csv
{
read :season_key, :race_key, :position, :driver_key, :team_key,
:engine_key, :laps_completed, :classification_key, :notes;
INSERT INTO car
(season_key, race_key, position,
driver_key, team_key, engine_key,
laps_completed, classification_key, notes)
VALUES
(:season_key,:race_key,:position,
:driver_key, :team_key, :engine_key,
:laps_completed, :classification_key, :notes)
COMMIT;
}
Baseline - Insert

Baseline - Insert
Redo Generation for each Insert
Statement
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo insert row in CAR tableINSERT
11.2Redo Insert row in CAR tableINSERT
Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT
Redo 10.2 Insert row into CAR_PK indexINSERT
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT
Redo 10.2 Insert row into CAR_DRIVER indexINSERT
Commit 5.4 End TransactionCOMMIT
Oracle 9.2 and below

Insert Statement
Redo Generation for each
Insert Statement
Oracle 10.1 and above

Baseline - Update
For each line in points.csv
{
read :season_key, :race_key, :position, :driver_points, :team_points;
SELECT driver_key, team_key, engine_key, laps_completed,
classification_key, notes
INTO :driver_key, :team_key, :engine_key, :laps_completed,
:classification_key, :notes
FROM car
WHERE season_key = :season_key
AND race_key = :race_key
AND position = :position
FOR UPDATE;
UPDATE car SET
driver_key = :driver_key, team_key = :team_key,engine_key = :engine_key,
laps_completed = :laps_completed,classification_key = :classification_key,
notes = :notes, driver_points = :driver_points, team_points = :team_points
AND position = :position;
COMMIT;
}

Baseline - Update
Update Statement
5.1 (11.1)Undo Undo update row in CAR tableUPDATE
11.5Redo Update row in CAR tableUPDATE
Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE
Redo 10.4 Delete row from CAR_DRIVER indexUPDATE
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE
Redo 10.2 Insert row into CAR_DRIVER indexUPDATE
5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE
11.4Redo Lock row in CAR tableSELECT FOR UPDATE
Oracle 9.2 and below

Baseline - Update
Oracle 10.1 and aboveRedo Generation for each
Update Statement

Baseline - Results
 Redo Generation in Bytes
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
 Note
 Amount of redo generated by both INSERT and UPDATE can be variable
due to
 Undo segment management
 Recursive DDL statements e.g. extent allocation
 Block cleanouts

Test 1
 Check for unused indexes
 CAR_PK indexes columns
 SEASON_KEY
 RACE_KEY
 POSITION
 supports primary key therefore mandatory
 CAR_DRIVER indexes columns
 SEASON_KEY
 DRIVER_KEY
 DRIVER_POINTS
 no longer required by current version of application
DROP INDEX car_driver;

Test 1 - Insert
Redo Generation for each Insert
Statement
STOP

Test 1 - Update
Update Statement
STOP

Test 1 - Results
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
Test 1 14,687,756 12,467,400 27,155,156
 Conclusion
 Eliminating redundant index reduced
 insert redo generation by 5,761,096 bytes
 update redo generation by 1,942,276 bytes

Test 2
 In UPDATE statements
 For tables undo and redo is generated for all columns in SET clause
 For indexes undo and redo are only generated for index keys that have
changed
 Statements often update all columns to reduce parsing e.g.:
UPDATE car
SET
driver_key = :driver_key,
team_key = :team_key,
engine_key = :engine_key,
laps_completed = :laps_completed,
classification_key = :classification_key,
notes = :notes,
driver_points = :driver_points,
team_points = :team_points

juliandyke.com© 2013 Julian Dyke
Update Statement
26
Test 2 - Update
Slot = 23
Col 3 = JMON
Col 4 = WIL
Col 5 = BMW
Col 6 = 71
Col 7 = C
Col 8 = <Null>
Col 9 = 10
Col 10= 10
Slot = 23
Col 3 = JMON
Col 4 = WIL
Col 5 = BMW
Col 6 = 71
Col 7 = C
Col 8 = <Null>
Col 9 = 0
Col 10= 0
11.5Redo
5.1 (11.1)Undo
Slot = 23
Col 3 = JMON
Col 4 = WIL
Col 5 = BMW
Col 6 = 71
Col 7 = C
Col 8 = <Null>
Col 9 = 10
Col 10= 10
Slot = 23
Col 3 = JMON
Col 4 = WIL
Col 5 = BMW
Col 6 = 71
Col 7 = C
Col 8 = <Null>
Col 9 = 0
Col 10= 0
STOP

Test 2
 Only update columns which can have new values
 DRIVER_POINTS
 TEAM_POINTS
{
SELECT ... FOR UPDATE;
UPDATE car SET
notes = :notes,
COMMIT;
}
{
SELECT ... FOR UPDATE;
UPDATE car SET
notes = :notes,
COMMIT;
}

Test 2 - Results
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
Test1 14,687,756 12,467,400 27,155,156
Test2 14,560,052 11,584,760 26,144,812
 Conclusion
 Eliminating unnecessary columns from update statements reduced
update redo generation by 882,640 bytes
 Would be significantly more if unchanged columns included long fields
e.g. CHAR, or VARCHAR2

Test 3
 Eliminate unnecessary SELECT FOR UPDATE statements
{
FROM car
FOR UPDATE;
UPDATE car SET
driver_points = :driver_points, team_points = :team_points
COMMIT;
}
{
FROM car
FOR UPDATE;
UPDATE car SET
COMMIT;
}

Test 3 - Update
Update Statement
STOP

Test 3 - Results
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
Test 1 14,687,756 12,467,400 27,155,156
Test 2 14,560,052 11,584,760 26,144,812
Test 3 14,554,428 8,475,484 23,029,912
 Conclusion
 Eliminating SELECT FOR UPDATE statement reduced update redo
generation by 3,109,276 bytes

Test 4
 Rows are inserted with default values of 0 for DRIVER_POINTS and
TEAM_POINTS
 Points only scored by
 first ten cars – 2010 onwards
 first eight cars – 2003 to 2010
 first six cars - pre 2003
 Only update rows with non-zero rows for DRIVER_POINTS and/or
TEAM_POINTS
Team
Driver
Points No Points
Points 3514 30
No Points 324 12313
Driver Team
324 3514 30 12313
STOP

Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
Test 4 - Update
Update Statement
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
UPDATE car SET
driver_points = 1
team_points = 1
WHERE ...
col9 = 0 col10 = 0
col9 = 1 col10 = 1
UPDATE car SET
driver_points = 0
team_points = 0
WHERE ...
col9 = 0 col10 = 0
col9 = 0 col10 = 0
UPDATE car SET
driver_points = 0
team_points = 0
WHERE ...
col9 = 0 col10 = 0
col9 = 0 col10 = 0
UPDATE car SET
driver_points = 9
team_points = 9
WHERE ...
col9 = 0 col10 = 0
col9 = 9 col10 = 9
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
UPDATE car SET
driver_points = 9
team_points = 9
WHERE ...
col9 = 0 col10 = 0
col9 = 9 col10 = 9
STOP

Test 4
 Only update rows with non-zero rows for DRIVER_POINTS and/or
TEAM_POINTS
{
UPDATE car SET
COMMIT;
}
{
IF driver_points != 0 OR team_points != 0 THEN
{
UPDATE car SET
COMMIT;
}
}

Test 4 - Results
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
Test 1 14,687,756 12,467,400 27,155,156
Test 2 14,560,052 11,584,760 26,144,812
Test 3 14,554,428 8,475,484 23,029,912
Test 4 14,683,408 2,070,316 16,753,724
 Conclusions
 Eliminating unnecessary update statements reduced update redo
generation by 6,405,168 bytes

{
INSERT INTO car
VALUES
COMMIT;
}
{
INSERT INTO car
VALUES
COMMIT;
}
COMMIT;
Test 5
 Eliminate unnecessary COMMIT statements

{
{
UPDATE car SET
COMMIT;
}
}
{
{
UPDATE car SET
COMMIT;MMIT;
}
}
COMMIT;
Test 5
 Eliminate unnecessary COMMIT statements (continued)

Test 5 - Insert
Insert Statement
STOP

Test 5 - Results
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
Test 1 14,687,756 12,467,400 27,155,156
Test 2 14,560,052 11,584,760 26,144,812
Test 3 14,554,428 8,475,484 23,029,912
Test 4 14,683,408 2,070,316 16,753,724
Test 5 9,516,512 1,028,084 10,544,596
 Conclusion
 Eliminating COMMIT statements reduced
 insert redo generation by 5,166,896 bytes
 update redo generation by 1,042,232 bytes

Test 6
 Default batch size is 1
 Test INSERT and UPDATE with different batch sizes
Batch Size INSERT Redo UPDATE Redo Total Redo
1 9,517,096 1,028,084 10,545,180
2 5,654,136 1,028,084 6,682,220
4 3,927,092 1,028,440 4,955,532
8 3,011,944 1,028,084 4,040,028
16 2,588,540 1,028,636 3,617,176
32 2,375,884 1,028,172 3,404,056
64 2,254,936 1,028,040 3,282,976
128 2,195,876 1,028,084 3,223,960
256 2,179,404 1,028,440 3,207,844
512 2,163,816 1,028,084 3,191,900
1024 2,163,084 1,028,084 3,191,168
2048 2,160,012 1,028,084 3,188,096

Test 6 - Results
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
9000000
10000000
1 2 4 8 16 32 64 128 256 512 1024 2048
Batch Size
Redo(bytes)

Test 6 - Results
Operation INSERT
(car.csv)
UPDATE
(points.csv)
Total
Baseline 20,448,852 14,409,676 34,858,528
Test 1 14,687,756 12,467,400 27,155,156
Test 2 14,560,052 11,584,760 26,144,812
Test 3 14,554,428 8,475,484 23,029,912
Test 4 14,683,408 2,070,316 16,753,724
Test 5 9,516,512 1,028,084 10,544,596
Test 6 2,195,876 1,028,084 3,223,960
 Conclusion
 Batch Size of 128
 reduced insert redo generation by 7,320,636 bytes
 update redo generation unaffected

Test 7
 Create global temporary table
CREATE GLOBAL TEMPORARY TABLE temporary_car
(
season_key VARCHAR2(4),
race_key VARCHAR2(2),
position NUMBER,
driver_key VARCHAR2(4),
team_key VARCHAR2(3),
engine_key VARCHAR2(3),
laps_completed NUMBER,
classification_key VARCHAR2(4),
notes VARCHAR2(100),
driver_points NUMBER,
team_points NUMBER
)
ON COMMIT PRESERVE ROWS;

Test 7
 Insert rows into global temporary table
{
INSERT INTO temporary_car
VALUES
COMMIT;
}
 Generated 64,140 bytes of redo

Test 7
 Update points in global temporary table
{
{
UPDATE temporary car SET
}
}
COMMIT;
 Generated 652,884 bytes of redo

Test 7
 Copy rows from global temporary table to permanent table
INSERT INTO car
(
season_key, race_key, position, driver_key, team_key, engine_key,
laps_completed, classification_key, notes, driver_points, team_points
)
SELECT
season_key, race_key, position, driver_key, team_key, engine_key,
laps_completed, classification_key, notes, driver_points, team_points
FROM temporary_car;
 Generated 2,166,724 bytes of redo
 APPEND hint had no effect
 Direct load is disabled if table has referential constraints

Test 7 - Results
 Conclusion
 Global Temporary Table reduced total redo generation by 340,212 bytes
Operation Description Total
Baseline Update all rows 34,858,528
Test 1 Update affected rows 27,155,156
Test 2 Update affected columns 26,144,812
Test 3 Drop index 23,029,912
Test 4 SELECT FOR UPDATE 16,753,724
Test 5 COMMIT 10,544,596
Test 6 Increase Batch Size 3,223,960
Test 7 Global Temporary Table 2,883,748

Test 8
 Create external tables
CREATE OR REPLACE DIRECTORY external_dir AS '/u01/app/oracle/gp';
CREATE TABLE external_points
(
position NUMBER,
driver_points NUMBER,
team_points NUMBER
)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_LOADER
DEFAULT DIRECTORY external_dir
ACCESS PARAMETERS
(
RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
)
LOCATION ('points.csv')
);

Test 8
CREATE TABLE external_car
(
position NUMBER,
driver_key VARCHAR2(4),
team_key VARCHAR2(3),
engine_key VARCHAR2(3),
laps_completed NUMBER,
classification_key VARCHAR2(4),
notes VARCHAR2(100)
)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_LOADER
DEFAULT DIRECTORY external_dir
ACCESS PARAMETERS
(
RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
MISSING FIELD VALUES ARE NULL
)
LOCATION ('car.csv')
);

Test 8
 Insert directly into permanent table joining contents of both external tables
INSERT INTO car
(
season_key, race_key, position, driver_key, team_key,
engine_key, laps_completed, classification_key, notes, driver_points,
team_points
)
SELECT
c.season_key, c.race_key, c.position, c.driver_key, c.team_key,
c.engine_key, c.laps_completed, c.classification_key, c.notes,
p.driver_points, p.team_points
FROM external_car c, external_points p
WHERE c.season_key = p.season_key
AND c.race_key = p.race_key
AND c.position = p.position";
 Generated 2,166,724 bytes of redo

Test 8 - Results
 Conclusion
 External Tables reduced total redo generation by 717,024 bytes
Operation Description Total
Baseline Update all rows 34,858,528
Test 1 Update affected rows 27,155,156
Test 2 Update affected columns 26,144,812
Test 3 Drop index 23,029,912
Test 4 SELECT FOR UPDATE 16,753,724
Test 5 COMMIT 10,544,596
Test 6 Increase Batch Size 3,223,960
Test 7 Global Temporary Table 2,883,748
Test 8 External Table 2,166,724

Conclusion
 We have seen that the following techniques can be used to reduce the
amount of redo generated:
 Eliminating redundant indexes
 Reducing the number of columns updated
 Eliminating redundant SELECT FOR UPDATE statements
 Reducing the number of rows processed
 Eliminating COMMIT statements
 Increasing the batch size
 Using Global Temporary Tables
 Using External Tables

Thank you for listening
info@juliandyke.com

B25 Reducing redo by Julian Dyke

Recommended

Recommended

More Related Content

Similar to B25 Reducing redo by Julian Dyke

Similar to B25 Reducing redo by Julian Dyke (7)

More from Insight Technology, Inc.

More from Insight Technology, Inc. (20)

Recently uploaded

Recently uploaded (20)

B25 Reducing redo by Julian Dyke