21. 21
Swingbenchの設定
Data Scale は 10(約10GB分) を指定
Automatic Indexingの効果を見るために、PK以外の索引を削除
Swingbench の 仮想ユーザー数は 6 で SOE V2 で負荷かけ
DROP INDEX SOE.WHS_LOCATION_IX;
DROP INDEX SOE.INV_PRODUCT_IX;
DROP INDEX SOE.INV_WAREHOUSE_IX;
DROP INDEX SOE.ADDRESS_CUST_IX;
DROP INDEX SOE.ITEM_ORDER_IX;
DROP INDEX SOE.ITEM_PRODUCT_IX;
DROP INDEX SOE.ORD_SALES_REP_IX;
DROP INDEX SOE.ORD_CUSTOMER_IX;
DROP INDEX SOE.ORD_ORDER_DATE_IX;
DROP INDEX SOE.ORD_WAREHOUSE_IX;
:
:
./charbench -c ../configs/SOE_Server_Side_V2.xml
-cf /home/opc/work/Wallet_ATPAYSHIBAT02.zip
-cs atpayshibat02_tp
-u soe
-p xxxxxxxxxx
-uc 6
-v users,tpm,tps,vresp,dml
-rt 0:20
:
27. 27
Automatic Indexing の レポート(1)
DBMS_AUTO_INDEX.REPORT_ACTIVITYファンクションで
Automatic Indexing の レポートを出力
SET LONG 1000000;
SET LONGC 1000000;
SET LINESIZE 170;
SET PAGESIZE 1000;
ALTER SESSION SET NLS_TIMESTAMP_FORMAT = 'YYYY/MM/DD HH24:MI:SS.FF';
SELECT DBMS_AUTO_INDEX.REPORT_ACTIVITY(
TO_TIMESTAMP('2019/11/23 00:00:00.00’),
TO_TIMESTAMP('2019/11/26 00:00:00.00’),
'TEXT', 'ALL', 'ALL’
) FROM DUAL;
28. 28
Automatic Indexing の レポート(2)
GENERAL INFORMATIONセクション と SUMMARYセクション
DBMS_AUTO_INDEX.REPORT_ACTIVITY(TO_TIMESTAMP('2019/11/2300:00:00.00’) …
----------------------------------------------------------------------
GENERAL INFORMATION
-------------------------------------------------------------------------------
Activity start : 23-11月-2019 00:00:00
Activity end : 26-11月-2019 00:00:00
Executions completed : 77
Executions interrupted : 0
Executions with fatal error : 0
-------------------------------------------------------------------------------
SUMMARY (AUTO INDEXES)
-------------------------------------------------------------------------------
Index candidates : 8
Indexes created (visible / invisible) : 6 (4 / 2)
Space used (visible / invisible) : 1.43 GB (1.18 GB / 251.72 MB)
Indexes dropped : 0
SQL statements verified : 14
SQL statements improved (improvement factor) : 5 (1639.7x)
SQL plan baselines created : 0
Overall improvement factor : 4.9x
-------------------------------------------------------------------------------
29. 29
Automatic Indexing の レポート(3)
MANUL INDEXESセクション と INDEX DETAILSセクション
SUMMARY (MANUAL INDEXES)
-------------------------------------------------------------------------------
Unused indexes : 0
Space used : 0 B
Unusable indexes : 0
-------------------------------------------------------------------------------
INDEX DETAILS
-------------------------------------------------------------------------------
1. The following indexes were created:
*: invisible
-------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------
| Owner | Table | Index | Key | Type | Properties |
----------------------------------------------------------------------------------------------
| SOE | ADDRESSES | SYS_AI_4bz3nuupj3kt5 | CUSTOMER_ID | B-TREE | NONE |
| SOE | CARD_DETAILS | SYS_AI_dt4w4vr174j9m | CUSTOMER_ID | B-TREE | LOCAL |
| SOE | ORDERS | * SYS_AI_3z00frhp9vd91 | WAREHOUSE_ID | B-TREE | NONE |
| SOE | ORDERS | SYS_AI_5p2zapcmkj174 | CUSTOMER_ID | B-TREE | NONE |
| SOE | ORDERS | SYS_AI_gbwwy984mc1ft | SALES_REP_ID | B-TREE | NONE |
| SOE | PRODUCT_DESCRIPTIONS | * SYS_AI_20tjdcuwznyhx | PRODUCT_ID | B-TREE | NONE |
----------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------
30. 30
Automatic Indexing の レポート(4)
VERIFICATIONセクション(抜粋)
VERIFICATION DETAILS
-------------------------------------------------------------------------------
1. The performance of the following statements improved:
-------------------------------------------------------------------------------
Parsing Schema Name : SOE
SQL ID : 29qp10usqkqh0
SQL Text : SELECT TT.ORDER_TOTAL, TT.SALES_REP_ID, TT.ORDER_DATE,
CUSTOMERS.CUST_FIRST_NAME, CUSTOMERS.CUST_LAST_NAME FROM
(SELECT ORDERS.ORDER_TOTAL, ORDERS.SALES_REP_ID,
ORDERS.ORDER_DATE, ORDERS.CUSTOMER_ID, RANK() OVER (ORDER
BY ORDERS.ORDER_TOTAL DESC) SAL_RANK FROM ORDERS WHERE
ORDERS.SALES_REP_ID = :B1 ...
Improvement Factor : 683.4x
Execution Statistics:
-----------------------------
Original Plan Auto Index Plan
---------------------------- ----------------------------
Elapsed Time (s): 1235522012 13024
CPU Time (s): 1231284830 9898
Buffer Gets: 257075897 590
Optimizer Cost: 5174 1222
Disk Reads: 0 3
Direct Writes: 0 0
Rows Processed: 551468 12
Executions: 816 1
31. 31
Automatic Indexing の レポート(5)
PLANS SECTION(抜粋)
PLANS SECTION
----------------------------------------------------------------
- Original
-----------------------------
Plan Hash Value : 2280072513
----------------------------------------------------------------
| Id | Operation | Name |
----------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | HASH JOIN | |
| 2 | JOIN FILTER CREATE | :BF0001 |
| 3 | PART JOIN FILTER CREATE | :BF0000 |
| 4 | NESTED LOOPS | |
| 5 | NESTED LOOPS | |
| 6 | STATISTICS COLLECTOR | |
| 7 | VIEW | |
| 8 | WINDOW SORT PUSHED RANK | |
| 9 | PARTITION HASH ALL | |
| 10 | TABLE ACCESS STORAGE FULL | ORDERS |
| 11 | INDEX UNIQUE SCAN | CUSTOMERS_PK |
| 12 | TABLE ACCESS BY GLOBAL INDEX ROWID | CUSTOMERS |
| 13 | JOIN FILTER USE | :BF0001 |
| 14 | PARTITION HASH JOIN-FILTER | |
| 15 | TABLE ACCESS STORAGE FULL | CUSTOMERS |
----------------------------------------------------------------
- With Auto Indexes
-----------------------------
Plan Hash Value : 3608924074
---------------------------------------------------------------------------------
| Id | Operation | Name |
---------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | NESTED LOOPS | |
| 2 | NESTED LOOPS | |
| * 3 | VIEW | |
| * 4 | WINDOW SORT PUSHED RANK | |
| 5 | TABLE ACCESS BY GLOBAL INDEX ROWID BATCHED | ORDERS |
| * 6 | INDEX RANGE SCAN | SYS_AI_gbwwy984mc1ft |
| * 7 | INDEX UNIQUE SCAN | CUSTOMERS_PK |
| 8 | TABLE ACCESS BY GLOBAL INDEX ROWID | CUSTOMERS |
---------------------------------------------------------------------------------
44. 44
Hybrid Partition表を作成(DBMS_CLOUD)
DBMS_CLOUD.CREATE_HYBRID_PART_TABLEプロシージャで
Hybrid Partition表を作成
BEGIN
DBMS_CLOUD.CREATE_HYBRID_PART_TABLE(
table_name => 'EXT_HYBRID_TEST', --テーブル名
credential_name => 'OCI_CLI_AYS_ATP', --クレデンシャル名
format => json_object(
'characterset' VALUE 'AL32UTF8' -- 文字コードを指定
, 'recorddelimiter' VALUE '''n''' -- 行区切り文字
, 'type' VALUE 'csv' -- ファイルのタイプを指定
, 'skipheaders' VALUE 1 -- 読み飛ばすヘッダ行数を指定
, 'compression' VALUE 'gzip' -- 圧縮形式を指定
, 'dateformat' VALUE 'YYYY/MM/DD HH24:MI:SS' -- DATE型のフォーマットを指定
) ,
column_list => 'ID NUMBER, ' ||
'HYBRID_STR VARCHAR2(43), ' ||
'HYBRID_DATE DATE' ,
partitioning_clause => 'PARTITION BY RANGE (ID) (' ||
' PARTITION P100 VALUES LESS THAN (101) ' ||
' EXTERNAL LOCATION(''https://objectstorage.ap-tokyo-1…/hybrid_test_p100.csv.gz'') ' ||
' , PARTITION P200 VALUES LESS THAN (201) ' ||
' EXTERNAL LOCATION(''https://objectstorage.ap-tokyo-1…/hybrid_test_p200.csv.gz'') ' ||
' , PARTITION P300 VALUES LESS THAN (301)' ||
' , PARTITION P400 VALUES LESS THAN (401)' ||
')'
);
END;
/
45. 45
Hybrid Partition表にデータをロード
データベース格納部分(P300, P400)のデータをロード
INSERT INTO EXT_HYBRID_TEST
SELECT LEVEL+200 AS ID
, 'STR' || (LEVEL+200) AS HYBRID_STR
, TO_DATE('20191201', 'YYYYMMDD') + (LEVEL+200)/24/60/60 AS HYBRID_DATE
FROM DUAL
CONNECT BY LEVEL <= 200;
COMMIT;
46. 46
Hybrid Partition表をGET_DDLしてみる。
Hybrid Partition表をGET_DDLしてみると下記のようになってます。
SELECT DBMS_METADATA.GET_DDL('TABLE','EXT_HYBRID_TEST','AYSHIBAT') FROM DUAL;
CREATE TABLE "AYSHIBAT"."EXT_HYBRID_TEST"
( "ID" NUMBER,
"HYBRID_STR" VARCHAR2(43) COLLATE "USING_NLS_COMP",
"HYBRID_DATE" DATE
) DEFAULT COLLATION "USING_NLS_COMP" EXTERNAL PARTITION ATTRIBUTES
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY "DATA_PUMP_DIR"
ACCESS PARAMETERS
( RECORDS CHARACTERSET AL32UTF8 DELIMITED BY 'n' IGNORE_HEADER=1 COMPRESSION GZIP NOLOGFILE NOBADFILE NODISCARDFILE READSIZE=10000000
CREDENTIAL 'OCI_CLI_AYS_ATP' FIELDS CSV WITHOUT EMBEDDED DATE_FORMAT DATE MASK 'YYYY/MM/DD HH24:MI:SS' NOTRIM )
REJECT LIMIT 0
) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255
PARTITION BY RANGE ("ID")
(
PARTITION "P100" VALUES LESS THAN (101) EXTERNAL
LOCATION( 'https://objectstorage.ap-tokyo-1.oraclecloud.com/…/hybrid_test_p100.csv.gz'),
PARTITION "P200" VALUES LESS THAN (201) EXTERNAL
LOCATION( 'https://objectstorage.ap-tokyo-1.oraclecloud.com/…/hybrid_test_p200.csv.gz'),
PARTITION "P300" VALUES LESS THAN (301) SEGMENT CREATION IMMEDIATE PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
STORAGE(INITIAL 8388608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT
FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT) TABLESPACE "DATA" ,
PARTITION "P400" VALUES LESS THAN (401) SEGMENT CREATION IMMEDIATE PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
STORAGE(INITIAL 8388608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT
FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT) TABLESPACE "DATA"
) PARALLEL
60. 60
統計ベースの問合せ変換・テストデータ
簡単なテストデータを作成(1000万件)して統計採取
CREATE TABLE STATISTICS_TEST
AS SELECT LEVEL AS ID
, 'STR' || LEVEL AS STATISTICS_STR
, TO_DATE('20191201', 'YYYYMMDD') + LEVEL/24/60/60 AS STATISTICS_DATE
FROM DUAL
CONNECT BY LEVEL <= 10000000;
EXEC DBMS_STATS.GATHER_TABLE_STATS('AYSHIBAT', 'STATISTICS_TEST', METHOD_OPT =>
'FOR ALL COLUMNS SIZE AUTO FOR COLUMNS ID SIZE AUTO', NO_INVALIDATE => FALSE,
DEGREE => 6);
61. 61
統計ベースの問合せ変換・SQL実行&PLAN確認
COUNT(*)なSQLで実行計画を確認したが……
SET LINESIZE 300;
SET PAGESIZE 1000;
SELECT COUNT(*) FROM STATISTICS_TEST;
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR());
PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------
SQL_ID 9w2bk9yx44317, child number 0
-------------------------------------
SELECT COUNT(*) FROM STATISTICS_TEST
Plan hash value: 709000224
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | 548 (100)| |
| 1 | SORT AGGREGATE | | 1 | | |
| 2 | TABLE ACCESS STORAGE FULL| STATISTICS_TEST | 10M| 548 (11)| 00:00:01 |
--------------------------------------------------------------------------------------
69. 69
リアルタイム統計・テストデータ
簡単なテストデータを作成(10000件)して統計採取
CREATE TABLE STATISTICS_TEST
AS SELECT LEVEL AS ID
, 'STR' || LEVEL AS STATISTICS_STR
, TO_DATE('20191201', 'YYYYMMDD') + LEVEL/24/60/60 AS STATISTICS_DATE
FROM DUAL
CONNECT BY LEVEL <= 10000;
EXEC DBMS_STATS.GATHER_TABLE_STATS('AYSHIBAT', 'STATISTICS_TEST', METHOD_OPT =>
'FOR ALL COLUMNS SIZE AUTO FOR COLUMNS ID SIZE AUTO', NO_INVALIDATE => FALSE,
DEGREE => 6);
70. 70
リアルタイム統計(動作前)・テストデータの表統計
リアルタイム統計動作前の
表統計(DBA_TAB_STATISTICS)は下記の通り
ALTER SESSION SET NLS_DATE_FORMAT = 'YYYY/MM/DD HH24:MI:SS';
SELECT TABLE_NAME, NUM_ROWS, LAST_ANALYZED, NOTES
FROM DBA_TAB_STATISTICS
WHERE TABLE_NAME = 'STATISTICS_TEST'
AND OWNER = 'AYSHIBAT';
TABLE_NAME NUM_ROWS LAST_ANALYZED NOTES
------------------------------ ---------- ------------------- -------------------------
STATISTICS_TEST 10000 2019/12/03 07:56:18
73. 73
リアルタイム統計・テストデータを追加投入
テストデータを追加投入(100件)+おまじない
INSERT INTO STATISTICS_TEST
SELECT (LEVEL+10000) AS ID
, 'STR' || (LEVEL+10000) AS
STATISTICS_STR
, TO_DATE('20191201', 'YYYYMMDD') + (LEVEL+10000)/24/60/60 AS
STATISTICS_DATE
FROM DUAL
CONNECT BY LEVEL <= 100;
COMMIT;
EXEC DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO;
74. 74
リアルタイム統計(動作後)・テストデータの表統計
リアルタイム統計動作後の表統計(DBA_TAB_STATISTICS)は
下記の通り、何か追加されてますね……。
ALTER SESSION SET NLS_DATE_FORMAT = 'YYYY/MM/DD HH24:MI:SS';
SELECT TABLE_NAME, NUM_ROWS, LAST_ANALYZED, NOTES
FROM DBA_TAB_STATISTICS
WHERE TABLE_NAME = 'STATISTICS_TEST'
AND OWNER = 'AYSHIBAT';
TABLE_NAME NUM_ROWS LAST_ANALYZED NOTES
------------------------------ ---------- ------------------- -------------------------
STATISTICS_TEST 10000 2019/12/03 07:56:18
STATISTICS_TEST 10100 2019/12/03 07:59:20 STATS_ON_CONVENTIONAL_DML