1. Mining the AWR repository for
Capacity Planning, Visualization,
& other real world stuff

2. Who am I?
• Karl Arao, Oracle ACE, OCP‐DBA, RHCE
• Currently @ Enkitec ‐ Senior Technical Consultant
• Formerly @ SQL*Wizard ‐ Solutions Architect
• Blog: http://karlarao.wordpress.com
• Wiki: http://karlarao.tiddlyspot.com
• Twitter: http://twitter.com/karlarao

3. What will I talk about?

5. Overwhelming

7. DBA_HIST_* views

8. My first close encounter

9. gc block lost – sudden slow down
http://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost

10. gc block lost – sudden slow down
http://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost

11. gc block lost – sudden slow down
http://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost

12. gc block lost – sudden slow down
http://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost

13. After gc block lost – normal workload
http://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost

14. Utilization = Requirement / Capacity

15. Double Y Axis Graph

16. t1 ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐> t0
335 – 336 – 337 – 338 – 339

23. delta issue

27. AWR Scripts

28. Visualization

29. Can’t go back in time?

30. AAS – Average Active Sessions
Kyle Hailey: http://www.perfvision.com/ftp/class/02_AAS.ppt
Max CPU
Max CPU

31. AAS – the Golden Metric
AAS & CPU count as a yardstick for a possible performance problem:
if AAS < 1
‐‐ Database is not blocked
AAS ~= 0
‐‐ Database basically idle
‐‐ Problems are in the APP not DB
AAS < # of CPUs
‐‐ CPU available
‐‐ Database is probably not blocked
‐‐ Are any single sessions 100% active?
AAS > # of CPUs
‐‐ Could have performance problems
AAS >> # of CPUS
‐‐ There is a bottleneck

32. AAS from V$ACTIVE_SESSION_HISTORY
AAS = Sample Count / Elapsed Time CPU count = 4
= 19410 / 600
= 32.35

33. AAS from DBA_HIST_ACTIVE_SESS_HISTORY
AAS = (Sample Count * 10) / Elapsed Time CPU count = 4
= (1950 * 10) / 600
= 32.5

34. AAS from AWR Report
AAS = DB Time / Elapsed Time CPU count = 4
= 291.81 / 9.10
= 32.07

35. AAS from AWR Top Events
AAS = DB Time / Elapsed Time CPU count = 4
291.81 / 9.10 = 32.07
AAS = Event Time / Elapsed Time
17410 / 546 = 31.9

36. awr_topevents.sql

37. Textual trends

41. AAS throughout the AWR retention
period!
http://karlarao.wordpress.com/2010/07/25/graphing-the-aas-with-perfsheet-a-la-enterprise-manager

42. Capacity Planning

43. Utilization is the ultimate metric!

44. awr_genwl.sql

45. http://karlarao.wordpress.com/2010/01/31/workload-characterization-using-dba_hist-tables-and-ksar

46. U = R / C

47. where aas > 1

48. Filter the data points
• AAS range
aas > 1
• Per SNAP_ID or range of SNAP_IDs
id in (336)
where id >= 336 and id <= 340
• Oracle CPU Utilization
oracpupct > 50
• OS CPU Utilization
oscpupct > 50
• Workload periods
AND TO_CHAR(s0.END_INTERVAL_TIME,'D') >= 1 ‐‐ Day of week: 1=Sunday 7=Saturday
AND TO_CHAR(s0.END_INTERVAL_TIME,'D') <= 7
AND TO_CHAR(s0.END_INTERVAL_TIME,'HH24MI') >= 0900 ‐‐ Hour
AND TO_CHAR(s0.END_INTERVAL_TIME,'HH24MI') <= 1800
AND s0.END_INTERVAL_TIME >= TO_DATE('2010‐jan‐17 00:00:00','yyyy‐mon‐dd hh24:mi:ss') ‐‐ Data range
AND s0.END_INTERVAL_TIME <= TO_DATE('2010‐aug‐22 23:59:59','yyyy‐mon‐dd hh24:mi:ss‘)

50. core need = # of cores * utilization * 1.25
Database Consolidation Best Practices
http://husnusensoy.files.wordpress.com/2010/05/database‐consolidation‐best‐practices.pdf

53. Total disk IOPS = (IOPS * Read Ratio) + (IOPS * Write Ratio * RAID penalty)
Number of disk = Total disk IOPS / IOPS per disk

55. awr_iowl.sql

56. Average latency issue
60 minutes interval 10 minutes interval

57. latency (ms) = (readtim / phy reads) * 10

58. Linear Regression

63. x data (CPU) = is the "independent value", used to predict the value of y
y data (AAS) = is the "dependent value", variable whose value is to be predicted

69. r2toolkit
Uses the following
inbuilt Oracle functions:
•regr_count
•regr_r2
•regr_intercept
•regr_slope

70. r2toolkit

71. Linear Regression – what’s the value?

72. Linear Regression on 2 node RAC
http://karlarao.tiddlyspot.com/#r2project
racnode1 racnode2

73. Drilling down on the peak workload...
with AAS of 10

74. Drilling down on the peak workload...
with AAS of 10

79. Now on the low workload period...
with AAS of 2.2

80. Now on the low workload period...
with AAS of 2.2

85. Recap
• Mine the beautiful data set
• Visualization tell a story immediately
• Statistics to make sense of data

86. Let the
AWR data set
change your mind set!

87. Thank you!

88. References and Tools
• http://karlarao.wordpress.com
– http://karlarao.tiddlyspot.com/#%5B%5BStorage%20IOPS%2Ccapacity%2Cperformance
%2Ccost%5D%5D
– http://karlarao.tiddlyspot.com/#Statistics
– http://karlarao.tiddlyspot.com/#OraclePerformance
• Tanel Poder @ http://blog.tanelpoder.com
– http://www.tanelpoder.com/files/TPT_public.zip
– http://www.tanelpoder.com/files/PerfSheet.zip
– Neil Gunther & Tanel Poder ‐ Multidimensional Visualization of Oracle Performance
using Barry007 http://arxiv.org/pdf/0809.2532
• Kyle Hailey @ http://ashmasters.com , http://www.perfvision.com
• Craig Shallahamer @ orapub.com
– Introduction To Oracle Server Consolidation
http://resources.orapub.com/product_p/server_consolidation_ppt.htm
• Husnu Sensoy @ husnusensoy.wordpress.com
– Database Consolidation Best Practices
http://husnusensoy.files.wordpress.com/2010/05/database‐consolidation‐best‐
practices.pdf
• Andy Rivenes @ http://www.appsdba.com/pubs.htm
• Neeraj Bhatia @ www.nioug.org/files/Linear_Regression.pdf

89. Contact me through:
karl.arao@enkitec.com