After passing all functional tests, many web applications fail under the regular load conditions. Just 100 active users may be sufficient to cause severe errors and a decline in performance. In this session, load testing of ADF applications with the Oracle Application Testing Suite and Apache JMeter will be discussed and demonstrated. With these tools, you have an analysis-instrument to do performance analysis and regression analysis after new patches/releases. A proper managed load test also gives insight in the effects configuration parameters. The speaker will share best test practices to perform load tests and offer typical solutions for creating an efficient, scalable ADF application.

1. Frank Houweling
AMIS preview Oracle Open World 2014
Real-World Load Testing of ADF Fusion Applications

2. Frank Houweling
•Frank Houweling
•Senior Java / ADF specialist
•Focus on performance diagnosis and performance management
•Lead developer of the ADF Performance Monitor

3. 3
Agenda
•Why is testing important
•Intro Apache JMeter and Oracle Load Testing (OLT)
•Demo
•Metrics analysis
•Top 3 scalability problems and solutions

4. 4
Why is load testing important - 1

5. 5
Why is load testing important - 2
•Is very useful to test scalability and SLA
–Avoid frustrated end-users
–Avoid lost revenue

6. Challenges Load Testing ADF Application
•Trying out / testing the most optimal:
–ApplicationModule pooling parameter settings
–JVM heap size & garbage collection strategy
–Datasource configuration (number of database connections)
–Number of WebLogic nodes
–Infrastructure/hardware usage & licenses

7. 7
You need metrics to be in control and know what's going on

8. 8
Introducing Apache JMeter
•Open source desktop application, designed to load test functional behavior and measure performance.
•open source software
•100% pure Java application
•not easy to configure
•but free

9. Introducing Oracle Application Testing Suite (OATS)
•Powerful, integrated solution for ensuring application quality.
Oracle Functional Testing
Oracle Test Manager
Oracle Load Testing
It is Oracle’s solution for testing Oracle Applications and the standard for testing at Oracle For example Fusion/ADF, EBS, Siebel, JDE, FMW, e.g.

10. 10
OpenScript Automates functional and load testing
•OpenScript is used for both creation of functional and load test scripts using record and playback
•Accelerators (templates) for Fusion/ADF, webservices, eBS, JDE, e.g.
•Integrated Eclipse IDE for script development

11. 11
OpenScript Concepts

12. 12
Installing OATS
•Current newest version: 12.4.0.1.0
•Download from OTN

13. 13
Agenda
•Why is testing important
•Intro Apache JMeter and Oracle Load Testing (OLT)
•Demos (NOTE: without sound, play in Firefox or IE)
–Record a load test scenario (1.20 min)
–Execute a load test (3 min)
•Metrics analysis
•Top 3 scalability problems and solutions

14. 14
Analysis load test metrics: Default graphs in OLT
•Default graphs
–Active Virtual Users
•number of virtual users currently running
–Virtual Users with Errors
•number of virtual users with errors
–Transactions Per Second
•number of times the virtual user played back the script per second
–Pages Per Second
•number of pages returned by the server per second
–Hits Per Second
•number of resource requests to the server per second
–Kilobytes Per Second
•number of kilobytes transferred between the server and browser client per second

15. 15
Analysis load test metrics: Determine the weakest links
•Collect relevant load test metrics
–Number of virtual users
–Average response times
–ApplicationModule pooling activations and passivations
–JVM heap and garbage collections
–Database performance
–Infrastructure metrics
•Bring the metrics together in time
•Analyze results and determine weak links

16. 16
Analysis load test metrics - 1 example: CPU seems to be a bottleneck
throughput
virtual users
CPU usage
• imported CPU metrics from WLS during load test – the CPU seems to be the bottleneck

17. 17
Analysis load test metrics - 2 Monitor in Database EM Control during load test
Example: High CPU Wait in Database EM Control

18. 18
Agenda
•Why is testing important
•Intro Apache JMeter and Oracle Load Testing (OLT)
•Demo
•Metrics analysis
•Top 3 scalability problems and solutions

19. Common scalability problems - 1 Too frequent ‘expensive’ ApplicationModule passivations & activations
Recommendations
•Don’t use the AM pooling default parameter settings; calculate how many AMs you will need during peek times
–Increase maxavailablesize and recyclethreshold to this value
•jbo.ampool.maxavailablesize = jbo.recyclethreshold
–Set minavailablesize to 80% of maxavailablesize
•jbo.ampool.minavailablesize = 80 % of jbo.ampool.maxavailablesize
Result:
•Avoids long running, ‘expensive’ passivations and activations of AMs under normal load
•Best practice: test it during a load test !

20. 20
Monitor ApplicationModule Pooling behavior; activations and passivations
•What is the average, total, min and max execution time of activations and passivations?

21. 21
What is the root cause of long passivations?

22. 22
What is the root cause of long activations?

23. 23
Common scalability problems - 2 Too little JVM Heap size, long running JVM garbage collections
•Avoid too frequent long running JVM garbage collections that freeze all requests
•Set ( -Xms and -Xmx) as large as possible within available physical memory
•Generational parallel garbage collection strategy is recommended to maximize throughput: -Xgc:genpar (JRockit)
•Consider more Weblogic nodes if you have many end-users
•Best practice: test it during a load test !

24. 24
Common scalability problems - 3 Not enough database connections
Solution: increase Maximum Capacity of datasource in WLS console

25. 25
Monitor nr of database connections in WLS console during load test
•Monitor the number of database connections in the Weblogic Console
•set jbo.doconnectionpooling=true and jbo.txn.disconnect_level=1

26. 26
Measure number of database connections in database during load test
(logged in as SYS or SYSDBA)

27. 27
Tuning Challenges
•Requires expertise and is time consuming
–Should be done in time (not 1 week before production)
–Relevant scenario’s should be tested
–All load testing tools take time to become familiar
•Analysis
–What type of operations is the ADF application or database spending most time on ?
–Which resources is the WLS server or database bottlenecked on?
–What is causing these bottlenecks?
–What can be done to resolve them ?

28. 28
Learn More
•Oracle Application Testing Suite on OTN
–Download
–Tutorials
–Demos
–Training
•Free alternative (but more configuration):
Apache Jmeter
•Fusion Developer Guide on ApplicationModule pooling (11.1.2)
•ADF Performance Monitor