3. Overview of PerformanceTesting
Performance testing is an non-functional testing to determine the
system responsiveness i.e. speed, stability, reliability and scalability.
Speed Throughput
Reliability Scalability
4. Purpose of Performance Testing
Performance Testing ensures that your application works under
real-world loads before your customers find out that it doesn't!
GOAL/OBJECTIVE IS TO FIND THE BOTTLE NECK IN THE SYSTEM
6. Goal of Performance Testing
Access the Production Readiness.
Compare two platforms with the same software to see which
performs better.
Compare Performance characteristics of system configurations.
Evaluate System against performance criteria.
Discover what parts of the application perform poorly and under
what conditions.
Finding the source of performance problems.
Support system tuning.
7. Pre-Requisites PerformanceTesting
Stable and defect-free dedicated environment.
Performance testing environment similar to production
environment.
No other testing should be performed while Performance Testing.
Performance testing before going live.
Performance testing plan development.
Test Data Preparation.
Performance testing requirements gathering.
Application architecture.
Servers information.
Application usage information.
Performance Acceptance criteria.
8. Performance Management
There are two approaches of managing the performance testing
activities:
Reactive Approach
Performance is only done after the System testing.
Proactive Approach
Performance parameter is analyzed and addressed in testing
environment before it really impact the production system and fix
it before launching the application.
Non-Functional Requirements Gathering phase
Design phase Development phase
Development phase
Test Execution phase
Maintenance
9. Performance Testing Life Cycle
Typical Performance testing cycle consists of following activities:
1.Establish Performance Test Objectives
2.Prepare test environment
3.Create and modify scripts
4.Execute Performance test
5.Monitor impact of load on servers/databases.
6.Analyze results
7.Tune system
8.Repeat from Step 4
9.Deploy mission critical application with confidence.
10. Why to use performance Testing
tool?
Almost impossible without one!
Without tool relies on massive hardware and personnel to
generate load.
The cost of repeating a 'manual' test never diminishes.
Without a tool, no end reports are created.
Reponses Time
CPU Utilization
Disk I/O
Network I/O
Other
11. Performance Symptoms and Issues
Application Server
Poor database tuning.
Poor cache management.
Poor session management.
Web Server
Poor server design.
Poor configuration & usage.
Database Server
Insufficient indexing.
Fragmented databases.
Faulty application design.
Network
Firewall throughput.
Load balancers, gateways,
routers.
Client Side
Browser settings.
Client system.
12. Typical order of Fixes
Improve current application design: Algorithms, caching, DB
calls, memory use
Upgrade hardware: RAM, CPU, network bandwidth
Upgrade software infrastructure: OS, web server, database
(database connection polling)
Upgrade system architecture: Client-server to basic n-tier, basic
n-tier to enterprise n-tier, software and hardware changes, Use
Apache HTTPD in front of Tomcat to serve static resources,
Use hardware load balancing /SSL.
13. Challenged with Performance
Testing
Following is a list of challenges with Performance Testing:
Test environment setup
Collection and Analysis of huge data
Bottleneck root cause identification
Obtaining accurate results
Client involvement
Testing inside the firewall
Performance testing of new technologies
Testing on live environment
Expensive
Cooperative effort required (Product vendors, Architects,
Developers, Testers, Database administrators, System
administrators, Network administrators)
14. PerformanceTesting Best Practices
Use user Ramp up and Ramp down approach.
Ignore the results collected during Ramp up and Ramp down
periods.
Run individual tests of performance scenarios before combining
them in a single test.
Run a baseline test with single user to validate the script.
Run a benchmark test with 15 to 20 percent of desired load
to validate the system matrices at lower load and check the
system readiness for high load.
Run the test for at least 10-15 minutes at stable peak load.
Repeat tests at least 3 times to confirm the results.
Run tests at different times.
16. What is Jmeter?
Open source tool
Friendly GUI Design
Platform Independent
Full multithreading framework
Visualize Test Results
Easy Installation
Support multi-protocol
Jmeter Works:
Send request to
target server
Get statistics
information of
target server
Generate test
report in different
format
17. What can you test in Jmeter?
Can load and performance test many different server types:
Web - HTTP, HTTPS
SOAP
Database via JDBC
LDAP
JMS
Mail - POP3(S) and IMAP(S)
18. Installing Jmeter
Install Java (2.2. or higher)
Download JMeter
http://jakarta.apache.org/site/downloads/index.html
Add path of java installation in environment path variables.
22. Test Plan
Test plan node is where REAL stuff is kept.
Used for containing the test.
A layout of how and what to test.
Test Plan describe a series of steps JMeter will execute once
the Test Plan runs.
A test plan must have at least one Thread Group.
23. Thread Group
Thread Group is used for representing users.
There could be one or more Thread Groups in a Test Plan.
Each thread group will execute completely independently from
each other.
Thread Group can control:
Number of users simulated (No. of threads).
Ramp Up time (how long it takes to start all the threads).
Number of times to perform the test.
24. Controllers
Controllers used for grouping and applying logic to test items.
It’s of two types:
Controllers
Sampler (Send Request to Server)
Logical Controller (Customize logic to
send request)
25. Samplers
Used for performing actual task.
It allow JMeter to send specific types of requests to a server.
The following is a list of all Samplers JMeter provides:
Sampler
HTTP
Request
FTP
Request
JDBC
Request
Web
Service
(SOAP)
Request
Access
Log
Sampler
Bean
Shell
Sampler
BSF
Sampler
TCP
Sampler
26. Logic Controllers
These allow you to customize the logic that JMeter uses to
decide when to send requests.
The following list consists of all the Logic Controllers JMeter
provides:
Logic Controller
Simple
Controller
Loop
Controller
Once Only
Controller
Random
Controller
Throughput
Controller
If Controller While
Controller
Switch
Controller
Transaction
Controller
Recording
Controller
27. Listeners
Listeners are used for displaying of data.
Listeners provide means to view, save, and read saved test
results.
The following list consists of all the Listeners JMeter provides:
Listeners
Graph Full
Results
Spline
Visualizer
Assertion
Results
View
Results
Tree
Aggregate
Report
View
Results in
Table
Aggregate
Graph
28. Timers
Allow JMeter to delay between each request that a thread
makes.
Think Time is used for emulating real-life visitors/users.
The following list consists of all the Timers JMeter provides:
Timer
Constant
Timer
Uniform
Random
Timer
Gaussian
Random
Timer
Synchronizing
Timer
Bean Shell
Time
29. Assertions
Used for Validating Test.
Allow you to "test" that your application is returning the
results you expect it to.
The following list consists of all the Assertions JMeter
provides:
Assertion
Response
Assertion
Duration
Assertion
Size
Assertion
Bean
Shell
Assertion
HTML
Assertion
XPath
Assertion
30. Configuration Elements
They are used to add or modify requests made by Samplers.
They Works closely with a Sampler, these can add to or
modify requests.
The following list consists of all the Configuration Elements
JMeter provides:
Configuration Element
CSV Data Set
Config
FTP Request
Defaults
HTTP
Authorization
Manager
HTTP Cookie
Manager
HTTP Request
Defaults
JDBC
Connection
Configuration
Login Config
Element
31. Pre-Processor Elements
They Execute prior to sampler request
It's used to modify the settings of a Sample Request just
before it runs.
The following list consists of all the Pre-Processor Elements
JMeter provides:
Pre-Processor Element
HTML Link
Parser
HTTP URL Re-
writing
Modifier
User
Parameters
Bean Shell
PreProcessor
32. Post-Processor Elements
They Execute some action after sampler request.
It's execute after a request has been made from a Sampler.
The following list consists of all the Post-Processor Elements
JMeter provides:
Post-Processor Element
Regular
Expression
Extractor
XPath
Extractor
Save Responses
to a file
Bean Shell
PostProcessor
34. Recording & Playback
Jmeter can act as a proxy server between your browser and
the web recording your actions.
This can help in writing your web tests.
Lets do another small demo?
◦ Proxy Server
◦ Script Recoding Tool (Badboy, Wireshark)
35. Adding Users
5 users send 2 requests on jmeter.apache.org and repeat it
twice. (5 users x 2 requests x 2 repeat = 20 requests)
Right click on test note >> Add >> Thread (users) >> Thread
Group.
36. Adding Default HTTP Request
Properties
This will let you define default http parameter for every
request.
You can add default http request setting from ADD > Config
Element > HTTP request default.
38. Adding Cookie Support
They are normally included to maintains a certain state for
each user, e.g. cookies.
You can add it from ADD > Config Element > HTTP Cookie
Manager.
39. Adding a Listener toView/Store the
Test Results
Listener responsible for storing all of the results of our HTTP
request and presenting in Visualize mode.
You can add it from ADD > Listener > Summary Report.
41. Preparing for Load Testing
Need to address a number of concerns with regards to the
target server under test.
A load testing helps to benchmark performance behavior of a
server, it is important to be able to identify the general
expectations and other matters that would normally be taken
into account in order to carry out a successful load testing.
42. Need to Know
A suitable time to load-test the application, for instance when
no development work is taking place on the server and/or no
other users are accessing the server.
The performance metrics, accepted levels, or SLAs and goals.
Objectives of the test.
The Internet protocol(s) the application is(are) using (HTTPS,
HTTP, FTP, etc.)
If your application has a state, the method used to manage it
(URL rewriting, cookies, etc.)
The workload at normal time and at peak time.
43. Some HelpfulTips to Get Better
Results
Use meaningful test scenarios to construct 'real-life' test cases.
Run JMeter on a machine other than that running the
application.
The machine running JMeter should have sufficient network
bandwidth, memory, CPU to generate load.
Let JMeter Test Plan run for long time periods, hours or
days, or for a large number of iterations.
Ensure that the application is stable and optimized for one
user before testing it for concurrent users.
Incorporate 'thinking time' or delays using Timers in your
JMeter Test Plan.
Keep a close watch on the four main things: processor,
memory, disk, and network.
44. Using Jmeter Components
We will test five key scenarios:-
Homepage
Keyword Search
Create Account
Select A Title
Add To Cart
These scenarios will be included in a single JMeter Test Plan, for
simplicity reasons.
45. Recording HTTP Requests
JMeter Proxy can use to record all request send to server.
Create test plan with default http testing
Add HTTP Proxy Server in Workbench node.
Define port number of proxy server
46. Creating the Test Plan
Right-click on the Test Plan element and select Add | Thread
Group.
Configure the Number of Threads to 10, Ramp-Up Period to 1
second, and Loop Count to 50.
Add to the Test Plan Config Element | HTTP Request
Defaults.
Add Pre-Processor | User Parameter Element or Config
Element | CSV Data Set Config.
48. Running the Test Plan
When Jmeter script gets ready.
Click on “Start”.
49. Interpreting the Results
Once the test is completed, we can now retrieve the results
we have saved for each Controller.
With the exception of the Assertion Result Listener, the saved
data can be viewed in numerous forms.
50. Monitoring the Server's
Performance
There is a special Listener that allows you to monitor the
target server's performance as Samplers make requests.
This Monitor Result Listener is designed to work with Apache
Tomcat version 5 and above.
Some server monitoring tools:
Perfmon
Blazemeter
52. What is Correlation
A Correlation is a Connection or Association.
Capturing the dynamic data that is being generated by the
server is called correlation.
Correlation will be done using the Regular Expression
Extractor in Jmeter.
53. Why Correlation
To overcome script fail we need a way which can capture these
dynamically generated session values and pass it subsequently
to any part of the script, wherever required. This method to
identify and set the dynamic generated value is known as
correlation.
Sample of Regular Expression and Usage:
Session ID = (.+?) to correlate the url/dynamic id.
EX: Session ID = jkjoujn434897h3jh35y9h&OrderID=ikikikke99874kmnjhh2
54. Using Regular Expression Extractor
in Jmeter Tests
Regular expressions are used to search and manipulate text,
based on patterns.
This element will extract text from the current page using a
Regular Expression to identify the text pattern that a desired
element conforms with.
Right click the node > Add > Post Processor > Regular
Expression Extractor
56. What is Parameterization
It is the way of replacing a hard coded value in the script with
a parameter which represents a list of values.
It is a script that contains the actual values used during
recording and during script enhancement phase test engineer
has to replace the recorded values with parameters is known as
parameterizing the script.
57. Why Parameterize
It allows you to test your script with different values.
Replace the constant values in Script with parameters.
simulate real user behavior while running the test
58. Identifying the test data on AUT
While login “Username” & “Password” are parameter for which
test data needed.
While registration “First Name”, “City” etc. is parameter for
which test data needed.
59. Using the CSV Data Config in
Jmeter Tests
Create CSV file with list of username and password.
Store in same folder where your test stores.
Add CSV Data set into your test tree from config elements.
Add ${username},${password} in request sampler as parameter.
61. What is Assertion
Assertions allow you to include some validation test on the
response.
To ensure the responses you receive.
62. Why Assertion
Assertions are used to perform additional checks on samplers.
They are processed after every sampler in the same scope.
63. Types of Assertion in Jmeter
Response Assertion: Compared against various fields of response.
Duration Assertion: Tests each response was received within a
given amount of time.
Size Assertion: Tests each response contains the right number
of bytes in it.
XML Assertion: Tests response data consists of a formally
correct XML document.
BeanShell Assertion: Checking using a BeanShell script.
MD5Hex Assertion: Check the MD5 hash of the response data.
HTML Assertion: Check the HTML syntax of the response.
XPath Assertion: Tests a document for well formedness.
XML Schema Assertion: The XML Schema Assertion allows the
user to validate a response against an XML Schema.
64. Running the tests and analyzing the
Assertion results
Assertions allow you to assert facts about responses received
from the server being tested. Using an assertion, you can
essentially "test" that your application is returning the results
you expect it to.
66. Testing a Database Server
A few things you will need before proceeding to build a
Database Test Plan are:
A working database driver, copy the .jar file contained in the
database driver and pastes it in the lib folder of your JMeter.
A valid database-schema.
Valid non-empty database table(s).
A valid user-level access to the database.
67. Testing an FTP Server
What you will need before proceeding to build a FTP Test Plan
includes:
A running FTP server on the target machine.
A valid path to the shared files in your FTP server.
Valid non-empty files in the FTP installation path.
A valid user-level access to the files.
69. Reducing resource requirements
Do not use GUI mode better use non-GUI mode.
Use Remote and Distributed testing for larger load testing.
Do not load more than 300 threads per Jmeter.
Use naming conventions for all the elements.
Use as few Listeners as possible.
Don't use "View Results Tree" or "View Results in Table"
listeners during the load test; use them only during scripting
phase to debug your scripts.
Rather than using lots of similar samplers, use the same
sampler in a loop, and use variables (CSV Data Set) to vary
the sample.
Use CSV output rather than XML.
Only save the data that you need.
Use as few Assertions as possible.
70. Where to Put the Cookie Manager
The cookie manager stores and sends cookies just like a web
browser.
Received Cookies can be stored as JMeter thread variables.
Manually add a cookie to the Cookie Manager.
71. Where to Put the Authorization
Manager
It’s specify one or more user logins for web pages that are
restricted using server authentication.
72. Bean Shell Server
The Bean Shell interpreter has a very useful feature it can act
as a server, which is accessible by telnet or http.
73. Distributed Testing
Master: The system running Jmeter GUI which control the test.
Slave: The System running Jmeter-server which takes commends
from the GUI and send the requests to the target system.
Target: The Web Server planned for the load test.
JMeter client machine may not able to simulate enough users to
stress server.
Control multiple machine to run JMeter without copying test
samples to different machine.
74. Sever Monitoring Tools (PerfMon)
During a load test, it is important to know the health of the
servers loaded.
To address this, the plugin package now supports server
monitoring!
Detail Link:
https://code.google.com/p/jmeter-plugins/wiki/PerfMon
75. Database Monitoring Tool (Jet
Profiler)
Jet Profiler for MySQL is a query profiling tool for the
MySQL database server.
Query, table and user performance
Graphical Visualization
Low overhead
User friendly
Detail Link:
http://www.jetprofiler.com/
76. Request Snipers Tools
It’s to monitors and analyzes all incoming and outgoing HTTP
traffic between the browser and the web servers.
HttpFox
httpwatch
Gtmetrix
Yslow
Web Page Test
google speed test
