This document discusses the importance of performance testing web applications. It notes that 53% of users will abandon a website if it takes over 3 seconds to load, and 79% of those users will not return. The document outlines different types of performance tests including load testing, endurance testing, spike testing, and stress testing. It emphasizes the need for performance testing to be realistic by simulating real user behavior, network conditions, workloads and data volumes similar to the production environment. The document also discusses analyzing test results and key performance indicators to understand how the system performs under different loads and over time.

1. NEED FOR SPEED - “HOW TO
PERFORMANCE TEST THE RIGHT WAY”
Annie Bhaumik
Senior Performance Test Engineer,
OCLC

2. NEED FOR
SPEED
53% of users abandon the
website, if it takes more than 3
seconds to load.
79% of them do not come
back to visit the website.
3 seconds
That’s how long it takes users to get
impatient and abandon your website.
3s

3. DOES A
SLOWER
WEBSITE
MEAN LOST
DOLLARS?
8.11%
6.32%
4.64%
2.93%
2.24% 2.20%
1.66%
2.11%
1.77% 1.73%
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%
8.00%
9.00%
0 1 2 3 4 5 6 7 8 9
Transaction Conversion Rates
Page Load Speeds(seconds)

4. WHAT IS THE
IDEAL PAGE
LOAD TIME
If 100 people visit a website for a $100 product, this could be
potential earnings:
0-2
seconds

5. REAL
WORLD
EXAMPLES
 Amazon - 1% sales loss for every 100ms extra
load time
 Walmart - 2% increase in conversion for every 1
sec improvement in load time
 Tagman - study showed that 1s delay in load time
resulted in loss of 7% revenue

6. WHAT IS
PERFORMANCE
TESTING
V
I
R
T
U
A
L
U
S
E
R
S
Application Under
Test

7. WHAT CAN
PERFORMANCE
TESTING DO
FORYOU
How many users
can the
application handle
?
What happens to
user experience
when users
increase to 500%
Do people in
different parts of
the world get the
same user
experience?
At what point
does the
application start
slowing or
crashing?
Which way is
performance
trending over
time?
What are my
Benchmarks and
Baselines

8. TYPES OF
PERFORMANCE
TESTPerformance
Testing
Load
Testing
Endurance
Testing
Spike
Testing
Stress
Testing
Volume
Testing
Scalability
Testing

9. LOAD TEST  Load testing is performed to
determine a system's behavior under
both normal and anticipated peak
load conditions.
Are desired
performance
objectives specified in
SLA met :
• Response Times
• Throughput
• Resource Utilization

10. ENDURANCE
TEST  Testing a system with expected
amount of load over a long period of
time to find the behaviour of system• Will performance be
consistent over
time?
• Are there slowly
growing problems
that have not yet
been detected?

11. SPIKE
TEST
 Spike Testing is to determine the
behavior of the system under sudden
increase of load (a large number of
users) on the system• What happens if the
production load exceeds
the anticipated peak
load?
• What kinds of failures
should we plan for?
• What indicators should
we look for?

12. STRESS
TEST
 Stress testing involves testing an
application under extreme workloads to
see how it handles high traffic or data
processing. The objective is to identify the
breaking point of an application
• Reveals
application bugs
that surface only
under high load
conditions
?
• Identify
application’s weak
points

13. VOLUME
TEST  Testing a software application with a
large amount of data to be
processed to check the efficiency of
the application
• Identify problems
related to high
volumes of data
• Real-world usage
readiness

14. SCALABILITY
TEST
 Scalability Testing is to determine the
software application's effectiveness
in "scaling up" to support an increase
in user load. It helps plan capacity
addition to your software system
• Plan for future
growth
• Scaling strategy

15. REALISTIC PERFORMANCE TESTING - IS IT POSSIBLE ?

16. SHIFT LEFT
AND
RIGHT
Shift left Shift Right
Test
Performance
of new code
Test
Performance
of every build
Test
Performance
of every
deployment
Test
Performance
in production

17. USER BEHAVIOR -
ANALYSIS
 Real User Monitoring

18. WORKLOAD
MODELLING
 The process of identifying one or more composite application
usage profiles for use in performance testing is known
as workload modeling.
 Adequate Coverage
 80 - 20 rule
New User Existing User Admin
Production
30% 60% 10%
Script 1 Script 2 Script 3
Test Environment
Application Under TestApplication in Production
30% 60% 10%

19. LITTLE’S LAW
 The long-term average number of customers in a
stable system N is equal to the long-term average
effective arrival rate, λ, multiplied by the average
time a customer spends in the system, W; or
expressed algebraically: N = λW.
 N = Throughput * (Response Time + Think Time)
Can be used in
Performance Testing
for:
• Workload Modelling

20. LITTLE’S LAW-
WORKLOAD
MODELLING
• Throughput
• Users
• Think time
N = Throughput * (Response Time + Think Time)
• User session lasts 555 seconds
• During session, the user views 8.78 pages
• Time between 2-page view = 555/8.78 = 63 seconds
• Response Time + Think Time = 63 seconds
• Assume throughput is 9.56, peak users 3904
N=9.56 *63 (assuming throughput is 9.56,peak users 3904)
N=602 users
602 users are enough to simulate peak load of 3904 users

21. TEST DATA
 Use realistic test data
 Enough data volume in database
 Unique users, different user roles
 Enough test data to avoid
caching problems
 Randomness in test data

22. TEST
ENVIRONMENT
SIMILAR TO
PRODUCTION
• Infrastructure similar
or comparable
• Any Other
Applications running

23. SIMULATE
DIFFERENT
NETWORK
SPEEDS - MOBILE
NETWORK
SPEEDS 
• Network Throttling
Growth of Mobile Usage

24. DISTRIBUTED LOAD
TESTING FROM
MULTIPLE
GEOGRAPHIC
LOCATIONS
 Virtual Users in
different Geographical
locations

25. THE CODE
FUNCTIONALLY STABLE CODE -
NO MAJOR FUNCTIONAL BUGS
PRESENT IN CODE
TEST WITH MOST RECENTVERSION
IF POSSIBLE
IDENTIFY ANY OTHER
PROCESSES/SYSTEM USING THE
ARCHITECTURE

26. EFFECTIVE
TEST
EXECUTION
 Tests conducted on an isolated network
segment
 Execute same tests twice to ensure
accuracy of results
 Ensure load generators are not themselves
bottleneck

27. PERFORMANCE
TESTING
RESULT
ANALYSIS - KPI
Label # Samples Average Median 90% Line 95%Min Max Error%
Throughp
ut Std.Dev
Login1 19381 177 124 147 163 4 20810 0.08% 87.3/sec 719.2848
Login2 19365 68 63 81 101 14 738 0.03% 87.5/sec 23.37372
Login3 19360 69 63 90 112 7 688 0.05% 87.6/sec 24.4878
TOTAL 58166 105 68 131 142 4 20810 0.05% 261.1/sec 418.8086
Login1
Login2
Login3

28. MATHEMATICAL
UNDERSTANDIN
G OF METRICS
 Average - This means arithmetic mean
 90 Percentile - To find the 90th percentile value for a data set
consisting of 100 page-response-time measurements, you
would sort the measurements from largest to smallest and
then count down eleven data points from the largest. 90
percent of the simulated users experienced a response time of
[the 11th-slowest value] or less for this test scenario
 Median - A median is simply the middle value in a data set
when sequenced from lowest to highest.
 Standard Deviation - One standard deviation is the amount of
variance within a set of measurements that encompasses
approximately the top 68 percent of all measurements in the
data set; in other words, knowing the standard deviation of
your data set tells you how densely the data points are
clustered around the mean.
Data with a
standard deviation
greater than half
of its mean should
be treated as
suspect.

29. PERFORMANCE
TESTING RESULT
ANALYSIS - KPI
• Resource
Utilizations

30. ANY
QUESTIONS?

31. THANKYOU