2. What is Performance Testing
• Performance testing – refers to test activities on checking system
performance
The major objectives of performance testing:
• To confirm and validate the specified system
performance requirements.
• To check the current product capacity to answer the questions
from customers and marketing people.
• To identify performance issues and performance degradation in
a given system
3. Differences b/w Performance, Load and Stress Test
• Performance Test determine the run time “behavior” of the
application and its supporting infrastructure, under certain
conditions.
• Performance testing is used to measure several system characteristics,
such as processing speed, response time, resource consumption,
throughput and efficiency.
• Load Test determine the applications “behavior under load”, up
to and including its limits (not just as its limits).
• Load tests specifically refers to the load size (number of concurrent
users) and related values.
• Stress Test determines the application ability to handle large
amount of data
• Stress testing can be much more successful with the full load applied to
the server.
4. Why Performance Testing?
• Scalability – Will the application handle the expected load and
beyond?
• Stability – Is the application stable under expected and unexpected
user loads?
• Availability – Is the application available to the end user without
any interruption?
• Serviceability – Can the system quickly recover from a Failure?
• Speed - Does the application respond quickly.
• Confidence – Are you sure that clients will have a positive
experience on go-live day?
5. Performance Requirements
• Following are some representative performance requirements
which system would be expected to meet
• The system should support peak load of x active users and y
transactions/sec. (for web based application it could be requests/sec)
• At peak load, system response to the X% of users should be Y Sec.
• System should be able to support 24 X 7 operation (reliability)
• System should be scalable to meet growth in demand.
• The response time requirement could vary based on
• Geographical location of users
• Connection bandwidth (especially true for Internet users)
• Transaction complexity.
6. Performance parameters to consider
• Throughput: The number of requests processed per unit time (per
second)
• Latency: The time taken in establishing the network handshake.
• Efficiency: Throughput - Latency
• Degradation: The Throughput of the application when the concurrent
requests are gradually increased – Test to see if the performance of
the application degrades when the number of concurrent requests is
increased.
• Longevity : Execute tests for a pre-determined time interval with a
pre-determined load. Monitor the health of the application throughout
the test execution and check for memory leaks and analyze the GC
dump
7. What Performance Problem may arise ?
• Memory-related problems:
• Application uses more memory than it should.
• Memory leaks.
• Excessive garbage allocation, i.e. application creates a lot of temporary
objects.
• Code-related problems:
• Application algorithms are not optimal, and there are performance
bottlenecks.
• Hardware/Software related Problem:
• Hardware you select for your database is not in vendor Hardware
Compatibility List
8. Scalability
• This section talks about the impact of scaling up the Hardware
resources from the suggested minimum Hardware requirements
• Processor impact
• Database impact
• I/O impact
• Network bandwidth Impact
9. Processor Impact
• Single Processor v/s Multi Processor
The response time for searching or archiving a document
decreased by 50 - 80% on adding an additional processor in a multi-
threaded scenario (info based on assumptions)
10. Data base Impact
• Single Processor v/s Multi Processor
• As the number of concurrent users using application services are
increased, it is observed that Oracle performs better than MS SQL
Server with a 5 - 8% improvement in the performance (info based on
assumptions)
11. I/O Impact
• I/O has considerable impact on the performance of the Application
with 15% to 18% reduction in the response time especially when
there are huge reports to be generated on the system.
12. Network bandwidth Impact
• A good Network connectivity between the Client, Application Server
and database improves the performance considerably. There was
40% (approx) improvement in the throughput of the application
when the Network Connectivity between Database system and
Application Server system was increased from 100 mbps to 1Gbps
(info based on assumptions).
13. Identify the Performance issues
Application
Database
Application Code
Application Design.
JVM settings
Application server configuration and usage
Web Server Configuration and usage
System OS
System OS
Client Side
Database Schema, design, configurations,
resource usage, SQLs, Indexes etc
OS Resources – CPU, Network, Disk, Memory,
OS kernel, Storage etc
Browser settings, Client system, Page size,
amount of data displayed
15. Database Server Causes
Other
5%
Slow specific query response
22%
Slow overall query response
16%
Slow overall DB response
13%
Overuse of row-at-a-time
processing
11%
Missing indexes
8%
Poor multi-user response
7%
Query plan cache too small
7%
Data cache too small
5%
Deadlocks
3%
Old optimizer statistics
3%
16. App Server Causes
Memory leak
15%
Inefficent garbage collection
12%
Expired sessions still active
12%
Poorly configured App Server
12%
Insufficient hardware resources
10%
Poorly configured DB
connection pool
9%
Inefficiently coded transaction
11%
Inefficient DB access
architecture
4%
Inefficient object access
method
5%
Other
10%
17. Network Causes
Bad network architecture
20%
Other
20%
Poorly configured/insufficient
NICs
10%
Security too tight
15%
Insufficent overall bandwidth
13%
Load balancing ineffective
22%
18. Software and Hardware Requirements
• Hardware Requirement
• Details of system to be tested
• Workstation required for load simulation
• Workstation required for running monitoring utilities
• Network Equipment (e.g. Router, Firewall, Modems etc.)
• Software Requirement
• System Software
• Load Testing Tools
• Performance Monitoring utilities
• Miscellaneous utilities like graph generation etc.
19. Performance Testing Process
• Understand system and identify performance requirements
• Identify performance test objectives.
• Define performance test strategy:
• Identify the needs of performance test tools and define
performance test environment
• Write performance test plan
• Set up the target system and performance test beds
• Design performance test cases and test suite
• Performance test execution and data collection
• Performance analysis and reporting
20. Performance Testing Approach
• Performance testing: (during production)
• Measure and analyze the system performance based on
performance test data and results
• Performance simulation: (pre-production)
• Study and estimate system performance using a simulation
approach
• Performance measurement at the customer site: (post-production)
• Measure and evaluate system performance during system
operations
21. Performance Modeling Project Overview
In order to analyze performance behavior, improve product
performance, and serve as the foundation for capacity planning
• The overall goal was to design and develop a model that would be:
• realistic and general enough to be widely applicable across a variety of
customer configuration and application scenarios in Altair target market.
• simple enough to be easily usable by the field sales and support
organizations to assist customers make system configuration decisions
• possible to validate so that it could be applied with confidence.
22. What is the resultant deliverable
The resultant of this performance testing exercise is a “Performance
and Scalability document” with results of the tests performed and
hardware recommendations. The document would not be a sizing
guide per se but will have:
• Throughput and scalability numbers for the end users to consume
• Test metrics - numbers and graphs related to the tests executed
• And an internally consumable document containing performance
recommendations for the developers to work on.
Also, note that the resultant document is not a Sizing guide.
But this document will definitely be a good input to the performance
team to come up with a sizing guide for the future releases.
