3. DownloadLatestVersionofthisFile!
This file was downloaded from:
chromeis.com/loadrunner
Download latest version of the slides from above URL.
Feel free to distribute the file and share knowledge
The file is available as Read-Only and does not require
password to open.
View in Presentation Mode – the slides animation is
designed purposely to nurture audience understanding
Direct your feedback to above URL or to author at
kamran.khan@chromeis.com
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 3
4. CourseObjectives
Discuss the value of load testing
Plan for effective load test
Establish load test goals
Run load test scenarios
Load and overload when executing
scenarios
Analyze and interpret load test results
Prerequisites:
Understanding of:
Client/Server,Windows,Web
Course Methodology
Slides, Experience sharing
Quizzes, Assignments
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 4
5. Trainer’sProfile
9+ years of IT experience as lead engineer, specializing in SQA,
SoftwareTesting andTest Automation
Vast experience of working with LoadRunner, winRunner, QTP,
TestComplete ,WatchFire and various other performance tools
Proficient with Load testing of SOA based, Net Centric web-
based products, web and RIA applications
Vast experience of conducting technical trainings – both
internally and externally.
BSCS Hons., MBA (Marketing)
Certified:CCNA, AIS HP Performance Center, LoadRunner, QTP
Expertise Areas:
QMS,TQM, AutomatedTesting usingWinRunner & QTP,TestComplete
PerformanceTesting using HP Performance Center, LoadRunner, IBM
Rational PerformanceTester, JMeter and others. PDA apps testing
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 5
8. Facts
Fact #1 :
Most users click away after 8 second delay
Fact #2 :
$4.4 billion in revenue lost annually – due to poor web performance
Source: Zona Research …continuedHP LoadRunner - Kamran Khan – ChromeIS.com/institute 8
9. NewtoLoadRunner
Reconfirm your core skills.The core skills which are needed for performance testing are a large superset of
those needed for automated functional testing.A search of this forum [LR-LoadRunner at Google] and others
will result in lists of what those core skills are.
After you have confirmed those core skills, attend standard product training. Don't scrimp.Take the classes
from an HP CertifiedTrainer using HP provided material. Some will argue whether this training is the best
possible training and they are probably right on that argument, but this training has the most consistent
delivery with the most consistently skilled at the end of the course.
Post training, work with a mentor. Becoming a performance tester is not unlike becoming a plumber, an
electrician, a barber/cosmetologist.There are core skills, a period of tool instruction and then a journeyman
phase where you are working under the supervision of a master in the profession who can help you bring
together skill, technique and process to mature you as a performance tester.This period will likely last three-
five performance testing projects. If you cannot stand on your own after five then you should probably head
to another profession:This is not a profession for everyone.
After mastering the core, begin giving back. Become a mentor to the next generation.
If your management will not commit to affirming your core skills (and touching them up where necessary),
training and mentoring then you should look for a new organization/set of management. It is a well
established pattern that such managers will throw you under the bus to salvage a customer relationship
where a performance testing gig has gone south. It is not a question of "if," only "when" when you have such
management in place. Go in with eyes wide open.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 9
10. Reground yourself in application architecture, particularly client-side architectures and communications methods.
Expect that with training and a mentor this will be an 18 month to 2 year path for you to become self sufficient.
There are thousands on the market with LoadRunner on their resume without training and mentoring that even after
five years of "work" have almost zero usable skills – they have been able to skate by in environments where they know
marginally more than the manager/client who is not capable of evaluating their work.
The testing world, by attempting to equate the skill sets of automated testers and performance testers has pushed
many thousands of automated testers into performance testing roles for which they are ill-equipped.They lack the
foundation process and mechanical skills to be successful. They lack the analytical skills to actually find the issues.
Heck, you would be hard pressed to find many who can give you a straight answer on what a performance requirement
actually looks like and how to evaluate whether the requirement is sound or not.All of this has led to an odd economic
situation: For the past ten years the demand for performance testing skills has been rising, but the average
compensation has been dropping. In a normal market with a skills scarcity rates would rise and bring more skilled
people into the market. But in this case the flood of low skilled low value individuals has actually caused value to drop
at a rate faster than demand is rising, leading to the odd economic situation of falling rates in a scarce resource high
demand market.
This is why the industry is "more fishy" due to this behavior…
James Pulley, LR-LoadRunner, 1st October, 2011`
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 10
12. AutomatedPerformance Testing
Performance testing is the discipline concerned with determining and reporting the
current performance of a software application under various parameters
Automated performance testing is about applying production workloads to pre-
deployment systems while simultaneously measuring system performance and end-
user experience.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 12
13. PerformanceEngineering
Performance testing is the discipline concerned with determining and reporting the
current performance of a software application under various parameters Performance
engineering is the process by which software is tested and tuned with the intent of
realizing the required performance.This process aims to optimize the most
important application performance trait, user experience.
Historically, testing and tuning have been distinctly separate and often competing
realms. In the last few years, however, several pockets of testers and developers
have collaborated independently to create tuning teams. Because these teams
have met with significant success, the concept of coupling performance testing
with performance tuning has caught on, and now we call it performance
engineering.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 13
14. WhyLoadRunner?
Benefits of choosing LoadRunner
LR Credibility - Industry Standard (77% market share)
Meeting the criteria
Scalable [supports add-in]
Integration with QTP
Integration with WR
Scenario modification
Others:
Record n Play, Custom Code, Errors and Exceptions
Envision - Direct Revenue in long term
Off shore services
Outsourced services
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 14
15. LimitationsfosteringLTteam
Current Limitations
Less resources available in market
Trade off between Workload /Training others
ZERO Support from vendor
99% unplanned (ad hoc) activities*
Stakeholders have incomplete, obscured or no knowledge
regarding Automation / LoadTesting
Deficient in Real Environment – scaling requires heuristics
Tool cons
Technology dependent
Initial scripting would be decisive
Different License for Desktop,Web, RIA and database protocols
Not all licenses can be installed at a time
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 15
16. AutomatedTesting:Objectives
Drawbacks of manual testing
Benefits of AutomatedTesting
in Functional RegressionTesting
in LoadTesting
WhichTest Case not to automate
in Functional RegressionTesting
in LoadTesting
ManualVs AutomatedTesting process
AutomatedTest Planning
CommonTesting Objectives
Improve current framework by spreading awareness of:
- technological constraints
- due responsibilities of other stakeholders
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 16
17. PerformanceTestingtypes
What is PerformanceTesting
Stress testing
Load testing
Volume testing
Soak / Endurance testing
…
When:
When to test under load
Why:
Why to automate
Why not to automate
What:
What to automate
What not to automate
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 17
19. AutomatedPerformanceTesting
automated performance testing is about applying production workloads to
pre-deployment systems while simultaneously measuring system
performance and end-user experience !
Load: Number of concurrent users
Web Site
Response
Time
Good
(0 - 8 sec.)
Borderline
(9 – 15 sec.)
Unacceptable
(15 – 30 sec.)
Virtually
Offline
(> 30 sec.)
The 8 Second Barrier
Crash!
Source: Presentation by LloydTaylor, VPTechnology &Operations, KeynoteSystems
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 19
20. Why loadtestyourapplication?
Failure of Mission Critical product can be costly.
Deploy with confidence:
assure business transactions work real-world
conditions
Locate and resolve potential problems before
customer identifies them
Scalability Testing buys us confirmation of
predictions about what will happen before it
actually happens, buying the lead time to do the right
thing when additional capacity is needed.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 20
21. Why loadtestyourapplication?
Performance is the key to success of any web based application in specific
and all others in general (mobile/desktop)
We will get to know the maximum capacity the system can handle for an
application.
We can decide whether we should go for Hardware upgrades or Performance
tuning.
Understand the issues involved in load testingWeb applications.
Learn your responsibilities as the LoadRunner expert.
Learn how to perform successful load tests using LoadRunner.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 21
22. Loadtesttypicalobjectives
Many concurrent users running the same application to see whether a
system handles the load without compromising functionality or
performance.
How many concurrent users can the system handle without increase in the
expected response time?
Can the system serve [x] concurrent users without any errors?
System works fine with X connections for Z number of hours.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 22
24. Whytousetestingtools
Time Consuming
Inconsistent
Low Reliability
Expensive
Human Resource
Machines
Complicated
Synchronize testers
Analyze results
meaningfully
Speed
Repeatability / Consistent
Coverage
Programming capabilities
Reliability
Reusability
Control all users—from a
single point of control
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 24
25. Whenautomationisapplicable
When a system combines a variety of software applications and hardware platforms to
determine the suitability of a server for any given application
To test the server before the necessary client software has been developed
Emulate an environment where multiple clients interact with a single server application
test an application under the load of tens, hundreds, or even thousands of potential
users
Test Repeatability is required
Automated Functional RegressionTesting MercuryWinRunner / HPQTP etc.
Automated Stress/Performance/SoakTesting HP LoadRunner / IBM Rational PerformanceTesterv
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 25
26. WhichTestCasetoautomate?
Tests that need to be run for every build of the web site (sanity level)
FRT
Tests that use multiple data values for the same actions (data driven tests)
FRT
Identical tests that need to be executed using different browsers FRT
Mission-critical pages FRT / LT
Transaction with pages which won't change in short term FRT
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 26
27. WhichTestCasenottoautomate?
Usability testing
–"How easy is the web site to use?"
"ASAP" testing
–"We need to test NOW!"
Ad hoc/random testing
–based on intuition and knowledge of web site
FunctionalTesting
Business validation
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 27
28. BetterCandidate
Business Processes which:
Involves large number of database tables
Single database table with large entry
Most frequently opened database tables
Requires big data to reside in RAM
Involves high processing like financial calculations
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 28
29. HowLoadRunnerdoesitall?
LoadRunner monitors the application performance online, enabling you to
fine-tune your system during test execution.
LoadRunner automatically records the performance of the application
during a test.You can choose from a wide variety of graphs and reports to
view the performance data.
LoadRunner checks where performance delays occur:
network or client delays
CPU performance
I/O delays
database locking
network and server resources
or other issues at the database server.
Because LoadRunner tests are fully automated, you can easily repeat them
as often as you need.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 29
30. PerformanceTesting:ATeamWork
Project Manager:
- Manages the implementation of website/SUL
Functional Expert / Business Analyst:
- Provide Usage Analysis of SUL & provides expertise on business functionality of
website/SUL
AutomatedTesting Expert:
- Creates the automated tests & executes load scenarios
System Architect:
- Provides Blueprint of SUL
Web Developer / SME:
- Maintains website & provide monitoring aspects
- Develops website and fixes bugs
System Administrator:
- Maintains involved servers throughout testing project
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 30
32. Planningaloadtest
What to Plan?
Understand the functional testing objective(s)
of the AUT
Know the user actions of each business
transaction of the AUT
Know what input data to use for each transaction
Know what to verify in the web site
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 32
34. CommonTestingObjectives
General page layout
Frames
Images
Tables
FunctionalTesting
of each transaction using different sets of valid data across
different browsers
BusinessTesting
core business or objective of any application being developed
RegressionTesting
hardware and software upgrades
web site enhancements
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 34
36. AutomatedTest:Objectives
Purpose of understanding test objectives:
Create effective automated tests that meet test
requirements
Gather the relevant information that will help you create
the automated test
System Architecture
System Components
Users profile (application users)
Clients behavior with application (TDD)
3rd party components, integration and other technical details
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 36
38. TestCasesforPerformanceTestingcont.
Certain information is “a must” when preparing for LoadTesting.
This includes:
Testing environment
Operating System (Windows XP Server 2002 SP2)
Hardware Specifications (Intel® Pentium 4 CPU 3.00 GHz, 2 GB RAM)
SQL Server 2005, IIS 6.0 etc
Databases with user data (preferably client DB replica)
Representatives from Dev,Testing, AT, Networks
Obtain master data – extract from database
obtain help from domain experts
Create valid transactional data
predict according to type of data
obtain from Functional Expert
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 38
39. VitalingredientsinOrganizing aTest
•The architecture of the system
•A brief idea about the application
•The list of business transactions
•The time at which the system has peak load
•The acceptable response time for the transactions
•No. of concurrent users the system is expected to handle during peak load
• Select theTransactions to automate
• Decide on theTransaction Mix
• Obtain User details for the transactions
•The tables these transactions query
•The web server / application server components invoked by the
transactions
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 39
40. VitalingredientsinOrganizing aTest
Select the transactions
• Give priority to business critical transactions
• Give priority to transactions that have high database activity
• Give priority to transactions that have high web server or application server activity
• Know the peak load time
Decide onTransaction Mix
• Impact of the failure of a particular transaction on business
• The transactions that are executed during peak load
• The no. of times they are executed during peak load
User details
• For each of the chosen transactions find
• Who are the users who executing these transaction?
• How many simultaneous and concurrent users are there?
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 40
41. VitalingredientsinOrganizingaTest
For each of the chosen transactions find:
• The tables which a transaction queries
• The tables which a transaction updates
Server components invoked:
• The servlet / jsp that are invoked by the transaction.
Environment specifications:
• Application architecture
• HW specifications of machines where application is hosted
• SW specifications of machines where application is hosted
• HW/SW specifications of machines where load will be simulated
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 41
42. Summary
• Understanding of various stages of PerformanceTest automation
• Planning should be done before automation begins
• Proper planning can increase your test coverage and reduce test
development time
• Understanding the testing objectives helps devise effective automated
tests
• Identifying better candidates for performance testing
• Document all relevant test information and team representatives
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 42
47. What'sNext
Understand the architecture of LoadRunner itself
To describe best practices of carrying efficient performance tests
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 47
53. GoalsofPerformanceTesting
Conceptual Goals
What are we trying to achieve?
SUL Mission CriticalTransaction ResponseTime
Is Booking Flight transaction fast enough?
xyz business process supports 250 concurrent users
SUL Environment Benchmarking
Are servers, network, software, etc. compatible?
100 concurrent users can file contracts
for 1 hour if web server RAM is 1GB
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 53
54. BusinessProcesses
Mission Critical
What BP
malfunctioning will put
you out of business?
Throughput
How many users going through identified MissionCritical BPs?
Dynamic Contents
How much data each transaction process through the SUL
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 54
55. SystemUsers
SUL User Profile
What are the SUL users: level of experience with the System
SUL Users Regions
Where all SUL users located:
America, Pakistan, KSA?
SUL Users Concurrency
What is a pick of SUL usage per BP at given hour?
User Roles (Access rights on system)
What types of user roles are in practice. Manager,TL, Dealer,
Auditor etc.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 55
56. DocumentingBPs
User Action Steps
Should BP start with User Login?
Can we navigate through the BP manually using these steps?
SUL Navigation Expected Results
What should we include in the Checkpoint once this step performed?
BPTransactions
What trigger in the BP will be measured for response time?
ThinkTime
How long user waits till the next step in the BP?
Parameters
What static input data should be replaced with dynamic content?
What type of parameter should it be? Data file, date, random?
What parameter sequence should be set?
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 56
57. DataManagement
Test Data Needed
What data will be needed
(User Names & Passwords, SSN #,
SKU # etc.)?
Procedures to Generate Data
How are we going to generate data
(querying database, data processing department,
etc.)?
Generated Data format
What format data will be coming in (Flat file, CSV,
etc.)?
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 57
58. PerformanceMonitors
Investigate SUL Environment
Set Monitor per each SULComponent
Add measure point gradually
Set default measurements
Add measurements as needed
Consult with components SME
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 58
59. Whatconstitutesasystem!
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 59
Database
Web Servers /
Application Servers
Operating Systems
External Storage
Front End GUI
Clusters/
Load
Balancers
Memory
Threads
Kernal
Heap
Third Party
Components
62. ATProcess:Planning
Step I: Planning theTest
Successful load testing requires that you plan to carry out certain business
process.A BP consists of clearly defined steps in compliance desired
business transactions – so as to accomplish your load testing objectives.
BusinessTransaction:
Examples:
Reserving flight tickets
Buying a computer from a online store
Logging into your web-based email service
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 62
A business transaction is a set of user actions performed with an application to
accomplish a business task.
63. ATProcess:Scripting
Step II: Creating theVuser Scripts
VUsers emulate human users interacting with yourWeb-based application.
AVUser script contains the actions that eachVUser performs during
scenario execution.
In eachVUser script, you determine the tasks that will be:
performed by eachVuser
performed simultaneously by multipleVusers
measured as transactions
For more information on creatingVUser scripts, see the HPVirtual User
Generator User Guide.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 63
64. ATProcess:CreateScenario
Step III: Creating the Scenario
A scenario describes the events that occur during a testing session. A scenario includes a
list of machines on whichVusers run, a list of scripts that
theVusers run, and a specified number ofVusers orVuser groups that run during the
scenario.You create scenarios using the Controller.
Creating a Manual Scenario
You create a scenario by definingVuser groups to which you assign a
quantity of individualVusers,Vuser scripts, and load generators to run the scripts.
You can also create a scenario using the Percentage Mode, in which you
define the total number ofVusers to be used in the scenario, and the load
generators and percentage of the total number ofVusers to be assigned to eachVuser
script.
Creating a Goal-Oriented Scenario
ForWeb tests, you can create a goal-oriented scenario, in which you define
the goals you want your test to achieve. LoadRunner automatically builds a
scenario for you, based on these goals.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 64
65. ATProcess:RunScenario
Step IV: Running the Scenario
You emulate user load on the server by instructing multipleVusers to
perform tasks simultaneously.You can set the level of load by increasing
and decreasing the number ofVusers that perform tasks at the same time.
Before you run a scenario, you set the scenario configuration and
scheduling.This determines how all the load generators andVusers
behave when running a scenario.
You can run the entire scenario, groups ofVusers (Vuser groups), or
individualVusers.While a scenario runs, LoadRunner measures and
records the transactions that you defined in eachVuser script.You can also
monitor your system’s performance online.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 65
66. ATProcess:MonitorScenario
StepV: Monitoring a Scenario
You can monitor scenario execution using the LoadRunner online run-time,
transaction, system resource,Web resource,Web server resource,Web
application server resource, application server resources, database server
resource, network delay, streaming media resource, firewall server
resource, ERP/CRM server resource, Java performance, J2EE & .NET
Diagnostics, application deployment, middleware performance,
application component, and infrastructure resources monitors.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 66
67. ATProcess:Analysis
StepVI: AnalyzingTest Results
During scenario execution, LoadRunner records the performance of the
application under different loads.You use LoadRunner’s graphs and
reports to analyze the application’s performance.
For more information about LoadRunner’s reports and graphs, see the HP
LoadRunner Analysis UserGuide.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 67
73. VirtualUserTechnology
Simulates a real-like virtual-user
Programming capabilities with greater control over test execution
Can synchronize actions performed by users
VUsers can communicate directly with a server by executing calls to the
server API-without relying on client software
Captures a business transaction by recording user actions performed
Monitors the communication between the application and the server
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 73
74. Beforewestart…
Scenario:
A scenario defines the events that occur during each testing session.Thus,
for example, a scenario defines and controls the number of users to
emulate, the actions that they perform, and the machines on which they
run their emulations.
VUsers:
In the scenario, LoadRunner replaces human users with virtual users or
Vusers.When you run a scenario,Vusers emulate the actions of human
users working with your application.While a workstation accommodates
only a single human user, manyVusers can run concurrently on a single
workstation. In fact, a scenario can contain tens, hundreds, or even
thousands ofVusers.
VUser Scripts:
The actions that aVuser performs during the scenario are described in a
Vuser script.When you run a scenario, eachVuser executes aVuser script.
TheVuser scripts include functions that measure and record the
performance of your application’s components.
Others:
System, ResponseTime,ThinkTime,Work Load,Transaction Mix,
Throughput
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 74
75. Beforewestart…
Transactions
A transaction represents an action or a set of actions that you are interested in measuring.You
define transactions within yourVuser script by enclosing the appropriate sections of the script
with start and end transaction statements. For example, you can define a transaction that
measures the time it takes for the server to process a request to view the balance of an account
and for the information to be displayed at theATM.
Rendezvous points:
You insert rendezvous points intoVuser scripts to emulate heavy user load on the server.
Rendezvous points instructVusers to wait during test execution for multipleVusers to arrive at a
certain point, so that they may simultaneously perform a task. For example, to emulate peak
load on the bank server, you can insert a rendezvous point instructing 100Vusers to deposit cash
into their accounts at the same time.
Controller:
You use the HP LoadRunnerController to manage and maintain your scenarios. Using the
Controller, you control all theVusers in a scenario from a single workstation.
Load Generator:
When you execute a scenario, the Controller distributes eachVuser in the scenario to a load
generator.The load generator is the machine that executes theVuser script, enabling theVuser
to emulate the actions of a human user.
Performance analysis:
VUser scripts include functions that measure and record system performance during load-testing
sessions. During a scenario run, you can monitor the network and server resources. Following a
scenario run, you can view performance analysis data in reports and graphs.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 75
77. VUGen:ScriptSections
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 77
Script Section Used when recording... Is executed when...
vuser_init a login to a server theVuser is initialized (loaded)
Actions client activity theVuser is in "Running" status
vuser_end a logoff procedure theVuser finishes or is stopped
Script View
Script Sections
Tree View
78. RecordingOptions…
Tools Recording Options
[Ctrl + F7]
…
General Recording
• HTML Mode
• URL Mode
This allows us to specify the information to be recorded and which
functions to be used when generating a script
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 78
79. HTMLMode–URLModecomparison
The HTML mode is based on user
actions, and the scripts contain
functions that correspond directly to
the action taken
Example:
...
web_link(“Enterprise Systems
Performance",
"Text=Enterprise Systems
Performance ",
"Snapshot=t4.inf",
LAST);
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 79
The URL mode is based on HTTP requests
sent to the server as a result of user
actions.
Example:
...
web_url(“Enterprise Systems
Performance",
"URL=http://www.tcs.com/esp.html",
"TargetFrame=",
"Resource=0",
"RecContentType=text/html",
"Referer=http://www.tcs.com/atc?. . . ,
"Snapshot=t4.inf",
"Mode=URL",
LAST);
80. DecidingonRecordingMode
Deciding on Modes
For browser applications, use HTML mode.
For non-browser applications, use URL mode.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 80
The option of mixing recording modes is available for very advanced users for
performance tuning
HTML Mode URL Mode
Intuitive and easy to understand Not as intuitive as the HTML scripts
Scalable More scalable and effective for creating a load test
82. DirectoryStructureofanLRScript
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 82
VuGen creates a series of config.
Files, data files and source code files
which containVUser run-time and
setup information.
The results of each iteration of a
script are stored separately.
The Iteration0 directory is for
system use and should be ignored
by the tester.
83. ResponseTime
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 83
App
Server
DB Server
Load
Injector
Time in which the system responds for a particular
Transaction request
84. ThinkTime
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 84
Time taken for selecting a new Transaction after the response for
the previous Transaction has been received
Transaction 1
Transaction 2
ThinkTime = 10 Seconds
85. Workload
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 85
Database
Web Servers /
Application Servers
Operating
Systems
External
Storage
Front End
GUI
The System
Typical Workload
No. of Users
Transactions
Batch Programs
Background Load
86. TransactionMix
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 86
Varying execution frequency of different Transactions
Credit
Loans
Debit
Query Account Info
Account Opening
87. HPLoadRunnerVUserTechnology
On each Windows load generator, you install the Remote Agent Dispatcher (Process) and a LoadRunner Agent
Remote Agent Dispatcher:
The Remote Agent Dispatcher (Process) enables the Controller to start: applications
on the load generator.
Agent:
The LoadRunner Agent enables the Controller and the load generator to
communicate with each other.When you run a scenario, the Controller instructs the
Remote Agent Dispatcher (Process) to launch the LoadRunner agent.The agent
receives instructions from the Controller to initialize, run, pause, and stop Vusers. At
the same time, the agent also relays data on the status of the Vusers back to the
Controller.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 87
88. VUGen-Introduction
Main components ofVuGen interface
StartingVuGen
Environment options (Protocols)
Script view
Tree view
Output window etc
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 88
89. Stepstowardsabaselinescript
Commencement
Selecting right combination of protocols
Protocol Advisor (9.5 or newer)
Record
Conventions and good practices
Prepare andAssure availability ofTest data
Understanding script
Manual verification
Playback andVerification
Through subject application
Parameterize/Correlation
Enhance
Rendezvous,Transactions, Output message,
Image/Text check, Correlation, and additional
steps, Configuring Pacing, Iterations, Actions,
Log/Error messages &Think time and more
Runtime Settings
Pacing, Iterations,Think time and more
Prepare for Execution
Prepare andAssure availability ofTest data
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 89
90. RecordingfortheWeb
Example Exercise
Create vuser script
Understand vuser script
Generation log
Script regeneration
User authentication information
web_set_user("domainuser, “password", “host:port");
web_set_user(“chrometechkhankamran", "password123(",
“machine112:80");
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 90
91. Replayingarecordedscript
Verify the baseline script
Identify and configure appropriate runtime settings for replay
Replay vuser script
Replay log
Verify the application behavior
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 91
93. Transactions
A mean to measure time taken by application
or its server side component in entertaining
certain client request.
Or, the time to complete specific takes or activity
Defining transactions
Preferably during recording else after recording
Syntax:
lr_start_transaction(“Transaction Name”);
lr_end_transaction(“Transaction Name”, <status>);
e.g. lr_start_transaction(“Transaction Name”);
lr_end_transaction(“Transaction Name”, LR_AUTO);
lr_end_transaction(“Transaction Name”, LR_PASS);
lr_end_transaction(“Transaction Name”, LR_FAIL);
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 93
94. Transactions cont.
Best Practice:
Assign Script ID as prefix to the name
Assign transaction number right before logical name
e.g. 0102Login
Removal any immediate following think time statement
Remove or document think time statements within a
transaction
Period (.) character is not allowed in transaction name
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 94
95. SubTransactions
/* Notify that a transaction is starting */
lr_start_transaction("purchases");
/* Breakdown the transaction into a smaller transaction */
lr_start_sub_transaction("electrical_purchases", "purchases");
status = purchase_electrical_items(6); /* call to Server API */
/* End transaction with operation result - pass or fail */
if (status == 0)
lr_end_sub_transaction("electrical_purchases", LR_PASS);
else
lr_end_sub_transaction("electrical_purchases", LR_FAIL);
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 95
96. Comments
Add comments for
To describe an activity
To provide information about specific operation
To separate two sections for distinction
Add comments:
Comments during recording
Comments after recording
Best Practice:
Mark any comments on the top of each script file
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 96
97. RendezvousPoints
Rendezvous Points
Introduce concurrency
Defining transactions
during recording or after recording
Syntax: lr_rendesvous(“Logical Name”);
Best Practices:
Prefix a rendezvous point with “rdv_”
Use logical name of transaction being subjected
e.g. rdv_Login
Removal any immediate following think time statement
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 97
98. InsertingFunctionthroughmenu
Log messages
Debug messages
Error messages
Output messages
Choose Insert > New Step.The Add Step dialog box opens. Scroll down and
select Output Message and clickOK.
lr_output_message
lr_log_message
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 98
99. Parameterization
A parameter is a placeholder which re-places a recorded value in aVuser script. At run
time, a value from an external source is substituted for the parameter against record
time value.
Parameterization solves certain problems that may occur during playback:
• Date constraints make recorded date invalid
• Unique constraints prevent reusing recorded data
• Data caching
• Response times are inaccurate
• Real user activity is not emulated
• Servers are not exercised
Problem examples:
Business process works only for the current date
Unique ID required as in case of Primary Key or fields in database with Unique constraint
How to measure the performance of the process which creates a new product?
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 99
100. Parameterization
What is Parameterization?
Which value to parameterize?
How to parameterize?
Properties of a Parameter?
Caution – line feed at the end of each
line in data file is mandated
space i.e. ‘%20’ is ignored by LR
Types of parameters
Date/Time
File
Group name
Random number
Iteration number
Unique number
XML
Vuser ID
Load generator name
User defined function
lr_output_message ( "We are on iteration #%s", lr_eval_string ( "{iteration}" ) );
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 100
101. Parameterconfiguration
Important options regarding properties of File /Table
parameters
Data assignment method
Update method
When out of values
Columns
Rows
When not enough rows
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 101
102. Whyexcessiveparameterization!
Problem: Since allVUsers use the same data, Data caching at servers makes
load test response times shorter than when real users load the system
Solution: Parameterize input data to force the system to get data from a
database
Data gets relocated in the server’s cache when it is reused for eachVuser
execution. As a result, server activity is not emulated during playback.While
Vuser execution is successful, it will not produce test results meaningful for
load testing because the server is not exercised. Using different data better
emulates server activity and exercises the system.That improves the
chances for meaningful load test results.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 102
103. CreatingDataPool
How to create data pool to be used in parameterization files?
1. Extract master data from database
1. Write queries on tables being accessed by the business
transactions.
2. Spool data to a file
3. Write AWK scripts to generate a proper data pool from the
spooled file
2. Create valid transactional data
1. Predict according to type of data
2. Obtain from a functional expert
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 103
104. Parameterization–Process
Determine which fields to parameterize
Replace recorded values with parameters
Decide which parameter type to use
Choose the data access method and the number of iterations
Run theVuser and analyze the results to verify correct
execution
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 104
105. Parameterization–Process cont.
Right click on the data to parameterize
select “Replace with a Parameter”
The Select or Create Parameter dialog box opens.
Type a suitable Parameter Name and Select a ParameterType
Right-click the parameter name and select “Replace More
Occurrences” to replace the data by the parameter name
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 105
106. Parameters–Exercise
Data table values:
Nida; Kamran; Michael; Jane; Ron; Alice; Ken; Julie; Fred
12 vusers and 3 iterations
Try all combinations of assignment method and update
method on provided data
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 106
107. AdvancedScriptingTechniques
Other protocols’ specific functions
Correlation
LR Standard Functions
variable_name = lr_eval_string(“{variable_name}”);
lr_save_string(“value”, “variable_name”);
Protocol Specific Functions
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 107
108. Correlation
Many applications use dynamic values that change each time you
use the application. For example, some servers assign a unique
session ID for every new session. When you try to replay a recorded
session, the application creates a new session ID that differs from
the recorded session ID.
To simplify / optimize code
To generate dynamic data
To accommodate unique data records
Protocol Specific Functions:
variable_name = lr_eval_string(“{variable_name}”);
lr_save_string(“value”, “variable_name”);
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 108
109. Correlation
In applications output of the previous transaction may be used as an input
for the next transaction.
For example:
System generated session ids
URLs that change each time you access theWeb page
Fields (sometimes hidden) recorded during a form submission
The data is only good for the current session
Solution using Correlation:
Capture output value from one preceding step
Use captured value as input to proceeding step
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 109
110. Dynamic data is correlated automatically by the client application (running in
the browser) while recording a script
Correlated data is data which is sent to the client from the server, and later
sent back to the server by the client.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 110
Network
Web Server
Client
Step 2 - Request
[Session ID =
22222]
Step 1
[Session ID =
22222]
111. Reusing recorded dynamic data will cause errors when you replay the script
Recorded dynamic data must be correlated using parameters
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 111
Network
Web Server
Client
Step 2 - Request
[Session ID =
22222]
Step 1
[Session ID =
23212]
112. Correlationtypes
Auto correlation
LR detects and help solving them
Manual correlation
LR detect but doesn’t provide direct interface to
correlate
LR may not detect code requiring correlation
Question: when manual correlation is required?
Question: when auto correlation is required?
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 112
113. ManualCorrelation
1. Determine the value to capture
2. Find the right and left text boundaries of the value to capture (WEB)
3. Find which occurrence of the text boundaries should be used
4. Add a web_reg_save_param function to the script, above the step which
requests the page with the value to capture
5. Add the parameter name, left boundary, right boundary, and occurrence
to the function
6. Parameterize the dynamic value in the script every time it occurs
7. Verify correct execution
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 113
114. AutomaticCorrelation
There are three types of automatic correlation solutions
1. Auto-Detect Correlation
2. Rule-Based Correlation
3. CorrelatingAll Statements
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 114
Rule Name When to Use
Auto-detect
Correlation
Detect and correlate dynamic data for supported
application servers
Rule-Based When working with a non-supported application server
whose context is known.
Correlate All Blindly correlate all dynamic data.
115. AutomaticCorrelation
Auto-Detect Correlation
Automatically correlate the
recorded statements
CorrelatingAll recognizable
statements
Rule-BasedCorrelation
Rule-based correlation requires
correlation rules to be defined
before recording a session
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 115
116. Run-TimeSettings
Run Logic
Iterations
Action blocks
Pacing
Log
ThinkTimes
No think time
As Recorded
Multiple of X-Recorded
Randomize think time
LimitThink time to Z sec.
Speed Simulation
Bandwidth controlling
Browser Emulation
Proxy
No proxy
Specific proxy
Pick proxy from browser
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 116
117. Run-TimeSettings
Control the behavior of the user when running the script
ConfigureWebVusers to emulate real users accessing theWeb site
Browser Emulation
Specify Browser type
Simulate cache
Emulate network capacity
Emulate modem speed
Log Settings
Can disable logging
Log a specific set of function calls and messages – Standard Log
Log parameter substitutions
Log response data from server
All the messages and function calls raised
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 117
120. NextGenerationofApplications–Web2.0
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 120
Applications Server
Web Server
SOA (WS, ESB, Etc.)
Network Convergence
(VOIP, IP Video)
AJAX frameworks
HTTPs Request - Response
Citrix
Microsoft RDP
Citrix
Microsoft RDP
.NET 1.02.0 Remoting
.NET 3.0 – WCFWPF
Rich Internet UI (AJAX, FLEX, APPLETS)
HTTPs RequestResponse & SOAP
121. LoadRunnersupportforWeb2.0
Web 2.0 Bundle
Ajax Framework Solutions
Flex 1.5 support forAMF version zero
Flex 2.0 support - in 9.1
Flex 3.0 support – in 11.0
Web services protocol
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 121
122. AJAX–Whatisit?
• AJAX is Asynchronous JavaScriptAnd XML
• A web development technique for creating interactive web applications
• Allows exchanging small amounts of data with the server behind the
scenes, without the page having to reload each time
• Meant to increase web pages interactivity, speed, and customer
usability
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 122
123. Flex–Whatisit?
Adobe Flash has unique animation capabilities
Traditional application programmers found it challenging to adapt to Flash
Flex minimizes the problem by providing a workflow and programming
model that is familiar to application developers.
Flex provides an easier way to build Flash applications
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 123
125. WCF–Whatisit?
Windows Communication Foundation, previously codenamed Indigo is a
communication subsystem to enable applications, in one machine or across
machines over network, to communicate.
TheWCF programming model unifies
Web Services
.NET Remoting
DistributedTransactions
Message Queues
LoadRunner support forWCF
WCF protocol support
.NET 3.5 support
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 125
126. NextGenerationofApplications–Web2.0
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 126
Applications Server
Web Server
SOA (WS, ESB, Etc.)
Network Convergence
(VOIP, IP Video)
AJAX frameworks
HTTPs Request - Response
Citrix
Microsoft RDP
Citrix
Microsoft RDP
.NET 1.02.0 Remoting
.NET 3.0 – WCFWPF
Rich Internet UI (AJAX, FLEX, APPLETS)
HTTPs RequestResponse & SOAP
127. RemoteAccessTechnologies–Whatisit?
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 127
Remote AccessTechnology
Citrix – Market Leader
Testing Challenges
Protocols support any activity
over the network not specific to
one application
Very large deployments to
support many customers
Remote Access is single point of
failure for business activity
128. RDP-Recording
A user creates a connection to a server using Remote desktop client:
After the desktop appears - user’s keyboard and mouse actions are recorded
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 128
135. AScenario
A scenario is the mean to emulate a real-
life behavior of application users – as in
real environment.
Scenarios include
1. a list of machines on whichVusers
run
2. a list of scripts that theVusers run
3. a specified number ofVusers or
4. Vuser groups that run during the
scenario
Scenarios are created using the
LoadRunnerController
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 135
136. Typesofscenarios
A Manual Scenario gives you control over the number of running
VUsers and the times at which they run, and lets you test how many
VUsers your application can run simultaneously. Create a scenario
by assigning VUsers quantity against each script/group and load
generators.
You can use the Percentage Mode to distribute the total number of
VUsers among scripts based on a percentage specified by your
business analyst.
A Goal-Oriented Scenario is used to determine if your system can
achieve a particular goal. LoadRunner automatically builds a
scenario based on these goals.
ResponseTime for specified transaction
Number of hits/transactions per second etc
…
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 136
137. Typesofscenarios
A Manual Scenario gives you control over the number of
runningVusers and the times at which they run, and lets you
test how manyVusers your application can run
simultaneously.You can use the Percentage Mode to
distribute the total number ofVusers among scripts based on
a percentage specified by your business analyst.
A Goal-Oriented Scenario is used to determine if your system
can achieve a particular goal.You determine the goal based
on, for example, a specified transaction response time or
number of hits/transactions per second, and LoadRunner
automatically builds a scenario for you based on these goals.
You will create a goal-oriented scenario in Lesson 9,
“Advanced Goal-Oriented Scenario.”
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 137
138. CreatingaManualScenario
CreateVuser groups
• Each script selected in the New Scenario dialog box is
assigned to aVuser group
• EachVuser group is assigned a number of virtual users
• AllVusers in a group can be assigned to run the same
script on the same load generator machine
OR
• Different scripts and load generators can be assigned to
the variousVUsers in a group
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 138
139. CreatingVUsersGruops
• Click the Add Group button on the
right of the Scenario groups
window.The Add Group dialog
box opens.
• Enter a name for theVUser group
and specify the number ofVusers
• Select a load generator from the
Load Generator Name list
• Select Add from the Load
Generator Name list if the load
generator is not listed.
• Select a script from the script list
• Click OK to close the Add Group
dialog box.The new group's
properties appear in the Scenario
Groups window
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 139
140. ConfiguringVUsersandVUsersgroup
Select theVuser group whose
VUsers scheme is to be modified,
and click theVusers button on the
right of the Scenario Groups
window.TheVusers dialog box
opens.
Here you can:
change the Script
change the Load Generator
Add/removeVUsers to the group
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 140
141. ConfiguringLoadGenerators
Click the Generators button to open
the LoadGenerator dialog box.
Name, Status, Platform and Details
of the LoadGenerators are
displayed
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 141
Changes LG status
from Down to ready
Add a new LG
Delete an LG
Open
Details of an LG
Disable an LG
142. Configuringascenario
Select group and click Details
button
Run-Time Settings from the
Controller shows the setting of
script saved inVuGen. If these
settings are modified in Controller,
LoadRunner runs the script using
the modified settings.This
overriding of settings is not saved
in script.
Clicking View will open script in
VUGen.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 142
143. SchedulingaScenario–Makeitreal!
Selecting Initialize allVusers before Ramp-Up, instructs LoadRunner to
initializeVusers before beginning to load them
To gradually run theVusers, select the number ofVUsers to begin running
concurrently and the amount of delay-time.
LoadRunner has to wait betweenVuser ramp ups.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 143
144. SchedulingaScenario–Makeitreal!
Click Next to navigate to Duration ActionType, there are 3 choices
1. Run until completion
2. Specify the amount of time for which the scenario will run, once all the Vusers have been
ramped up
3. Run indefinitely
Selecting Initialize allVusers before Ramp-Up, instructs LoadRunner to initializeVusers
before beginning to load them
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 144
145. SchedulingaGroup
After creating aVUser group, a group’s
execution settings can be scheduled
Select Schedule By Group to set the schedule
for theVUser group.
The options are similar to that of scheduling
the scenario – in new versions.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 145
VUser group Execution Settings:
Amount of time that the group must wait
before it starts running
The number ofVusers that will run within a
specified period of time
The amount of time the group will run
146. OldScheduler
A fixed structure: Ramp Up –
Duration - Ramp Down
Very hard to simulate real life
behavior
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 146
147. DynamicScheduler
Accurate load testing capabilities
Easy to simulate real life scheduling
Increased usability, more Flexible
First step towards automatic generation of real life scenario from imported
production data
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 147
148. DynamicSchedulerContents
Multiple Ramp ups / downs
Easier to use
Interactive Graph - edit actions, drag, zoom
Grid for detailed editing
Timeline at runtime
Pause/Resume scheduler
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 148
150. RealLifeSchedule
Each schedule (by scenario / by group) is composed of a list of Actions:
Fixed actions – Start Group, Initialize
Dynamic actions
StartVusers, StopVusers, Duration
Can be used multiple times, in any order
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 150
151. VUsersQuantity
Determined by the scheduler – not by the group !
No Ramp Up “All”
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 151
152. ManualOperations
Do not affect the planned scheduling - not reflected in the design graph
Affect the runtime behavior - manually run vusers will be missing for next
ramp
Example: at a certain point, the user manually started some more vusers
Blue – expected scheduling
Red – actual behavior
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 152
Note: the red line is not
visible in the scheduler graph
153. All slides after this are in draft state.Those will be modified and updated on
website soon.
Keep checking back.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 153
158. GoalOrientedScenario–DefiningaGoal
• the number of users that you want
to use to reach that goal
• the scenario duration
• load behavior
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 158
159. DefiningaGoalcont.
• Specify that the test should run for 5 minutes after the target has been
achieved, and select Continue scenario without reaching goal.
• Once the load of three hits per second has been reached, the
Controller keeps running the scenario for an additional five minutes,
adding or subtractingVusers as needed to keep the actual
measurement within 6% of the specified target.This is to ensure that
the server can sustain the load for a period of time.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 159
160. ServiceLevelAgreement–SLADefinition
The user may define goals for the run on several measurements, like:
Average transaction response time
Total throughput (bytes) per run
The measurements are checked at run time, to see if there is a violation
The results can be viewed in the Analysis application
Transactions success/failure stand up to SLA expectations
LoadTest SLA violations, and the time they occurred
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 160
161. SLA Wizard
A wizard for easily defining SLA goals
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 161
166. What'sNext
Understanding HP Analysis [LR component]
Understanding various graphs, merging graphs, collating, correlating
Analysis and diagnostics
Reporting
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 166
167. AnalysisofTestResults
There comes a time after the tests are run when someone who's reviewing
the results asks the deceptively simple question, "So what, exactly, does all
this mean?!?"This point beyond performance testing is where the
capabilities of the human brain come in handy.
"Computers are good at swift, accurate computation and at storing great
masses of information.The brain, on the other hand, is not as efficient a
number cruncher and its memory is often highly fallible; a basic
inexactness is built into its design.The brain's strong point is its flexibility.
It is unsurpassed at making shrewd guesses and at grasping the total
meaning of information presented to it," writes British journalist Jeremy
Campbell inChapter 16 of Grammatical Man: Information, Entropy,
Language, and Life (1982).
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 167
168. Graphs:HitsperSecond
• The Hits per Second graph shows the number of hits (HTTP requests)
made to theWeb server byVusers during each second of the scenario
run.
• You can see that after a short period the required level of load is
reached.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 168
169. Graphs:Trans.ResponseTime
• TheTransaction ResponseTime graph shows the amount of time it
takes for each transaction to be completed.
• Its important to watch
transaction response time to see the response times your customers
will have when the server is under load.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 169
170. Graphs:WindowsResources
• You can monitor the server’s Windows Resources usage for processor,
disk, or memory utilization problems.
• Monitoring during a test can help you pinpoint the source of poor
performance immediately.
You can check theWindows Resources graph legend for a list of measurements.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 170
172. LoadRunnerAnalysis–cont.
Session Explorer:
In the upper left pane,Analysis shows the reports and graphs that are open
for viewing. From here you can display new reports or graphs that do not
appear when Analysis opens, or delete ones that you no longer want to
view.
GraphViewingArea:
In the upper right pane,Analysis displays the graphs.
By default, the Analysis Summary report is displayed in this area when you
open a session.
Graph Legend
In the lower right pane, you can view data from the selected graph.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 172
173. MoreinAnalysis!
There is a spectrum of graphs to
analyze.You can apply filters or even
cross with results. For every chart,
diagram, table or picture you add to
your presentation, you can go from
understated to wow or anywhere in
between.
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 173
174. AnalysisReport - WordandExcel,too!
Export to and spreadsheets that
match your presentation for a
“branded” look.
Cross with Results
Global Filters
Add Graphs
How to generate reports
Exporting reports into HTML and
*.doc
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 174
179. What'sNext
Understanding HP Analysis [LR component]
Understanding various graphs, merging graphs, collating, correlating
Analysis and diagnostics
Reporting
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 179
180. TriggerLeadTimetoReduceUpgradeRisk
Time
Predicted
point of failure
Upgraded Capacity
ExistingCapacity
Lead
TimeTrigger
Point
Anticipated
Peak Loads
Work
load
Usable
Reserve
Capacity
Point ResponseTime
degradation becomes
noticeable
Current
load
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 180
185. PerformanceDesignandManagement
A real example of client requirement (A Fortune 100 company):
Describe the performance aspects of your system, e.g. from user actions
throughWeb server through application server through database and
back, including number of registered users/simultaneously-logged-on
users/simultaneously-active users, aggregate expected network traffic for
all segments, CPU loads/memory utilizations for client/app
server/database server, anticipated disk space requirements and growth,
and any other salient performance design assumptions or provisions.
Describe how these usage patterns will vary by time of day, day of the
week, and month of the year.]
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 185
186. RequiredCapacityPlanningProcesses
A real example of client requirement (A Fortune 100 company):
[Describe your expected capacity design rules and processes: what
resources need to be adjusted as application loads change, how much
disk/CPU/memory per user is needed, anticipated growth over time, and
the processes you expect the support teams to use to adjust capacity
during the lifecycle of the application.]
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 186
187. PerformanceTest ExitCriteria:
For PerformanceTest completion, the following items must be accomplished (Real example by Fortune 100 Client):
Performance test results have been reviewed and accepted by client.
All defects raised during test execution have been fixed and any deviations have been documented and agreed by client.
Chrome will provide effort estimates for all agreed deviations or faults that have been accepted by client.
Transaction response times during the course of the test runs should adhere to the target response times defined in SLA
Transactions meeting response time SLAs = 100% (See KeyTransaction table below for pass criteria)
Running / passed users during the test = 100%
Expected transaction throughput during Peak state = 100%
Average 15-minute CPU utilization should be < =70% for database and Application Server.
Average 15-minute Memory utilization on servers should be < =70%
No gradual loss of free memory over the course of the tests
Network bandwidth utilization in peak state <=70%
On database server, 15-minute average IO wait times must not exceed 10 milliseconds for more than 5% of test duration.
Any deviation (outside of agreed upon failure rate of 1%) to be discussed with client on a case-by-case basis.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 187
188. TTFBandTTLB
In the case of objects with longTTFB times, like index.jsp, the solution often requires a
software-level optimization. It could involve adding a database index, introducing some
object-level caching, or a configuration change (such as database connection pooling).
Be careful to fall in to the trap of throwing more hardware at the problem to solve these
types of issues.While it might work in the short term, these issues almost always are
due to sub-optimal software and throwing extra hardware at the problem will be like
putting a band-aid on a bullet hole.
In the case of objects with relatively shortTTFB times but overall longTTLB times, the
solution is usually very different.While there may be a software solution, such as
configuring Apache’s connections to be better optimized for the server it runs on, most
of the time the root cause is due to network/hardware-related issues. Check with the
ISP that hosts the server to confirm the max bandwidth throughput allowed. If the
object response is slow during peak times but fast during off-peak times, it may need
extra web servers (ie: hardware).
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 188
189. ColdStartupandWarmStartup
Understanding Cold Startup andWarm Startup of an application
Cold Startup
Cold startup occurs when your application starts for the first time after a system reboot,
or when you start your application, close it, and then start it again after a long period of
time.When an application starts, if the required pages (code, static data, registry, etc)
are not present in theWindows memory manager's standby list, page faults occur. Disk
access is required to bring the pages into memory.
Warm Startup
Warm startup occurs when most of the pages for the main common language runtime
(CLR) components are already loaded in memory, which saves expensive disk access
time.That is why a managed application starts faster when it runs a second time.
Ref: http://msdn.microsoft.com/en-us/library/cc656914.aspx
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 189
197. GetCoordinatedwithThemes!
OfficeThemes give you effortless
coordination of the colors, fonts and
graphic effects applied to your
presentation. Everything you insert
into your presentation will be
automatically styled to match.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 197
199. SlidePerfect!
PowerPoint has new layouts that
give you more ways to present your
words, images and media.
Everything you add into the layout
placeholders is arranged perfectly
for you.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 199
200. ArrangeitYourWay
With custom layouts you design
your own slide arrangements.You
can create new kinds of documents
with PowerPoint. Easily create
everything from business pitchbooks
to family photo albums.
HP LoadRunner - Kamran Khan – ChromeIS.com/institute 200
Scenario Modification: Adding/ Removing Vusers, Real-life schedule, group or scenario configuration and execution etcDirect Revenue:LR Consultants making up to 85 USD per hourRef: http://www.hotgigs.com/rates/skill/LoadRunner-hourly-consultant-bill-rates/
Is not part of Load Testing, however, if downloading of HTML, javascript files or CSS fails because of non-responsiveness of Web Server, then it is a concern of Performance Testing.
Vuser Script Sections Each Vuser script contains at least three sections: vuser_init, one or more Actions, and vuser_end. Before and during recording, you can select the section of the script into which VuGen will insert the recorded functions. The following table shows what to record into each section, and when each section is executed. You use the VuGen script editor to display and edit the contents of each of the script sections. You can display the contents of only a single section at a time. To display a section, highlight its name in the Sections box.
HTML-Based Scripts The HTML based script option, which is the default recording mode, instructs VuGen to record HTML actions in the context of the current Web page. The actions are recorded as URL steps (web_url), link steps (web_link), image steps (web_image), and form submission steps (web_submit_form). VuGen records only the HTTP requests that return HTML pages, but not javascript or applications. /* HTML-based *//... web_link("Click Here For Additional Restrictions", "Text=Click Here For Additional Restrictions", "Snapshot=t4.inf", LAST); web_image("buttonhelp.gif", "Src=/images/buttonhelp.gif", "Snapshot=t5.inf", LAST); . . .
Benefits of HTML Recording• Reduces need to capture dynamic values – action tag values and hidden data are NOT hard coded – they are retrieved from memory during playback – if they are dynamic, the Vuser still runs• Script is only as big as the business process–one step per pageDisadvantages of HTML Recording• Scripts are less scalable• Memory (cache) is searched during playback – requires more memory – requires more CPU powerBenefits of URL Recording• Flexibility – support for Java Applets and ActiveX objects on the page – ability to replay on Unix• Scalability – scripts are more scalable than HTML scripts because they require fewer resourcesDisadvantages of URL recording• Scripts require more correlation (nothing is retrieved from cache)• Context-sensitive checks won’t work (parser is disabled)*• Scripts are large (all images and frames are recorded as separate steps)
Setting Advanced Recording Options Use the Advanced recording options tab to record custom headers, apply a content type filter, specify non-resources, and set the think-time threshold. Recording Think Time Think time emulates the time that a real user waits between actions. To record user think time, select Record think time. You can specify the minimum amount of time that a user waits that should be recorded as think time by defining a Think time threshold. For example, you can set the think time threshold to 5, so if a user waits for less than five seconds, think time is not recorded. Use the Think Time run-time settings to influence how the Vuser uses the recorded think time when you run the script. Reset Context for Each Action You can instructs VuGen to reset all HTTP contexts between Actions, to their states before the initial recording. This setting allows the Vuser to more accurately emulate a new user beginning a browsing session. It resets all cookies, closes all keep-alive connections, clears the cache, and resets the user-names and passwords. (enabled by default) Creating a Trace Log Select Full trace recording log to create a trace log during recording to use when debugging your script. (disabled by default) Recording Custom Headers Web Vusers automatically send several standard HTTP headers with every HTTP request submitted to the server. Click Headers to instruct VuGen to record additional HTTP headers. You can work in three modes: Do not Record Headers, Record Headers in list, or Record Headers not in list. When you work in the first mode, VuGen does not record any headers. In the second mode, VuGen only records the checked custom headers. If you specify Record headers not in list, VuGen records all custom headers except for those that are checked. By default, headers are not recorded.Filtering Content Type VuGen allows you to filter the content type for your recorded script. You specify the type of the content you wish to record or exclude from your script. You can work in three modes: Do not Filter Content Types, Exclude content types in list, or Exclude content types not in list. When you work in the first mode, VuGen does not filter any content type. In the second mode, VuGen only excludes the selected content types. If you specify Exclude content types not in list, VuGen filters all content type except for the ones that are checked. By default, no filters are active. Handling Non-HTML Resources Many Web pages contain non-HTML elements, such as applets, XML, or javascript. These elements usually contain or retrieve their own resources. For example, a javascriptjsfile, called from the recorded web page, may load several images. An applet may load an external text file. Using the Advanced options, you can control how VuGen records non-HTML elements. The following options are available: Do not record non-HTML elements Record non-HTM elements as separate functions Record non-HTML elements in current HTML function (default)
Data gets relocated in the server’s cache when it is reused for each Vuser execution. As a result, server activity is not emulated during playback. While Vuser execution is successful, it will not produce test results meaningful for load testing because the server is not exercised. Using different data better emulates server activity and exercises the system. That improves the chances for meaningful load test results.A date that is hard-coded in the VUser during recording may no longer be valid when that date has passed. Parameterizing the date allows Vuser execution to succeed by replacing the hard-coded date.A field that uses unique data is constrained because the business process will not allow reuse of the data after it has been added to the system and processed.
lr_output_message ( "We are on iteration #%s", lr_eval_string ( "{iteration}" ) );
Creating Parameters You create a parameter by specifying its name and type. There is no limit to the number of parameters you can create in a Vuser script. There is no limit to the number of records present in a parameter file, however, LR may show only first 100 parameters in view state. In Web Vuser scripts, you can create parameters in the text-based script view using the procedure below, or you can create parameters in the icon-based tree view.
Creating Parameters You create a parameter by specifying its name and type. There is no limit to the number of parameters you can create in a Vuser script. In Web Vuser scripts, you can create parameters in the text-based script view using the procedure below, or you can create parameters in the icon-based tree view. To create a parameter: Right-click on the string to parameterize. The right-click pop-up menu opens. From the pop-up menu, select Replace with a Parameter. The Select or Create Parameter dialog box opens. Type a name for the parameter in the Parameter name box, or select an existing parameter name from the list. Select a parameter type from the Parameter type list. Click OK to close the Select or Create Parameter dialog box. VuGen replaces the selected string in your script with the name of the parameter, surrounded by angle brackets. To replace additional occurrences of the string with the same parameter, right-click on the parameter. From the pop-up menu, select Replace More Occurrences. The Search and Replace dialog box opens. The Find What box displays the value you want to replace. The Replace With box displays the parameter name in angle brackets. Select the appropriate check boxes for matching whole words or case. Click Replace or Replace All.
Question: write benefits of correlation?
The cause of this Vuser execution problem is that the dynamic value returned from the server was recorded and, therefore, was part of the Vuser script. When the script is played back, the Vuser attempts to reuse the recorded value, while the server is expecting another value, assigned for the current playback session.
Ref: http://msdn.microsoft.com/en-us/library/ms731082.aspxWindows Communication Foundation (WCF) is a framework for building service-oriented applications. Using WCF, you can send data as asynchronous messages from one service endpoint to another. A service endpoint can be part of a continuously available service hosted by IIS, or it can be a service hosted in an application. An endpoint can be a client of a service that requests data from a service endpoint. The messages can be as simple as a single character or word sent as XML, or as complex as a stream of binary data. A few sample scenarios include:A secure service to process business transactions. A service that supplies current data to others, such as a traffic report or other monitoring service.A chat service that allows two people to communicate or exchange data in real time.A dashboard application that polls one or more services for data and presents it in a logical presentation. Exposing a workflow implemented using Windows Workflow Foundation as a WCF service.A Silverlight application to poll a service for the latest data feeds.
You can instruct LoadRunner to execute a scenario or Vuser group with a delay. The time duration of a scenario or Vuser group can be limited. The scenario or Vuser group stops running after the elapse of this time.You can stipulate how many Vusers LoadRunner runs within a certain time frame during a scenario or Vuser group.
i.e. no way to keep on fluctuating user ramp up/ramp down behavior
Once a problem has been isolated, a corroborative effort involving developers, DBAs, network, and other systems experts is required to fix the bottleneck. After adjustments are made, the load test is repeated to confirm that the adjustments had the desired effect. You repeat this cycle to optimize system performance.