1. FINAL PROJECT:
SOFTWARE VERIFICATION & VALIDATION
Date: June 10, 2009
Authors: Amin Bandeali (amin.bandeali@csu.fullerton.edu)
Instructor: Dorota Huizinga

2. Contents
1. Introduction 1
2. NUnit 1
a. Introduction 1
b. URL 1
c. Primary Features 1
i. Test Harness 1
ii. Assertions 1
iii. Custom Assertions 2
iv. Setup and Tear Down 2
v. Attributes 2
vi. Test Runner 2
vii. Results in XML format 2
d. Software Verification 2
e. Software Phases 2
f. Author’s Comments 2
3. HP Load-Runner 3
g. Introduction 3
h. URL 3
i. Primary Features 3
j. Virtual User Generator 3
k. Controller 3
viii. Load Generators 3
ix. Analysis 4
x. Launcher 4
l. Software Validation (Performance Validation) 4
m. Software Phases 4
n. Author’s Comments 4
4. References 6

3. Final Project: Software Verification & Validation 1
1.Introduction
The emerging need for good software requires detailed specifications and thorough
verifications against them. There are two ways to ensure the quality of a software
component or software system. One way is to identify and then remove defects
introduced; this is done by means of different verification and validation activities.
Another way is to prevent the defect introduction; this is done by means of an
appropriate development process [1].
The purpose of verification is to assure that the software components or software
systems meet their specified requirements [2]. Specified requirements could be
found in SRS, architecture, design and regulations (standards) documents. The
purpose of validation is to demonstrate that a software component or system fulfills
its intended use when placed in its intended environment [2]. Inspection and tests
could be used for verification and validation [3]. In this paper I would discuss two
verification and validation tools namely NUnit and HP Load-Runner.
2.NUnit
a. Introduction
Software testing measures the verification of software against both functional and
nonfunctional acceptance criteria. Unit testing ensures that software does what it
commits or is expected to do. Unit testing started with xUnit. NUnit is a port from
Java’s JUnit. It facilitates unit testing by providing a framework to support test
development and test runners to run the tests and assess their results [4]. It comes
with a console application or a GUI application for testing.
b. URL
http://www.nunit.org/index.php
c. Primary Features
i. Test Harness
In NUnit, a test harness is called a test fixture. It provides an infrastructure to hold
related tests. It contains the logic to setup and tear down the collection of tests in
the harness, as well as for each of the individual tests. [4]
ii. Assertions
Assertions test whether the program is behaving as expected by verifying that an
actual post-condition matches the expected post-condition [4]. If multiple assertions
exist, the test case would stop executing as soon as an assertion fails. These are the
most important tools provided by NUnit for testing as they actually verify
functionality.

4. Final Project: Software Verification & Validation 2
iii. Custom Assertions
These are created by extending the NUnit framework. The primary purpose of a
custom assertion is to have application specific assertions which are easy to maintain
and extend.
iv. Setup and Tear Down
Setup and teardown code allow you to segregate and reuse common initialization,
and clean up code across tests.
v. Attributes
NUnit provides four attributes (two for tests and two for test fixtures) that can be
used to mark methods that provide setup.
vi. Test Runner
NUnit provides two types of runner interfaces – GUI and console. Both interfaces
provide similar functionality [4]. Test runner GUI is most useful while developing,
which test runner console is useful in automated testing environments.
vii. Results in XML format
NUnit allows you to save results in XML format from both the test runner GUI and
console [4].
d. Software Verification
Unit testing begins with acceptance criteria, which is identified during the
requirement gathering/analysis process. Unit testing verifies exactly what the code is
supposed to do and if the results match the acceptance criteria. Unit testing reduces
both integration and system testing efforts since it verifies the component
functionality right after development. NUnit uses two techniques to verify code:
Equivalence sets – a set of input values that are processed in an equivalent fashion
and produce qualitatively the same behavior to make a test succeed and Boundary
values – a set of input values that are used to test conditions where system behavior
should or might change [4].
e. Software Phases
NUnit (unit testing) is used in the implementation and verification phase of the
waterfall model. It is used in implementation phase because if one component
implementation is dependent on another component, the dependency should be
thoroughly unit tested. It is also used in verification phase, where each component is
thoroughly tested for verified functionality.
f. Author’s Comments
I have used NUnit since I have started working on the .NET framework. I personally
use Test Driven Development and NUnit is a very important part of my work. TDD is

5. Final Project: Software Verification & Validation 3
the iterative process of designing and building software one unit test at a time [4].
This is how I use TDD with NUnit:
• I write a test for a new piece of functionality that I would like to implement
(Test currently fails).
• Implement the minimum amount of functionality that the test passes with.
• Refactor, improve as necessary.
The benefit of TDD in my opinion is that it gives maximum code coverage and
maximum verification. Hence, a better quality code.
3.HP Load-Runner
g. Introduction
The explosion of software complexity introduced tremendous risk into the software
development process with an incredible rate of change. Rigorous testing and
verification is the most common strategy to both quantify and reduce this risk to the
business. Performance testing accurately tests the end-to-end performance of a
system prior to going live. Performance testing solutions should be able to:
• Emulate hundreds or thousands of users interacting with the system without
large hardware demands
• Measure end-user response times
• Repeat the load in a consistent way
• Monitor system components under load
• Provide robust analysis and reporting engines
HP LoadRunner software is used more than any other testing solution in the world
today [5]. HP LoadRunner captures the end-user response times of key business
processes and transactions to determine if service-level agreements (SLAs) can be
met [5].
h. URL
https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?
zn=bto&cp=1-11-126-17^8_4000_100__
i. Primary Features
j. Virtual User Generator
The Virtual User Generator (VuGen) is a scripting tool. It is based on a Record and
Playback rule. VuGen records a tester's actions into automated scripts (these scripts
are the base of the load tests) as we walk through a business process on the
application [6].
k. Controller
The Scripts which were generated by VuGen are run via Controller. Controller lets
you organize, manage and monitor the load test [6].
viii. Load Generators
Load generators are the computers used to create load on the application under test
by running virtual users (Vusers). You can use a number of load generator machines,
each hosting many virtual users [6].

6. Final Project: Software Verification & Validation 4
ix. Analysis
For evaluating the Performance and to create graphs and reports we can use
analysis. Bottlenecks in the application are easily recognized and isolated and this
helps us in determining what changes are needed in order to improve its
performance [6].
x. Launcher
The Launcher offers a single point of access for all of the LoadRunner components.
l. Software Validation (Performance
Validation)
LoadRunner helps in validating non-functional requirements like quality attribute.
How the system should scale and behave under certain load conditions is defined in
part of the business requirement analysis phase. LoadRunner helps you prevent
costly performance problems in production by detecting bottlenecks before you
deploy a new system or upgrade [7]. With performance and load testing, a system
would behave as expected as soon as it is out of its alpha testing phase into the beta
testing phase and beyond.
m.Software Phases
LoadRunner is used in the testing phase of the waterfall model. However,
performance testing can be broadly described as having four phases by itself—
Design, Build, Execute and Analyze/Diagnose/Tune [7]. How these phases are
actually divided over the waterfall model is specific to each company. All the
following phases could either be carried out in the testing phase or being as soon as
the implementation phase starts.
The Design phase involves defining the business processes to be tested, the business
process mix of an average or peak production hour, and the overall user and
response-time goals for the system [7].
The Build phase involves setting up and configuring the test system and
infrastructure, and using the automated performance testing solution to build both
test scripts and load scenarios [7].
The Execute phase consists of running the load scenarios and measuring system
performance [7].
The Analyze, Diagnose and Tune iterative phases go beyond measuring system
performance and take load testing to another level. Here, the focus is on pinpointing
problems to help resolve them rapidly and tuning system parameters to enhance
performance in an ongoing fashion [7].
n. Author’s Comments
LoadRunner has been used by almost every company that I have ever worked for.
The reason is simple, it does one thing, and it does it well! Every piece of software
that has passed the LoadRunner test according to the performance requirements

7. Final Project: Software Verification & Validation 5
acceptance criteria has in my experience never failed in production. In my current
work place we have very robust web services which receive anywhere from 5-10
million requests a day. We used LoadRunner to test those web services and
LoadRunner helped us scale them and hence, a successful deployment with minimal
to no problems.

8. Final Project: Software Verification & Validation 6
4.References
[1] Christina Wallin, quot;Verification and Validation of Software Components and
Component Based Software Systemsquot;, ABB Corporate Research.
[2] CMU/SEI-2000-TR-028. CMMISM for Systems Engineering/Software Engineering,
Version 1.02, Software Engineering Institute, November 2000.
[3] Ian Sommerville, Software Engineering 6th Edition, ISBN 0-201-39815-X,
Addison-Wesley, 2001
[4] Bill Hamilton, “NUnit – Pocket Reference”, O’Reilly
[5]https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?
zn=bto&cp=1-11-126-17^8_4000_100__
[6] http://www.editorial.co.in/loadrunner/what_is_loadrunner.php
[7] HP Automated Solutions, “Automate performance testing to predict system
behavior and improve application performance”, Hewlett Packard.