Agile
Testing
Transforming a Traditional QA team to
Agile: A 4 Step Approach
White Paper

ADOPTING AGILE TESTING
Executive Summary
A growing need for quicker and adaptive solutions to tech problems is pushing firms to adopt the agile methodology.
Today more and more companies are addressing different technology issues by adopting this iterative approach to
software development and releasing high quality software, faster and more efficiently. Organizations see agile software
development as a faster way to create products, thereby reducing the Go To Market time. The benefits of adopting the
agile software methodology include improved quality, flexibility to midcourse correction, improved business satisfaction,
better alignment between the teams and improved time to market.
However, adopting the agile development methods means that traditional software development organizations must
change, in particular the functioning of QA teams. This white paper discusses the approach to transition a traditional QA
team into an agile QA team, when an organization decides to adopt the agile methodology
2 Page

ADOPTING AGILE TESTING
What is Agile QA?
The objective of any testing is to uncover defects (deviations from the defined business requirements) in
software and to ensure that it works before it is shipped
to customers. This objective stays the same for traditional as well as agile QA; and the traditional testing
techniques are very relevant to testing in agile as well. In
fact, testers on agile teams exploit the benefits of traditional QA within an iterative development process, but
the structure, communication, test execution frequency
and escalation methods are different in agile testing.
In the traditional QA model, testing is performed by a
separate group with its own line of reporting and organization. In this model, while the developers design the
code, the QA team develops the test plan. For both
teams, the requirements specifications from the
requirements engineering group form the baseline. The
QA team receives the product, often after months of
design and coding for an agreed set of requirements by
stakeholders. Then the QA team tests this code and
reports defects for the development team to fix. When
developers fix the defects filed by QA, they release the
newer version of the software to QA; this cycle repeats
until the product is ready to be shipped.
Although, this practice has been followed for many
years across different organizations, it has its own
limitations.
• Developers test their code less and rely more on
testers, leaking defects to testers who find defects
late in development lifecycle, impacting time to
market
• Significant delays before developers receive feedback
from testers, again impacting time to market
• Frequent changes to test plan and test cases, as
business requirements change.
• If the changed requirements are not communicated
to all stake holders, then there is a mismatch in expec-
tations in the final product across all stake holders
• QA is usually the last activity before the published
release date, and any delay in either requirements
gathering or development, leads to a shorter time
frame for testing. Extreme pressure from management
to complete testing within the available time will
compromise quality of the final product.
• Business owners are involved in UAT after testing is
formally completed. Observations from business users
come very late in the project lifecycle before deployment, and development may have moved to the next
version of the release.
On the contrary, in the agile methodology, QA is normally
entrenched in the agile project team, working alongside
the developers and business users at every stage of the
process. This early involvement and transparency ensures
that the QA team is engaged much earlier in the software
development life cycle, and gets the requirements directly
from business users. This leads to early identification of
dependencies, technical or testing challenges, project
roadblocks and these are discussed with stake holders and
built into estimation and project planning.
Some of the characteristics of agile testing include:
• Agile testers are part of a cross functional team and talk
directly to developers and have their say in all phases of
the software development lifecycle.
• The test team is colocated with the dev team and their
participation in daily scrum meetings ensures information from all stakeholders is uninterrupted.
• Continuous feedback loop to development team
• Frequent iterative releases mean, more regression,
hence the agile test team frequently makes use of test
automation as a way to cut down testing time.
3 Page

ADOPTING AGILE TESTING
• Agile testing drives development by refining acceptance criteria and questioning stories during iteration
planning
• Test cases always exist, are expanded / modified and
executed continuously
• Usually the unit, integration and acceptance tests are
automated.
These characteristics give agile testing certain advantages
over traditional testing, such as
• Ongoing feedback to developers, allowing testers to
ask the right questions and fine tune testing
• As the final acceptance criteria are established prior
to the work beginning, testing is always embedded in
agile process and cutting corners such as skipping
testing cannot happen. In fact testing measures the
project progress in this methodology
• The agile process emphasizes testing at lower levels
such as unit and integration tests. Regression risks are
managed in lower levels of testing, improving time to
market.
Agile Testing
Transformation
in Organizations
When an organization decides to move from traditional
development to agile development, transitioning the
traditional QA team to an agile QA team is a critical task
for the management. This task is not difficult if planned
properly. This transformation not only requires changing
the way the testing team works, but also requires changes throughout the entire organization, from management to business organizations to programmers and
support teams. This section describes the different
aspects to be considered while transitioning a traditional
QA team to an Agile QA team. Management must
emphasize the following critical aspects though training,
counseling and coaching the traditional QA team before
starting the transition.
1. Change in mindset
Team Work: In the traditional model, testers start testing
after the coding is complete. Testers and developers often
communicate through the defect tracking system and they
belong to different groups within the organization. However, an agile tester is tightly integrated with the development process, and participates in planning, estimating,
scheduling and story definition. An agile tester does not
merely check the quality of the software; instead he/she
moves from a mindset of “how can I break the system” to
a mindset of “how can I deliver quality software successfully”.
Collaborate and communicate: Agile teams do not
produce hundreds of pages of requirements documents
that form the basis for test cases in traditional QA. Agile
testers must proactively work with customers and developers to get the requirements. In fact, agile testers need to
serve as a bridge between customers and developers.
When requirements are unclear; customers, programmers, and agile testers discuss together to clarify requirements. The agile methodology values constant feedback
and communication in order to avoid surprises in the later
stages of development. This means that unlike the traditional QA team which usually keeps test design, test document, test progress and QA specific metrics within QA
boundaries, and publish them once in a while, agile
testers provide visibility into testing activities during daily
standup meetings, so issues can be addressed early in the
sprint cycles.
4 Page

ADOPTING AGILE TESTING
Unlearn: When an organization moves from the traditional approach to agile, many testers from the traditional QA
team move to agile QA with the traditional QA mindset.
Learning the agile way of testing is pretty straight forward,
as agile is a light weight process. However unlearning past
behaviors and practices is the difficult part. For example, in
traditional QA, an organization may be comfortable with
manual testing, and may not invest in test automation.
However, with the time boxed nature of agile, investing in
test automation will pay dividends, since manual testing
will increase cost and time. In the beginning, when test
automation is in progress, it makes sense to keep the traditional manual testing. Once test automation is completed,
manual system testing is both expensive and inefficient.
Practices and beliefs such as manual testing, that have
made positive contribution in the traditional QA, now
need to be unlearned if the team is to reach full potential.
2. Develop new Skills
Automation: The time boxed nature of the agile practice
presents some challenges to the QA team. Traditional QA
considers testing as a mass effort at the middle or near the
end of the project. However in agile, the dev team adds
new features in each sprint and testing has to ensure that
the features previously built, continue to work by incorporating testing in each sprint. Also agile encourages change
and must keep changes under control, preventing bugs
from leaking into production. To achieve this, an agile QA
team must shorten the feedback cycle to the development
team, on features implemented. This is where the importance of test automation comes into picture. Teams that
are transiting from traditional QA to agile QA, must invest
more in test automation. Without automation, the test
team will be bogged down with running an ever increasing
set of test cases manually, under a tight schedule. This is
not scalable and sustainable. With automation at all levels
of development i.e. unit, integration and system levels and
continuous execution of automated test cases, the agile
test team achieves speed, quality and efficiency
Change the Way Traditional QA Measures Quality: Many
organizations transitioning from traditional QA to agile QA,
continue using the same metrics they used for traditional
QA. Because the two processes are fundamentally different, organizations and project teams adopting the agile
practice must come up with a different set of metrics for
quality. For example, the metric “test pass and fail ratio”,
in traditional QA, does not have any meaning in the agile
methodology, since the moment tests start failing, the dev
team stops and fixes the regression right away. Teams
transitioning to agile QA, can come up with metrics related
to parameters such as velocity, cycle time, code coverage,
burn down and defects raised by customers; depending
on business needs of stakeholders, customers and
projects.
Programming Skills: Any tester who can write a test script
in a simple word processor should be able to work on an
agile team. However, a tester with very good programming
and automation skills will be able to make the testing
activity more productive and efficient by continuously
automating features in each sprint.
Exploratory Testing: Test automation is an integral part of
agile testing to address the challenges of the time boxed
nature of the agile practice. However, test automation
poses its own challenges to development work, as automation cannot always address issues related to usability,
reliability, performance, compatibility and other quality
criteria. Using exploratory testing, some of the gaps left by
test automation may be covered. The other reason why
agile testers need exploratory testing is that in an agile
project, the requirements may not be concrete, and
changes are part of any agile project. As it is difficult to add
test cases for unexpected changes and review them,
exploratory testing is an essential skill to uncover the
additional considerations.
5 Page

ADOPTING AGILE TESTING
Exploratory testing helps agile test teams to cope with the
ever changing context of the project. Management must
invest in training the QA team on exploratory testing skills.
Once the testers adopt exploratory testing, they will
understand which risks are important to the team, where
they should focus, how to improve test coverage and make
appropriate daily decisions to deal with unexpected
changes in the project.
Working with Lean Process and Documentation: A typical
agile practice moves QA activities upstream in the project
lifecycle, enabling QA to prevent defects rather than
finding them at the end of project lifecycle as in traditional QA. Agile believes in less documentation and agile
testers may not get the required documents at the beginning of the project lifecycle or for that matter, throughout
project lifecycle. For this reason, agile testers must develop skills to document test case/test script or test requirements. Testers can share these test requirements with
customers, so the gaps in the original requirements can be
filled immediately. Test Driven Development (TDD) or
Behavior Driven Development (BDD) can be adopted for
this purpose. A traditional QA team, transitioning to agile
QA, must learn how techniques such as TDD and BDD work
and how to write these test cases.
3. Adapt new tools
A traditional QA team transitioning to an agile QA, must be
deeply involved with advance practices such as Test Driven
Development, Behavior Driven Development, Test Automation and Continuous Build and Integration, All these
have significant impact on the day to day activities of the
QA team. These new practices also influence the selection
of tools for testing activities. In addition, as the Agile QA
team is tightly integrated with development and project
activities, they cannot distinguish between QA, Project
Management and Development tools, instead should have
knowledge on all the tools used by different personnel on
the agile team as a whole.
Some of the ALM tools that are used in the agile methodology that testers must be familiar with, are shown in the
table below.
ALM Phase
Tools
Exepctations from QA
Project Planning
Project Management tool
Testers must be familiar
Project Status
Sprint and Release status,
Resource allocation and status,
Burn Down charts
Testers must be hands on
Story Capture
Theme, epic and story capture,
acceptance criteria
Story Capture
Testers must be hands on
Documentation management
Simple tools like wiki or any other
tools
Testers must be hands on
Software Development
Continuous integration, unit testing
tools, build tools, source control,
checklist, code coverage
Software Development
Testers must be hands on
Manual test case creation, TDD &
BDD creation, automated test case
management,
Defect Management
Integral part of Agile QA team
Defect creation and tracking, Defect
integration with sprints, stories and
requirements.
Integral part of Agile QA team
Test Management
Defect Management
6 Page

ADOPTING AGILE TESTING
As can be seen from the table, an Agile QA team uses
most of the tools in different stages of the ALM. Organizations must provide, tools, support and training to agile
QA teams, to help them adapt to this new landscape of
tools.
may get frustrated to test multiple builds with
changed requirements in the same iteration. It is the
responsibility of management to make these QA
teams understand the importance of iterative system
testing and motivate them to adopt the agile testing
practice.
4. Position your System
Testing Phase in Agile QA
• In many organizations, system testing slots are
booked for different projects to optimize the hardware resources shared by many projects. If agile test
teams decide to execute system testing whenever a
feature is changed within any given iteration, it is very
difficult to block the hardware in advance. Organizations adopting agile practices must come up with
plans to address such situations, including any external dependencies.
In traditional QA, System Testing is positioned towards
the end of the development lifecycle, starting after
development is completed, but before the product is
released to customers. This is well controlled by entry
criteria to start system testing. In contrast, the agile
process demands that the build from every sprint cycle,
be ready for deployment. When organizations transition
to agile QA, they must come up a proper strategy,
addressing the following issues:
• Typically, traditional QA segregates system testing
specialists to a single group, performing system
testing for different development teams. Breaking
down the virtual walls between different teams and
tightly coupling the system testers to agile teams may
be a challenge. Organizations must design a strategy
where system and performance testing are an integral
part of the agile development process.
• Testers may not get fully completed features in each
sprint, making system testing a challenge. Organizations must work with different stakeholders and set
expectations that system testing cycles will be executed every iteration (say, every third or fourth sprint)
rather than at the end of every sprint.
• An agile process is adaptive, designed to accommodate and encourage inevitable changes. This may lead
to frequent changes in the system, and the testers
Conclusion
Organizations adopt the agile software development
methodology as a faster way to create and release products. The benefits of adopting the agile software methodology include improved quality, flexibility to
midcourse correction, improved business satisfaction,
better alignment between the teams and improved time
to market. However, adopting agile development methods means that traditional software development organizations must change the way QA teams work. Traditional QA teams must change their mindset, break down
the virtual wall with development teams, learn newer
testing techniques such as exploratory testing, testing
based on TDD and BDD. Organizations must see automated testing as an integral part of agile testing and not
just rely on manual testing. The testers in the agile methodology must learn newer tools and skills. They must ask
the right questions, while working collaboratively with
other groups, in overcoming some of the challenges of
testing in the agile methodology.
7 Page

ADOPTING AGILE TESTING
About the Author
Harsha B N works as a Test Architect in the Mobility division of Idexcel. He has twelve years of experience in development and testing mobile applications. Prior to joining Idexcel Harsha worked with Nokia for eight years in various
capacities as Program Manager, Chief Test Engineer, Project Manager working on OTA infrastructure development,
Mobile Payments services, S60 SDK.
About Idexcel
Idexcel is an innovative provider of IT Products & Services focused on emerging technologies. We help world leading
companies build efficiencies and stronger businesses. With more than 15 years into existence Idexcel’s main focus is
client satisfaction and technology innovation. Our industry expertise and a global, collaborative workforce forms the
backbone of our services. We offer high degree of skills in Enterprise Applications, Cloud Services, Data-warehousing,
Big Data, Analytic, QA & Testing Services, IT consulting and Staffing. Idexcel product line includes: NDS, ERP, and
Cync - A revolutionary credit monitoring application for the manufacturing and financial management.
For more information log on to www.idexcel.com.
Global Head quarters
459 Herndon Parkway Suite 11
Herndon, VA 20170
Tel: 703-230-2600
Fax: 703-467-0218
Email: inquiry@idexcel.com
India Operations
“Crystal Plaza” 9, 10 ,11
Bhuvanappa Layout, Hosur Road
Bengaluru – 560 029
Karnataka
Tel: +91-80-2550 8830
Email: inquiry@idexcel.com
© Copyright, Idexcel. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means,
electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Idexcel. The information contained herein is subject to
change without notice. All other trademarks mentioned herein are the property of their respective owners.