1. How to Make Automation An
Asset to the Organization
Vipin Jain
Metacube Software, Jaipur, India
QA&TEST 2012
11th International Conference on
Software QA and Testing
October 17-19, 2012 • Bilbao Spain

2. Introduction
• Software Testing is a process that consists of all test life cycle
activities like static and dynamic testing concerned with planning,
preparation and evaluation of software products to determine that
the software products satisfy customers requirements and are fit for
customers use.
• Software Testing is done to find software defects or failures in
advance.
• Software testing can also be stated as the process of validating and
verifying that a software program/application/product:
– Meets the business and technical requirements that guided its
design and development.
– Works as expected

3. Manual Testing
• Manual testing is the process of manually testing software for
defects. It is a laborious activity that requires the tester to possess a
certain set of qualities; to be patient, observant, speculative,
creative, innovative, open-minded, resourceful, and skillful.
Advantage of Manual Testing
• Running the test case is less cost than automation.
• It allows the tester to perform more Ad-hoc testing (1) (random
testing).
• More time for testing enables a tester to find more bugs.
Disadvantage of Manual Testing
• Running tests manually can be very time consuming
• Each time there is a new build, the tester must re-run all required
tests - which after a while would become very dull and tiresome.

4. Automation Testing
• Software test automation refers to the activities and efforts that
intend to automate engineering tasks and operations in a software
test process using well-defined strategies and systematic solutions.
• The major objectives of software test automation is to free engineers
from tedious and redundant manual testing Operations.
• To speed up a software testing process, and to reduce software
testing cost and time during a software life cycle
• To increase the quality and effectiveness of a software test process
by achieving pre-defined adequate test criteria in a limited schedule
• The major key to the success of software automation is to use a
systematic solution to achieve a better testing coverage.

5. Why Automation?
• Over the last decade, test automation has become a crucial part in
the Test planning activities for various QA groups.
• A well written automated test suite is of enormous help in daily
testing activities, especially in today’s agile world.
• With the Advent of Mobile technologies, a new dimension has been
added in Software testing. There are millions of mobile applications
flooding markets each day. They need to be tested effectively with
shorter QA cycles. But, there are a billion combinations of hardware
devices, OS’s, carriers, and networks today and traditional manual
testing cannot cover all these scenarios – especially when the app
to market life cycle has to be short. Automation is the need of the
hour.

6. Advantages of Automation
Reliable1 Repeatable2 Programmable3
Comprehensive4 Reusable5 Fast6
Better Quality
Cost Reduction8
Software7

7. Why it fails and
what factors
contribute to its
failure?

8. Reasons for Failure
• Management unwillingness for it – Be it lack of vision, cost associated or
any other issue, management doesn’t often give a nod for it.
• Lack of Vision - There is no clear vision behind what the automation will
do, what we intend to achieve with it and what are the strategies to do this.
• Time - No time to develop and maintain the automation scripts
• Cost associated - We need the tools for it and we want management to
assign the appropriate budget, but they are not interested.
• Skills needed – The tool is new and appropriate training and time to master
the tool are required.
• Lack of Automation Matrix - There are no clear factors listed against which
we measure our automation results to label it as success or failure
• Automation engineers – There is no in-betweens Developers and QA
Engineers. Developers demoted to "test development", can backfire, especially if
they do NOT have the mindset of what they're testing for.

9. What to Automate?
• Use cases that are fully developed and with clear understanding
should be automated first
• Relatively stable areas of the application over volatile ones must be
automated.
• Automate the tests that are repetitive over multiple builds.
• Tests that tend to cause human error.
• Tests that require multiple data sets.
• Frequently used functionality that introduces high risk conditions.
• Manual tests that are virtually impossible to be carried out.
• Tests that involves a variety of different hardware or software
platforms and configurations.
• Tests that take a lot of effort and time when testing manually

10. When should we do Automation?
• To get maximum benefit, automation testing should be started as
early as possible. It should be run as often as needed. It’s a well
known fact that the earlier testers get involved in the life cycle of the
project, the better. Moreover , the more you test, the more bugs you
find.
• Automated unit testing can be implemented on day one and then it
should be grown into automated test suite.
• Always remember that bugs detected early are a lot cheaper to fix
than those discovered later in production or deployment. Hence
Start early.
TEST EARLY AND TEST OFTEN

11. A Software Test Automation Process
Test Automation
Planning
Design Automation Evaluate tools and select the
Strategies & Solutions best tool(s)
Develop & Implement Test
Automation Solutions
Introduce and Deploy Test
Automation Solutions
Review and Evaluate
Software Test Automation

12. Automation Myths/Realities
Test Plans covering all resource No commercially available tool that
requirements, time needed and can create a comprehensive test
strategy can be auto-generated. plan.
Any application can be tested using No single test tool exists that can
the tool. be used to support all operating
system environments.
It won’t take much time for testing This will take a lot of time in
once automation is done. running and maintaining and re-
running.
An automation test tool is always An automated tool requires new
easy to learn and use. skills; therefore, additional training
is required.
100% test coverage can be Test coverage breadth and depth
achieved. can be increased but 100%
exhaustive testing cannot be done.

13. Automation Myths/Realities
All possible Input combinations can There are instances where it will
be tested using automation take years to test all input
combinations. 1
All possible paths can be traversed Research shows there are
using automation. instances when possible paths are
in Millions and more. 2
Automation is just record/capture Is it? Not at all. Try changing id of
and play back one control on the page and run
your script.
I won’t require a specialist for that It’s a special task, which is a mix of
programming, knowledge of tool
and understanding of application.
Not every tester can do this.

14. Then what should I do to reap benefits
from my automation efforts?

15. Let’s begin !!!
• Identify the interfaces our tooling need to interface with. For
instance, are we going to drive a browser, are we send
webservices requests and validate the responses, are we
validating email notifications, database validations, or
document or image validations etc.
• Did the QA or DEV develop any tool/libraries that we can use?
• What is the focus on? Is it automating new functionality tests
(a challenge with agile development) or catching up with
regression tests (when there are several years of
development with very little focus on automating).
• Which test case tracking system our automated tests need to
be integrated with.
• What is the triggering mechanism for these tests and when?
• Who need to contribute to the test scripts development? QA,
Dev or both?

16. The Initial Plan
• People make mistakes when they try to build an elaborate solution
that attempts to achieve all requirements at once.
• People plan for results but do not plan for the ways to achieve this.
It’s necessary to understand what you eventually want to achieve
through automation, but to get there, plan to have your Test
automation development as an iterative process.
• Try to first figure out solving which problem would give you the
biggest bang for your buck. E.g. Building the Build Acceptance tests
and automatically running them with, at every check-in, or at code
deployment on a QA system, ends up being a good choice as the
first goal. After that you can always build upon your successes.

17. Technology choices
• Identify your goals and what resources you have to achieve
them, then the technology choices become a lot simpler. For
instance, if you intend to use APIs created by the development team
or if they are directly contributing to the solution, then often the
programming/ scripting language choice needs to be something
developers are comfortable with. If you are working in a java shop,
you can’t expect the developers to create VBScript libraries to be
used in QTP. Several tools these days, like Selenium and Fitnesse
support multiple language choices.
• Once you narrow down on the tools that fit the requirements, the
next steps are to start evaluating and prototyping them using a
few simple test cases.

18. Which tool should I use?
Commercial solutions
Pros Cons
Published feature road maps Vendor lock-in
Institutionalized support Lack of interoperability with other
products
Stability (whether real Lack of control over improvements
or perceived)
Licensing costs and restrictions

19. Which tool should I use?
Open Source solutions
Pros Cons
No licensing fees, maintenance, or The downsides of commercial
restrictions tools might be applied to some
open-source projects, but the
Free and efficient support (though varied) advantages of leveraging the
open-source community and its
Platform portability
efforts are holding sway with
Modifiable and adaptable to suit your more and more companies.
needs
Comparatively lightweight
Not tied to a single vendor

20. Which tool should I use?
In-House solutions
Pros Cons
No licensing fees, maintenance, or This involves a complete product
restrictions development life cycle leading to time
and resources requirement.
Free and efficient support The financial aspects associated with
(though varied) its development makes organization
skeptical of its use.
Platform portability Maintenance is costly as it need to be
adjusted with changing requirements.
Modifiable and adaptable to suit Management needs to be patient with
Your needs it.
Comparatively lightweight

21. Criteria in Finding & Evaluating tools
Simple installation/un- Comprehensive and complete
installation of the product documentation. Regularly
maintained with code changes
Configurability—the ability to Strong Support Team, 24x7
be adapted to each evaluation availability and online
activity 1 forums/user groups
Expandability – The tool Tool should be extensible and
should support various flexible to deal with new objects
infrastructures and apps 2 in the new architectures
developed as part of revisions. 3

22. Using Coding Standards in
Automated Software Testing
• Automated software testing efforts can fail if the software
development doesn't take into account the automated testing
technologies or framework in place.
• There should be provisions kept for automation and resources
should be assigned to it
• Changes made in application code will force changes in automated
software testing scripts and hence developers can contribute to the
success of automated testing efforts if they consider the impacts on
them when making code or technology changes.

23. Using Coding Standards in Automated
Software Testing - Guidelines
• Build testability into the application – Make your application
testable.
• Design to facilitate automation tool recognition of objects: All
objects should be uniquely named, consider various
platforms—client/server, Web, etc.—and GUI/interface testing
considerations, such as in the case of Windows development,
for example, within the Windows architecture.
• Don't change the object names without automated software
testing considerations.
• Follow standard development practices; for example, maintain
a consistent tab sequence.
• Follow techniques, such as the library concept of code reuse,
i.e., reusing existing already tested components, as
applicable.

24. Thank You!
Vipin Jain
Sr. Software Test Lead,
Metacube software, India
Vipin.jain@metacube.com