Test Automation Journey - The Past, The Present and The Future

1. The Journey of Test Automation
The Past
Software testingisanecessaryprocessandeveryone seemstoagree onthat.Where people differ is
the timing and depth of the initiative. The industry has long moved from a stage when testing was
consideredapost-development,maintenance activity. We gradually interleaved the development
and testing process. Both the development and the testing process tend to merge, bringing both
newbenefitsandnewchallenges. The obviousbenefitisfastfeedback. We needtocheckan error as
quick as possible. The not-so-obvious challenge is maintenance and overhead. This will come to
haunt us later.
Long since the early days, the industry adopted Continuous Integration. Whenever a developer is
readyto pushsome code,he firsthas to pass itthroughthe UnitTests. If all isgreenthenonlycan he
actually commit the code. Even per-commit jobs are gaining popularity. People are delegating the
responsibilityof code validationtothe buildserveritself. The idea is to integrate the Code – Build –
Test– Deploy processina single pipelinedworkflow. Everystage inthe pipeline providesimmediate
feedback to the preceding stage.
On the developers’ front automated Unit Tests have established themselves as an invaluable
checkpoint. What about the managers and the business users? Everyone wants things to be well
tested. But Unit Tests and their reports are not much meaningful for them. The unit tests are also
not reliable in integration testing. Components having passed their unit tests may still fail to
integrate.Here comesthe needof functionaltesting. Organizations have dedicated departments of
manual-testers. They validate the product at whole build level. Detailed reports are generated,
modules’ health is monitored and bug tracking is integrated to this process.
The Present
Bulk of functional testing has always been manual. A big reason was lack of tools which can
automate applications, detect and report the failures in doing so. But the last decade saw an
exponential growththe automationsector.The authorityof pioneering products was challenged by
emerging tools. A majority of them were open source, promoting accessibility to even small
companies. Sowhenwe can automate the functional testing process, why not integrate it to the CI
server? Why not unit test and functional test every code batch? Here comes the hurdle which we
knew but overlooked at the initial stages – maintainability.
Automationscripts are traditionally codes. They are codes which test and validate other code; and
they being code, need to be designed, written, maintained and reviewed. But they have a big
advantage – theycan execute justasany othercode.That’swhyJUnit and similarframeworks are so
popular.Problemistheydon’tscale wellwith functional testing. For one thing, functional tests are
very broad, covering many workflows, scenarios, test cases and unique steps. Such big a code is
difficult to maintain and soon outgrows the product itself. Another thing is being code; they don’t
trulylie in tester’s domain. They need seasoned developers. You can have programmers write the
test scripts but then a load mismatch remains.

2. Welcome the eraof KeywordDrivenAutomatedFunctional Testing.Encapsulating the programming
processintestscripts givesmanybenefits.Firstandforemost,we nolongerneedhardprogrammers
for testing task. Any programming-literate can do the task. The Test scripts are more maintainable
and language agnostic. Now we can actually have both unit tests and functional tests which can be
triggeredoneachcode checkin. While unittestsvalidate code conformity, functional tests validate
product’s behavioral consistency. And both work unattended. Developers can get feedback
meaningful tothemfromtheirunittests.Businessuserscangetmodule healthstatusmeaningful to
them from their functional tests, all in near real time.
The Future
There is no limit to innovation. Imagine the development of a web application. Let’s say the
application needs to be supported on all major browsers, IE, FireFox, Chrome, Safari, and Opera.
Acrossall Platforms:Windows/Linux/Mac Desktops/Laptops. Let us push the limits, ask for support
on all major mobile devices: Android, iPhone, Blackberry, Windows; talk about orientations,
resolutions. Nowletsusaskthe testengineerstoautomate the whole thing.Runall regressionflows
on all Platforms/browsers on all check-ins. After all we want a strict quality control.
Difficulttoacceptbut thissimply cannotbe done!Notwithoutanautomation framework which can
do the heavylifting.Noneof toolsorframeworksinthe marketcan cope withthe requirements that
we will be facing in coming years. What the industry needs is a test automation framework which:
1. Can integrate with the existing CI solutions.
2. Performfunctional testsasseamlesslyasUnitTests.Run a testeverynight,everycheckin or
on every Run-Test request.
3. Scriptingprocessisbetterto be keyword driven than language based. We want one simple
ClickOnButton; not a bunch of find, hooks, performs, again recheck hook and so on.
4. Provides high maintainability of Scripts. Should not need programmers to modify that
validation.
5. Can gracefullyorchestrate multiple tools to perform activities which otherwise don’t come
under ambit of any one single tool. If Selenium cannot click on an image, get it done with
Sikuli.
6. Can launchmultiple configurationsof asame testinparallel. We want to take the same test
on all five major browsers.
7. Gives a uniform interface over heterogeneous tools. Tools may vary in language support
evenonplatformsupportbutwe needit.We needto testour applicationon both Linux and
Windows.
8. Generates business oriented reports. We want reports in terms of Business flows not
individualtestcases.We wanthealthcheckof modules. We want basic metrics. We want to
visualize defects over last five builds.
9. Drill down to step level failures, so that both testers and developers can get workable
information regarding the failure.
10. Integrate with major Defect management tools like Bugzilla, Jira etc. Should be capable of
automatically logging confirmed bugs.
11. Integrate with major Application Lifecycle Management software like HP ALM.

3. Now that is a whooping wish list and no product in the market claims to do it all. But this wish is
soongoingto be our requirement.Successandfailure of products will depend on timely validation
on all the target environments. Evenif we somehow manage the sheervolumeof test scripts and all
the plumbing, we are stuck at the test infrastructure. We need to test on five browsers, each on
latestversionandthree backwardversions.We needtotestthose browserson Windows, Linux and
Mac. Nowthat makesabouta hundredparallel executionsonahundredinfrastructure.How are you
goingto manage that infrastructure?Whathappenswhenyouneedabunchof Mobile devices? Are
you going to purchase it all? Rent it all? Of course not.
The future isof cloudcomputing. We will rentbrowsersoncloud.We will rentdevices on cloud. We
will rent the execution systems on cloud. Volumes test data will be stored on cloud, analyzed on
cloud systems and reported from cloud. All we need on premises is just one CI tool and one test-
automation-tool. The cloudwill provideall the rest.We will rentresourcesonpay-per-usebasic. Pay
for the tests we run. No need to inventory the hardware resources, browsers, devices and all.
And our steps to it
That future isnot so distant. Youcan get the machines fromIaaSprovideslike AmazonorRackspace.
SauceLabs can rent you browsers; Xamarin can run your tests on rented mobile devices. These
solutionsare already available. But are these true solutions to your pains? No. you need an end to
endautomationsolutionandthese isolatedinitiatives don’t help much. A service is required which
can bring all these together. A lot of tools have targeted the challenges and most have succeeded
too.But until nowit hadbeenmostly isolated,unorganized attempts. Notable efforts are from QTP
– which along with its Open Test Architecture is the big brother of the current generation of Test
Automationsolutionsandthe otherisOpKey –the nextgenerationof Test Automation Framework.
OpKeyisdetermined to solve the problems which the future automation engineers and managers
are about to face. It is designed for scalability and maintainability. The future is of multi platform
deploymentandsothe challenge lies in multi-platform simultaneous quality control. OpKey is the
only player which can tackle this challenge. Being scalable means anyone, even small scale
developerscanstart witha basicOpKeysetup andthenscale upas the applicationgrows. A time will
come when a use case will span Web, Database, Web-service and Mobile interactions. At that
momentyouwill realizethatonlyOpKey can orchestrate that many tools, platforms and resources.
OpKeyisnotjust anotherframework.OpKey is the smartest Test Automation Framework. It adapts
to your needs. It scales to your needs. It works for web, desktop and even mobile. It can
simultaneously run all your tests on all the target platforms. What is more, you do not have to
manage the entire hardware infrastructure yourself. OpKey is introducing Test as a Service
paradigm. Withthat inplace,all yourinventorywoeswillbe gone. You will trigger the tests and you
will getthe results. That is all. OpKey is the beginning of no-frills-testing. In next few articles I will
demonstrate how it meets all that it claims.