Although most testing organizations have some automation, it's usually a subset of their overall testing efforts. Typically the processes have been previously defined, and the automation team must adapt accordingly. The major issue is that test automation work and deliverables do not always fit into a defined manual testing process. Learn how to transform your manual testing procedures and how to incorporate test automation into your overall testing process.

1. Howtotransformthe
ManualTestingprocessto
incorporateTestAutomation
JimTrentadue
EnterpriseAccount Manager - Ranorex
jtrentadue@ranorex.com

2. Agenda

3. 3
Agenda
• Test activities and deliverables within a Waterfall and Agile SDLC
• Test Automation working models and frameworks in the industry
• Incorporating key aspects fromTest Automation procedures and fusing them
into standard testing processes
• Deep-dive intoTestingArtifacts: Plan & Avoid particular topics
• Additional areas of impact among the project team members
• Case Study review
• Session recap

4. Testactivities/deliverableswithinthe
AgileandWaterfallSDLC

5. 5
TestActivitieswithinanAgileSDLC
Project Initiation
Typical Agile process of activities to test deliverables
The End Game
(SystemTest)
Release Planning
Release to Prod/Support
Each Iteration
1
…
X
Get an understanding of the project
Participate in sizing stories; Create test plans
Participate in sprint planning, estimating tasks
Write and execute story tests
Pair-test with other testers, developers
Business validation (customers)
Automate new functional test cases; Run automated regression test cases
Run project load tests
Demo to the stakeholders
Release mgmt tests mock deploy on staging; Smoke test on staging
Perform load test (if needed)
Complete regression test; Participate in releases readiness
Business testers perform UAT
Participate in release to production
Participate in retrospectives
Agile Testing – Crispin & Gregory

6. 6
TestActivitieswithinawaterfallSDLC
Standard V-Model set of testing activities
Test
Strategy
Test
Scenarios
Test
Cases
Test
Scripts
Test
Results
Test
Summary
Project
Initiation
Analysis
Design
Develop
Testing
Deploy

7. 7
TestAutomationFrameworks
Record
Playback
Structured
Testing
(invokes
more
conditions)
Data Driven
Keyword
Driven
Model /
Object
Based
Actions
Based
Hybrid
(combines 2
or more of
the previous
frameworks)
Below is a list and evolution of available test automation frameworks

8. TestAutomationworkingmodelsand
frameworksintheindustry

9. 9
Keypoints withTestAutomationwithinAgile
0-10%
UI Centric
Automation
20-40%
Service or MiddleTier
Automation
50-60%
Unit levelAutomation
+80%, UI Centric Automation
0-10%
Service or MiddleTier
Automation
0-10% Unit level
Automation
Velocity Partners –
The Agile Test Automation pyramid, Mike Cohn
Agile automation
• Smaller units of
work pieced
together as
opposed to end-
to-end
• Iterative review of
the automation;
constant review of
backlog
• Automation of a
given screen even
if not complete

10. 10
MappinganATLMagainstaWaterfallSDLC
Automated Software Testing: Introduction, Management and Performance – Dustin, Rashka, Paul
Illustration of how typical
activities of an ATLM align with
SDLC activities
D. System Design & Development C. SmallTool Pilot/Prototype
A. System Life-Cycle ProcessF. Production and Maintenance
B.BusinessAnalysisandRequirements
E.IntegrationandTest

11. FusingaTestAutomationprocessinto
anexistingTestingProcess

12. 12
TestAutomation-Agilefusion
Agile Testing –
Crispin & Gregory
Questions facing each of the quadrants
• How can it be
coded test-first?
• Do we know how
to unit test our
presentation
layer; do we need
a tool for that?
• How are we going
to prototype?
• What tool will we
use to create
executable
business-facing
tests to guide
development?
• Do we have
regression tests
that will need
updating?
• Requires more
advance planning
• Will need to track
the users’ activities
for further analysis
• Load scripts might
use the old UI; time
has to be budgeted
to update these for
the new one

13. 13
TestAutomation-Waterfallfusion
Outline the scope of automation in the ReleaseTest Plan
Divide test scenarios:Automated vs. Manual
Prepare the automated test workflow
with error-handling
(new and regression)
Record your modules,
ensuring all objects have
been recognized
Replay until test
executes as
expected
Check-in
CM
Planning test automation activities into your testing process
Project
Initiation
Test
Strategy
Analysis
Design
Develop
Testing
Deploy
Test
Scenarios
Test
Cases
Test
Scripts
Test
Results
Test
Summary
Test Automation activities:

14. Deep-diveintoTestingArtifacts:
Plan&Avoidparticulartopics

15. 15
TestStrategy/TestPlan
Not listing automation in
the MasterTest Plan for
accomplishing the testing
on schedule
Not documenting all of
the related variables for
automation, such as
resources, environments,
data, controls & licensing
Omitting a
section for
automation That the tool selection
preparation has been
done on the technologies
being used and how team
members will use this
To include automation
(developing or executing)
in every iteration or build,
as part of the overall test
approach in the project
To obtain
stakeholder
buy-in for
automation
initiatives

16. 16
TestScenarios/TestStories
Writing the scenarios or
stories at a level that
does not allow for
automation as a solution
Trying to automate those
areas that focus on
usability, reliability,
maintainability or other
abstract areas
Topics that
can’t be
automated To define the ‘how’ along
with ‘what’ for your given
topic to ensure it gives
clear understanding on
responsibility
A second review of the
topic if it’s tagged for
automation to ensure
more precise wording for
automation
To parse
automation
candidates

17. 17
TestCases/TestScripts
Assuming that previously
written manual test cases
can be automated, thus
planning the automation
until later in the schedule
Delaying the inclusion of
automation development
and execution at the start
of the schedule or sprint
Automation
as a final
activity Frequent component-
level runs with Agile
development, designed
as automated test cases
from the onset
End-to-end regression
tests runs for those tests
that are applicable to the
changes deployed
Continuous
Integration

18. 18
TestSummary/Retrospectives
Considering the
automated tests as a
simple test case artifact
as opposed to a
development activity
Losing the opportunity
on scheduling a window
for test maintenance on
the automation outside
of project activities
Not
reworking
automation
failures
Automation metric
reviews on each iteration
and sprint to show the
time savings and velocity
for each tester involved
On incorporating the
manual tests that have
been completed into the
regression test library
either in a maintenance
window or during a sprint
On
improving
automation
at the next
chance

19. Additionalareasofimpactamong
projectteammembers

20. 20
Projectteammembersimpact
Project Manager
• Allocates the
right resources
and schedules
accordingly
• Accounts for
budget planning
for licenses,
environments or
head count
• Works withTest
Lead on scope
of effort and
separating the
manual from
automated tests
Business Analysts
• Formulates a
requirements
process for
selecting the
right solution
• Publishes
business rules to
consider in the
automation
architecture
• Serves as
conduit to UAT
for automating
business process
Development
• Codes any test
functions that a
tester may
request
• Documents
standards for
control names
and objects
• Assists in any
debugging of
the test cases
SystemTesters
• Supplies altered
manual test
cases to be used
as a guide for
automating
• Sets direction
on testing types
for manual vs.
automated
scenarios
• Guides test
automation
team creating
end-to-end test
scenarios
DBA’s
• Connects data
source (RDBMS)
to automated
solution
• Builds related
SQL queries for
use in data-
driven testing
• Maintains data
integrity and
database
optimization
processes
IT Management
• Champions
initiative with
analysts driving
the effort
• Provides capital
support for
expenditure:
(licenses,
training,
resources)
• Reviews and
invests in the
initiative as the
business case
warrants
Looking at the different roles of who can assist withTest Automation

21. CaseStudyreview

22. 22
CaseStudy:Background&Solution
Titled: Automation Through the Back Door (By Supporting Manual Testing)
Experiences of Test Automation – Graham & Fewster
Background:
To improve the rate of test automation in the
organization, modifications were made to the test
automation framework to support manual testing.
Technical Solution:
Develop a framework that is based off of keyword-
driven testing called command-driven testing.
What is Command-Driven Testing?
• Uses keywords that are simple commands
(SELECT, BUTTON)
• Interpreter scripts are the same for all products
• Using the advantages that come from Data-
Driven testing, navigation was placed into a
DRIVER-File; data in a DATA-File
• These two files built together would form a
command script
• The script-runner reads sequentially the
commands in the DRIVER-File. DATA-Codes are
substituted with data from the DATA-File
With Command-Driven testing:
• Testers don’t necessarily need to learn tool scripting
• The separation in navigation and data sections makes the command scripts
flexible and reusable
• DRIVER-Files can be easily ported to different applications
• DRIVER-Files need not be changed on migrating to another tool
• The test tool is needed only to prepare the templates and to run the tests

23. 23
CaseStudy:Processforimplementation
Titled: Automation Through the Back Door (By Supporting Manual Testing)
Experiences of Test Automation – Graham & Fewster
Prerequisites:
1. You cannot start if you don’t
know the application and the
test cases that are to be
automated
2. You need a working engine
that can interpret all the
commands you are going to
need for your test cases
3. You must have registered all
the GUI elements that will be
used in test execution in the
proprietary mapping of the
deployed capture/replay tools
in order to normalize the
names of the GUI controls
Step 1
Record the test case with the
capture functionality of the test
automation solution
Step 2
Translate
generated
script to
command-
scripts
Step 3
Developing plannedTC’s
from template and build
test suites

24. 24
CaseStudy:KeyPoints
• Due to limited testing resources for test automation, significant effort still had
to be spent on regression testing. However, the command-driven framework
was adopted for manual testing as well as automated testing
• This approach helped limit the number of times a tester would simultaneously
work on the same template, which was a current weakness
• Defect reporting became much easier. It avoided the testing team having to
repeat the same steps to recreate the defect
• Continuous reviews were done to assess what features were available and what
was needed to support manual testing
• Implementation included a feature that supports the execution of partially
automated tests, that could aid with tedious test preparation tasks
• Manual tests focused on customer-specific conditions now instead
Experiences of Test Automation – Graham & Fewster

25. SessionRecap

26. 26
Recapofthepresentation
Reviewing the main points of the presentation
• Outline your testing process by phase and deliverable, with consideration on
where automation would apply
• Investigate the various test automation processes and frameworks available in
the industry to determine what’s most suitable to your organization or initiative
• Fuse test automation topics into each testing phase or milestone accompanied
with a deliverable
• Delve deeper into your testing artifacts to incorporate automation through
planned activities and avoidance areas
• Include other project team members in the automation initiative to contribute
in their area of expertise
• Present a real-life case study to management, to emulate good examples or to
ensure this does not occur during your research and implementation

27. Thankyouforattending!
JimTrentadue
EnterpriseAccount Manager- Ranorex
jtrentadue@ranorex.com