Amalgamation of BDD, parallel execution and mobile automation

Amalgamation of
Behavior driven
development (BDD),
parallel execution and
mobile automation
1
- Jigesh Shah
- Mradul Kapoor

2 Amalgamation of Behavior driven development, parallel execution and mobile automation All rights reserved.
Author’s profile
Mradul Kapoor
Mradul Kapoor is working in one of the leading consulting firm based in Mumbai, India. His experience,
knowledge and skills include Business Analysis, Automation and Functional Testing. Mradul has worked on
various testing tools like QTP, Selenium, Oracle Oats, Coded UI, SpecFlow, SpecLog, Appium, Microsoft team
system and various test management tools. He has over 7.5 years of experience in US Healthcare, Insurance,
eLearning, Life Science and Human and Health Services
Jigesh Shah
Jigesh Shah is working in one of the leading consulting firm based in Mumbai, India. His experience, knowledge
and skills include Business Automation and Functional Testing. Jigesh a Certified Java Programmer has
worked on various testing tools like QTP, Selenium, QC, Rally, JIRA , SpecFlow, SpecLog, Appium, Microsoft
team system and various test management tools. He has over 7.5 years of experience in US Healthcare,
Insurance, and eLearning

Preface
With the rise of Mobile first approaches, a substantial traffic has now shifted from the desktops to the
Mobile phones. To ensure quality in mobile apps, Testing needs to be continuous and inclusive. This makes
Mobile Test Automation a formidable business driver. Along with that there needs to be strong collaboration
between developers, testers and non-technical or business participants in a software project to maintain
application consistency across multiple platforms.
The intent of this presentation will be to provide a comprehensive mobile automation testing strategy using
Behavior Driven Development, optimized selection of target devices, and an effective combination of
manual and automated testing; use of correct tools to cover appropriate functional testing essential for
getting your mobile applications to market on time and within budget.
This presentation will also discuss on how the scenarios which are derived from the requirement can be
automated using BDD approach across Web and Mobile. Also, it will highlight on how parallel execution
can be achieved using BDD & Mobile Automation.
Target audience : Senior Leadership, Head Testing Practice, Head Quality Assurance, Delivery Head, Test Managers, Test Leads and Test Professionals

Current market analysis and trends
With the fantastic growth of mobile computing platforms there has been a dramatic increase in the value
of mobile applications for most companies.
One of the biggest challenges that one faces when developing a mobile application is how to test it.
Period Samsung Apple Lenovo Huawei Others
Q1 2015 24.60% 18.30% 5.60% 5.20% 46.30%
Q1 2014 30.70% 15.20% 6.60% 4.70% 42.80%
Q1 2013 31.50% 16.90% 4.70% 4.20% 42.70%
Q1 2012 28.90% 22.90% 5.00% 3.40% 39.80%
Period Android iOS
Windows
Phone
BlackBerry
OS
Others
Q1 2015 78.00% 18.30% 2.70% 0.30% 0.70%
Q1 2014 81.20% 15.20% 2.50% 0.50% 0.70%
Q1 2013 75.50% 16.90% 3.20% 2.90% 1.50%
Q1 2012 59.20% 22.90% 2.00% 6.30% 9.50%
Source : IDC May 2015

What are the testing challenges faced by today’s organizations while
automating mobile and web automation?
Mobile automation challenges
Outcome
Parallel execution
Rapid rollout schedule
• Rapid rollouts of bug fixes
• Constant expectation to meet
Application quality
Device fragmentation
• Issues with multiple Devices and
Browsers
• Object Identification support issues
Mobile lab
• Difficult to test in the field
• No VPN on old devices
• How to push updates and test them?
Mobileautomation
Behaviordrivendevelopment
Communication issues
• Issues with understanding
the requirements
• Each team has developed
their own understanding
mechanism, making
communication ineffective
Testing challenges
• Identify devices to test
on to support your user
base
• Identify device limitations
and characteristics
Right tools & approach
• Applications supported
• Lack of Frameworks
• Low Automation expertise
Lack of documentation
• Documentation is
neglected at times
• Document not inline with
the code changes
• Delays in
timely decisions
• Lack of proper
communication
channel
• No correct ROI of
automation
• High script
maintenance
• Low resource moral
• No correct channel
is followed for
coding standards

Common
approaches/options
Description Benefits Challenges
Web automation,
manual on mobile
• All manual don’t automates • No additional cost for transition or
to build Mobile Automation
Framework
• No need for Mobile Lab Setup
• No additional
learning required
• Does not address underlying issues
• Risk of credibility and brand value
• Loss of revenue and business
Web and mobile
automation, no
parallel execution
• Automate Web and Mobile
Application without Continuous
Integration and Parallel
Execution
• Efficient Testing
• Upgradation and Reusability
• User Environment Simulation
• Reduces Human Error
• May result into code redundancy for
mobile and web scripts
• Higher transition cost
• mobile technologies changes too fast –
automation need continuous support.
• May result into inconsistency
Recommended
approach
Description Benefits Challenges
Automated
customer centric
parallel execution
• Automate Web and Mobile
Application including Customer
Centric Solution, Continuous
Integration and Parallel
Execution
• Thinking through solutions,
before starting to code
• Documentation of Assumptions
and Specifications
• Addresses Device Fragmentation
issue
• Remote and cloud automation
• Test consistently. Each test is
executed in the same way
• Active involvement from Business and
Technical Team
• Resistance to change
• Mobile technologies changes too fast –
automation need continuous support
Here are some key options…
Mitigation strategies

Approach to overcome
the issues and
challenges

Architecture and design
Object repository
• Captures the unique
identifier to handle the
object
Configuration files
• Stores the environment
and other global
variables to determine
the config values
Data repository
• Handles the Application
under Test data and
dynamic data
Controller scripts
Function Libraries
• Reusable page/module
level functions
Reporting module
• Reports and Logs
Generation
Recovery scripts
• Contains the logic to
handle exceptions
Common utilities
• Reusable functions for
Native and Web Apps
Test scripts
• Contains the actual Test
Execution Steps
Scenarios
• Given
Set of Preconditions
• When
Set of Test Condition
Mobile and web applications
• Then
Set of Expected Results
BDDWeb and mobile automation

Continuous integration for mobile and web
automation
Execute
Automated
Scripts
Create mobile project for testing :
QA creates the scripts and
submits it to the Code
Repository
Tester
Stakeholders
Execution on node machines
Hub sends the tests to each
node and when it is executed
the result is sent back to the Hub
machine
Source code
repository
CI Servers
Hub machine
Run deployment scripts on hub
CI Server compiles and builds scripts
and sends it to the Hub Machine
Continuous reporting
The Automation Test Execution results
are sent to QAs and Stakeholders
Send test results
The Hub Machine sends the
consolidated result back to the CI
Server

These elements can serve as a foundation to build an efficient solution
Business case study
Category Impact
Test Efficiency
• Ran the Test Suite which comprised of 20 Test Scripts on Mobile and Web Application
concurrently.
• Out of these 80 Test scripts(20* 4) , 96.25 % scripts passed
Test Coverage
• Developed automation test suite comprising of 20 test scripts covering 80% of application screens
on both Mobile and Web
• Also covered 80% of the gestures supported by the Mobile Automation Tool
Effort Saving and
Flexibility
• Test Execution time for Manual Test Cases is around 20 Test Cases * 4 Devices * 15 min =
Approx. 1200 min/Cycle =20Hr/cycle
• Test Execution time for Automation Test Cases is around 20 Test Cases * 4 Devices * 8 min =
Approx. 640 min/Cycle =11Hr/cycle.
• Executing parallel on 4 nodes will again reduce the time by 11 / 4 = ~ 3 hours
Enhanced Validation
• Defects are detected early whenever a new build is deployed
• Recovery scenarios helps the Test Cases to proceed ahead in case of any unwanted failures
• Details trace of the test is also provided for analysis which can help in concluding an issue.
Reusability and
Extensibility
• Using BDD Components, the same scenario can reused across multiple device with minimal
effort.
• The Web scripts that were used to automate the Web Application have been extended to
incorporate the code for Mobiles

Benefits/value delivered
• Reduced coding & maintenance efforts as the tests runs on multiple platforms
• Reaping the rewards in the future as we can add more test cases or update existing application
flows.
Common Code
Base
• The Mobile Device Lab mandates a particular process for adding devices to the lab. And also
makes the lab easily scalable. Common processes can be set for tasks such as resetting
installing/uninstalling software on the devices, software or OS upgrade, etc.
• It helps in drastically reducing the time required to get the test results
Framework and
Programming
scalability
• Everyone in the chain knows exactly what test cases are being executed and the test data being
used, as the test cases are written in Gherkin (Given-When-Then) format which are easily
readable and can be understood by everyone involved in the SDLC
• QAs can just pick these tests and plug them into the framework saving the time & effort of re-
writing automation scripts.
Automation
Execution
transparency to
Customer
• Testing results are more reliable since we test on devices.
• Issues can be reported in the software encountered for low bandwidth, security, performance, etc.
Testing on
Real Devices
• Automation runs on real devices can be kicked off immediately without any manual intervention or
waiting for the QA team to start the process.
• Parallel execution of test suites can be easily configured and fired under CI tool
Continuous
Testing

There are four phases to implement BDD with mobile automation
Implementation approach
Feasibility Study and POC
• Benefits of automation in this project?
• Identifying parts that can be automated on mobile and integrate
with web
Pilot
• Implement the POC along with BDD
• Identify challenges and scope of optimization
Implementation
• Implement the framework along with parallel execution
• Integrate mobile and web automation effectively
• Add BDD along with required traceability
Rollout and Optimize
• Rollout the implementation across project
• Optimize the common components based
on learning from different projects
• Rollout the updated implementation

Best practices and
key takeaways

Lessons learned and best practices
Shift focus from Tests to Behavior :
It helps to describe behaviors in a single notation which is directly accessible to domain experts, testers and
developers, so as to improve communication
Invest in Continuous Integration :
Having an automated job that will build and deploy applications on the mobile, trigger automation runs, email
notifications, etc. would really help in achieving a 24/7 service.
Invest in Mobile Labs :
Testing on real devices helps to uncover some critical bugs and failures and also helps to determine the
actual behavior of the devices in various conditions
Use Page Object Patterns :
It helps the operations and flows in the UI to be separated from verification. This concept makes the code
cleaner and easy to understand.
Develop Strong Test Recovery :
Develop agents that can restart test components on devices. Enhance test case recovery, capability to
restart test case.

For organizations facing the challenges related to merging of multiple technologies and platforms, the
customer centric solution amalgamated with parallel execution and Mobile Automation will help to address
these needs.
If implanted properly it can help Organizations to achieve
• Collaboration between Business stakeholders, Business Analysts, QA team and Developers
• Multilingual Test Scenarios (35+ Languages)
• Right tools and the right approach
• Remote and Cloud Execution
• Parallel Executions
• Framework and Programming scalability
• Continuous Integration
Key takeaways

• IDC : IDC May 2015
• Jigesh Shah & Mradul Kapoor : “Traceability and Automation using BDD”
• Cucumber : https ://cucumber.io/
• Parallel and Web Automation : http ://www.seleniumhq.org/
• Mobile Automation : http ://appium.io/
References

Amalgamation of BDD, parallel execution and mobile automation

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Amalgamation of BDD, parallel execution and mobile automation

Ähnlich wie Amalgamation of BDD, parallel execution and mobile automation (20)

Mehr von Agile Testing Alliance

Mehr von Agile Testing Alliance (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Amalgamation of BDD, parallel execution and mobile automation

Hinweis der Redaktion