Eran Kinsbruner session on mobile test automation and optimizing the testing within the DevOps Pipeline from Quest 2018 conference

1. Optimize Your Mobile Test Automation
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Eran Kinsbruner
@ek121268, http://continuoustesting.blog

2. About Me
• Lead Technical Evangelist at Perfecto
• Blogger and Speaker
• http://continuoustesting.blog
• https://www.infoworld.com/author/Eran-
Kinsbruner/
• 18+ Years in Development & Testing
• Author of The Digital Quality Handbook
Weekly Podcast - Testiumpod
To get your copy of the book, come by the Perfecto Booth

3. Agenda
• DevOps Transformation & Trends
• 3 Topics on Optimizing Mobile Test Automation
• The Device/OS/Test Debate
• Optimization Techniques
• RTDD for Test Automation Optimization
• Mobile Testing Landscape
• Q & A
+

4. DevOps == Continuous Everything
5/29/2018 4© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Credit: TestingWhiz

5. Digital is breaking new grounds
Drop of 32% in branch visits in last 4 years
Chatbots as the new Digital
Interface - everywhere
$380B industry by 2024 (26% CAGR)
Face ID Authentication
AR As a Digital Engagement
Enabler

6. The Rise of PWAs – Challenging Native Mobile Apps
A Progressive Web App is
• Progressive - Works for every user
• Responsive - Fits any form factor
• Connectivity independent
• App-like - Feels like an app
• Fresh - Always up-to-date
• Safe - Served via HTTPS
• Discoverable -allowing search engines to find it.
• Re-engageable -Support push notifications.
• Installable
• Linkable - does not require complex installation (store
access)
5/29/2018 6© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Source :medium.com

7. Optimizing Test Automation
© 2015, Perfecto Mobile Ltd. All Rights Reserved.

8. 3 Layers Involved In Test Automation Optimization
5/29/2018 8© 2016, Perfecto Mobile Ltd. All Rights Reserved.
App
• Info.Plist (iOS)
• Manifest.MF
(Android)
Test Cases
• Define Desired Capabilities
Device
• Supports Feature
• Doesn’t Support
• Resource
Consumption

9. Generic Login Into Apps on iOS - Testing Diversity
Test Scenario Apple iPhone X Apple iPhone
7/8
Apple iPhone
5C
iPad Air 2/
iPad Mini 3
Apple iPad
Mini/iPad 2
Login Using a
User/Password
YES YES YES YES YES
Login Using Fingerprint NO YES NO YES NO
Login Using Face-ID YES NO NO NO NO

10. Android Doze & Multi Window Support
5/29/2018 10© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Android 6 and Above
Android N and Above

11. Test Development Strategy – Solutions
- Test Suite Planning (Easy Path) – Common Denominator
- Test code reusability using common supported methods (e.g. Login using User/PW) across all scenarios
- Unique device features tested outside of the regression on the relevant devices
- Work with your Dev team to develop Deep Links bypassing the Login Screen (Appium)
- Leverage mobile OS property files (More Complex)
- Scan and direct test code accordingly (Manifest.MF, Info.Plist)
- Lab setup based on device capabilities (divide execution files e.g. testNG according to device buckets)
5/29/2018 11© 2016, Perfecto Mobile Ltd. All Rights Reserved.

12. Automate Via Risk Based Approach
5/29/2018 12© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Source: Fred Beringer, Shutterfly

13. Validate Feature Compliance & Test Accordingly
5/29/2018 13© 2016, Perfecto Mobile Ltd. All Rights Reserved.
IOS Developer Library
INFO.PLIST

14. Appium Deep Links
5/29/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Code Sample: GIT

15. Device Cloud Selection Based on Supported Caps
5/29/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Code Sample: Perfecto Blog

16. Device Cloud Selection Based on Supported Caps
5/29/2018 16© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Code Sample: Perfecto Blog

17. Optimizing Test Automation Analysis - RTDD
© 2015, Perfecto Mobile Ltd. All Rights Reserved.

19. Tagging Isn’t New
5/29/2018 19© 2015, Perfecto Mobile Ltd. All Rights Reserved.

20. Introducing RTDD (Reporting Test Driven Design)
RTDD Is a method that enables teams to design quality with the debugging and insight in
mind (context based).
Among the key benefits of RTDD are
1) Structured quality dashboard that can be sliced and diced by various persona’s
2) Fast MTTR of defects unrelated to the test suite size
3) Reliable and much more stable test suite (less flaky tests)
4) Low risk Go/No-Go decision due to a data-driven dashboard and quality artifacts
5/29/2018 20© 2015, Perfecto Mobile Ltd. All Rights Reserved.

21. Implementation Basics
5/29/2018 21© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• Structured test authoring via SDK
• stepStart()
• stepEnd()
• testStep()
• testStop()
• Context based Tagging (“regression”)
• withJob()
http://developers.perfectomobile.com/display/PD/Java

22. Benefits of Tagging Strategy
5/29/2018 22© 2016, Perfecto Mobile Ltd. All Rights Reserved.
Logical Steps (incl. BDD support) Tags
Heat Map Based on Tags
Fast Quality Analysis/On-Demand Status
Context and Tags Drive Distribution for Analysis
For quality drill down and triaging process

23. CI Dashboard Enables Build Trending And Overall Quality History Visibility
5/29/2018 23© 2015, Perfecto Mobile Ltd. All Rights Reserved.

24. Mobile Testing Landscape
Criteria Appium Espresso XCUITests
Language any Java swift/objective-C
By Open source google Apple
App supported APK and IPA APK IPA
Code required No yes yes
Testtype Black box White box White box
Speed 8t t 2t
Setup Hard Easy Medium
CI medium easy Hard
Flakiness of test very low Low
Object Locators Xpath (external) Id (from R file) id
Used by QA Android dev* iOS dev*

25. Thank You
Eran Kinsbruner
@ek121268
erank@perfectomobile.com
http://continuoustesting.blog
© 2015, Perfecto Mobile Ltd. All Rights Reserved.