Lviv QA Automation Talk

1. Building an Automation Solution
based on Appium
Anton Sirota
QA Automation Expert
DataArt

2. About me
QA Automation Architect/TeamLead
Participated in more than 30 projects
Leader of QA Automation Community
Certified Trainer and Lector at DataArt QA School
The author of Automation courses
С#

3. Statistics

4. • Shift from QA to Quality Engineering
• IoT
• DevOps
• Big Data Testing
• Mobile Test Automation
• Open Source Tools adoption c
World Quality Report Summary

6. QA Automation statistic DataArt

7. Tools Comparison
Appium Espresso XCUITest Calabash Selendroid Robotium
iOS + - + + - -
Android + + - + + +
Script Language Almost
any
Java ObjectiveC
/Swift
Ruby,
.NET, Java
Almost Any Java
Emulator + + + + + +
Real device + + + + + +
Same test for
two platforms
+ - - - - -
Cost Free Free Free Free Free Recorder -
paid.
Community Very
Active
Google Apple Average Average Average
Speed Slow Fast Fast Medium Slow Medium

8. Where is your pain?

9. Non-accessible elements

10. Find coordinates to click and send keys
Use tesseract tool to get needed text
For non-accessible elements

11. Bash scripts to reset all data before start
Top hardware (ex. latest mac mini with SSD)
Worse hardware - less parallel runs
Get better stability on emulators

12. Additional required actions
Open system settings
Start and stop and switch application
Change System Parameters (date, location, etc.)
Read and write files
Get system notifications

13. Use additional adb commands
Run Appium sever with -relaxedSecurity parameter

14. Use additional adb commands

15. Parallel tests execution

16. Parallel tests execution

17. Using Mobile Element

18. Using custom Mobile Element

19. Using the custom Mobile Element

20. AppObject methods

21. Link

22. Mobile Automation a part of CI Process

23. 1. Build Android Application
2. Upload build to cloud or file server
3. Run your tests always under the latest build
Android

24. 1. Get 3rd party (CocoaPods, Carthage).
2. Fastlane:
2.1. Establish code signing and provisioning.
2.2. Make the build.
2.3. Upload the build on cloud or file server
2.4. Upload the build to TestFlight
3. Run your tests under the latest build
iOS

25. Tips and Tricks

26. Android Connection via wi-fi
NOTE: DEVICE HAS TO BE IN THE SAME NETWORK AS YOUR COMPUTER
26
• Connect your device via usb cable
• Get an ip address of your Android device
• Run in console: “adb tcpip 5555”
• Run in console: “adb connect your_phone_ip”
• Remove usb and run in console “adb devices” to make sure your
device is connected

27. NOTE: DEVICE HAS TO BE IN THE SAME NETWORK AS YOUR COMPUTER
27
iPhone connection via wi-fi
• Connect your device via usb cable
• Open your project in xcode
• Open devices list and find your device
• Click on the checkbox : "Connect via
network“
• Once done (*You'll see a globe icon in
front of device), remove the usb

28. DEMO

29. Case Study
Application to scan QR Codes
Challenges
• Scanner supposed to be connected to a real device and use it’s only one port
• Smoke suite needs to be automated
Solution based on Appium
• Connect real android device using Wi-Fi and run tests as usual
29

30. Case Study
Call quality verification
Challenges
• Make a call from one phone to the other
• Pass some voice (audio file) from one phone and record it on the other side
Solution based on Appium
• Use two real devices: one that enters the phone number and the second that answers
• Use custom cable and VLC command line to record the audio from the second device
30

31. Tips and Tricks for iOS Real device
Certificates
To be able to run tests on iOS real device you need to have:
• Apple’s developer account
• Certificate installed on your Mac machine
• Provision profile for an application you want to test
• Device should be included into provision profile
31

32. Real Device vs Emulator/Simulator

33. Real devices
Advantages Disadvantages
33
Good tests run time
Easy to setup (Android)
Stable long time testing on the same
device
Available: GPS, Network, Bluetooth,
Camera, Gyroscope
The real devices are costly compared
to the emulator/simulators
Maintenance of real devices (OS
updates, reboots, etc.)

34. Emulators/Simulators
34
Advantages Disadvantages
Mostly free
Easy to setup
Easy to debug
Can run many configurations
Sometimes behavior can be different
on real device and Emulator
It is slower as compared to actual
devices
No possibility to do performance
testing
Not all functions are present

35. What’s my choice?
Real Device Testing = Real Results 35
Real Devices Emulators/Simulators
Should be used on the most projects
with Native/Hybrid applications.
Acceptable for Mobile Web
Applications testing
Acceptable during the PoC stage
Need to run under the many
configurations

36. Cloud Services

37. Cloud Services
37

38. Advantages of Cloud Services
38
• Don’t spend any time for environment setup and support
• Stable environment and test results
• On some cloud services tests run faster than locally
• Much easier and faster to setup multithreading
• You can use bunch of different configurations
• Mostly good support and help with problems solving

39. Parallel tests run
39

40. Real devices on Cloud Services
40

41. Useful Links
41
• https://appium.readthedocs.io/en/latest/en/writing-running-appium/android/android-shell/
• https://developer.android.com/studio/command-line/adb#shellcommands
• https://appiumpro.com/editions/54
• http://appium.io/docs/en/drivers/ios-xcuitest-real-devices/
• https://www.browserstack.com/app-automate

42. Q&A

43. Thank you!
Contacts:
Skype: yader-2007
E-mail: anton.sirota@qa-experts.com