1. MOET
https://github.com/eing/moet
Mobile End-to-End Testing
Mobile Exploratory Testing
Eing Ong
Intuit, Inc.
@eingong|eing.ong@intuit.com
1
1

2. Mobile'automation'technologies1
• Two categories
• Instrumented technique
• Non-instrumented technique
• What is instrumentation?
• Tests are compiled with the app
• Tests are installed & launched with the app
• Source code is required and may need to be modified
• Only one app can be executed at a time
• White box approach
2

3. Advantages'of'both'techniques1
Non$instrumentation- Instrumentation-
• Device'platform'agnostic1 • Elements'can'be'accessed1
• Test'code'reuse1 • Debugging'ease1
• Test'language'&'test'harness' • Test'verification'ease1
autonomy1 • Reduce'tools'dependencies1
• Support'for1 • Support'for1
Multi>applications'testing1 Installing'application1
Custom'composite'UI'elements1 Launching'application1
Database/server'API'assertions1 Cleanup'(kill'application)1
Use'of'external'libraries1 Test'execution'on'device1
'(e.g.'hardware/image'libraries)1 Code'coverage1
3

4. Which'technique'should'I'use'?1
Non-instrumentation Instrumentation
External3
interaction-
Hardware3 Text$based3
interaction- features-
Image3based3
features-
4

5. Mobile'automation'tools1
Mobile3- Non$ Instrumentation-
OS- instrumentation-
Android1 eggPlant,'Sikuli,'MOET,' Robotium,'Calabash,'
MonkeyRunner1 MonkeyTalk1
iOS1 eggPlant,'Sikuli,'MOET1 UIAutomation,'KIF,'
iCuke,'Frank,'UISpec,'
Zucchini,'Bwoken,'
Calabash,'MonkeyTalk,'
Applecart,'iOS>driver1
5

6. MOET1
Test
• Think design Login(“user1”,”passwd1”)
o Interfaces
o Creational pattern iPhone implementation
touch(100,100)
• Think reuse enter(username)
touch(100,200)
o Device independent tests
enter(password)
• Think One touch(150, 300)
o IDE
o Test harness and language iPhone Sikuli library
void enter()
void touch(x,y)
6

7. Anatomy'of''MOET1
Mobile3Application3Interface-
Device3Independent3Tests-
Runtime binding
Simulator/Device libraries
Android3application3 iPhone3application3
implementation- implementation-
Android3MonkeyRunner3
iPhone3Sikuli3Library-
Library-
7

8. Top Ten Wish List
a.k.a.
Mobile test automation
Wall of Pain
8
8

9. Android'Wall'of'Pain1
10. There’s more to test execution
o Command with report file generation :
adb shell am instrument -w -e reportFile junit-report.xml
com.intuit.test/com.zutubi.android.junitreport.JUnitReportTestRunner
o Date-time fix if using device snapshots :
adb shell date -s `date +"%Y%m%d.%H%M%S”`
9. Where are the test results ?
o Pulling results from device :
adb pull /data/data/com.intuit.test/files/junit-report.xml junit-
report.xml
9

10. Android'Wall'of'Pain1
10. There’s more to test execution
o Command with report file generation :
adb shell am instrument -w -e reportFile junit-report.xml
com.intuit.test/com.zutubi.android.junitreport.JUnitReportTestRunner
o Date-time fix if using device snapshots :
adb shell date -s `date +"%Y%m%d.%H%M%S”`
9. Where are the test results ?
o Pulling results from device :
adb pull /data/data/com.intuit.test/files/junit-report.xml junit-
report.xml
10

11. iOS'Wall'of'Pain1
8. iOS simulator…the singleton pattern
o Multiple instances of simulator
o Headless execution
7. What test harness ?
o Test grouping, test assertions, base test class
o Results needs to be displayable in browser
6. How about a CI plugin
o Test execution
o Code coverage
11

12. iOS'Wall'of'Pain1
8. iOS simulator…the singleton pattern
o Multiple instances of simulator
o Headless execution
7. What test harness ?
o Test grouping, test assertions, base test class
o Results needs to be displayable in browser
6. How about a CI plugin
o Test execution
o Code coverage
12

13. iOS'Wall'of'Pain1
8. iOS simulator…the singleton pattern
o Headless execution
o Multiple instances of simulator
7. What test harness ?
o Test grouping, test assertions, base test class
o Results needs to be displayable in browser
6. How about a CI plugin
o Test execution
o Test results
o Code coverage
13

14. iOS'Wall'of'Pain1
5. Where’s the IDE plugin ?
o Code formatting
o API completion
o Breakpoints, debugger
o “javadoc” help
o One click test execution of test method/testsuite
14

15. All'mobile'platforms1
4. Problem ? Search ?
o Blogs, stackoverflow, developer groups for devices, etc.
o Mobile testing one-stop tips & tricks (not questions)
o Community driven effort
Tip – Remove authorization prompt
Administrative rights
Update /etc/authorization
<key>system.privilege.taskport</key>
<dict>
<key>allow-root</key>
<!-- previous value <false/> -->
<true/>
Tip – Clean cache, preferences, SQLite
~/Library/Application Support/iPhone Simulator/{SDK}/Applications/<uuid>
Library: Preferences, Caches
Documents: sqlite3 <app>.sqlitedb
15

16. All'mobile'platforms1
3. How about a mobile test report ?
o Mobile test report standard e.g. mobile-test-report.xml
• Screenshots and video recording of failed tests
• Device logs
• Device detection
2. Wireless?
o OTA app installation
o Testing via wifi and carrier networks
o Selenium grid-like test infrastructure
16

17. All'mobile'platforms1
3. How about a mobile test report?
o Mobile test report standard e.g. mobile-test-report.xml
• Screenshots and video recording of failed tests
• Device logs
• Device detection
2. Mobile testing is anything but mobile
o OTA app installation and execution
o Testing via wifi and carrier networks
o Test tool independent component
o Selenium grid-like test infrastructure
17

18. All'mobile'platforms1
1. Yet another proprietary language?
o Open sourced test harness & language for
• Instrumented tests
• Non-instrumented tests
• Hybrid apps tests
18

19. Next'steps1
Eclipse mobile Test Test Test Continuous
testing plugin development execution reporting integration
Mobile3Application3Interface-
Device3Independent3Tests-
Runtime binding
Simulator/Device libraries
Non$instrumented3 Instrumented3application3
application3implementation- implementation-
Monkey$
Sikuli- ?Android?- ?iOS?3-
Runner-
19