1. Automated
Tes,ng
for
Mobile
Applica,ons
An
experimental
technique
Eing
Ong,
Intuit
Inc.
Louis
daRosa,
Intuit
Inc.

2. Session
outline
•
Introduc,on
•
Simulator
basics
•
Simulator
library
•
Building
your
mobile
app
library
•
Wri,ng
reusable
test
cases
•
Mobile
test
automa,on
framework
•
Advantages
and
limita,ons
•
Q
&
A
2

3. Introduc,on
What
are
we
solving
for
?
•
Various
BB
OS
and
devices
•
End-­‐to-­‐end
mobile
tests
•
Low
cost
solu,on
•
Mul,ple
mobile
plaLorms
3

4. Simulator
Basics
BlackBerry®
JDE
tool
kit
• Mobile
data
service
(MDS)
• Simulators
suite
• App
installa,on
• Image
capture
4

5. Simulator
Basics
BlackBerry®
simulator
• Star<ng
simulator
fledge.exe
/app=jvm.dll
/session=<model>
/app-­‐param=
JvmAlxConfigFile:<model>.xml
/handheld=<model>
• Communica<ng
with
simulator
fledgecontroller.exe
/session=<model>
5

6. Simulator
Basics
Ac<ons
Steps
Start
9630
Tour
fledge.exe
/app=jvm.dll
simulator
/session=9630
/handheld=9630
/app-­‐param=JvmAlxConfigFile:
9630.xml
Install
1.
Copy
app.jar,
app.jad,
app.cod
to
applica,on
Javaloader
directory
2.
JavaLoader.exe
–u
load
app.jad
3.
Delete
app.jar,
app.jad,
app.cod
Save
screenshot
1.
JavaLoader.exe
–u
screenshot
as
test.png
in
test.png
$TEST_OUTPUT
2.
mv
test.png
$TEST_OUTPUT
6

7. Simulator
Basics
Ac<ons
Steps
bblib.py
Start
9630
Tour
fledge.exe
/app=jvm.dll
fledgeStart()
simulator
/session=9630
/handheld=9630
/app-­‐param=JvmAlxConfigFile:
9630.xml
Install
1.
Copy
app.jar,
app.jad,
app.cod
to
install()
applica,on
Javaloader
directory
2.
JavaLoader.exe
–u
load
app.jad
3.
Delete
app.jar,
app.jad,
app.cod
Save
screenshot
1.
JavaLoader.exe
–u
screenshot
getScreenShot(‘test’)
as
test.png
in
test.png
$TEST_OUTPUT
2.
mv
test.png
$TEST_OUTPUT
7

8. Simulator
commands
Ac<on
Steps
Enter
'Hello
StringInjec,on(Hello)
World'
KeyPress(SPACE)
KeyRelease(SPACE)
StringInjec,on(World)
Backspace
2
KeyPress(BACKSPACE)
,mes
KeyRelease(BACKSPACE)
KeyPress(BACKSPACE)
KeyRelease(BACKSPACE)
Thumbwheel
up
ThumbWheelRoll(-­‐1)
twice
ThumbWheelRoll(-­‐1)
8

9. Simulator
commands
Ac<on
Steps
bblib.py
Enter
'Hello
StringInjec,on(Hello)
enter(‘Hello
World')
World'
KeyPress(SPACE)
KeyRelease(SPACE)
StringInjec,on(World)
Backspace
2
KeyPress(BACKSPACE)
backspaces(2)
,mes
KeyRelease(BACKSPACE)
KeyPress(BACKSPACE)
KeyRelease(BACKSPACE)
Thumbwheel
up
ThumbWheelRoll(-­‐1)
thumbwheel
('up',
twice
ThumbWheelRoll(-­‐1)
2)
9

10. Simulator
commands
Ac<on
Steps
bblib.py
Touch
screen
at
TouchScreenPress(10,
100,
0)
touch(10,
100)
(10,
100)
TouchScreenClick()
TouchScreenUnclick()
TouchScreenUnpress(0)
10

11. bblib.py
Setup
• fledgeStart()
• javaloaderStart()
• mdsStart()
Basic
key
events
• backspaces(count)
• enter()
• enter(string)
• menu()
• escape()
• pause(seconds)
11

12. bblib.py
Naviga<on
• thumbwheel('up',
count)
• thumbwheel(‘down',
count)
• trackball('up’,
count)
• trackball(‘down’,
count)
• trackball(‘lek’,
count)
• trackball(‘right’,
count)
• touch(x,y)
12

13. Case
study
–
Address
Book
Demo
• Devices
– Pearl
– Storm
• Simulator
library
• Address
book
– Add
contact
– Find
contact
– Delete
contact
• Automated
tests
13

14. Test
Automa,on
Overview
1. Define
applica<on
interface
This
interface
is
device-­‐agnos,c.
2. Implement
the
interface
Implement
interface
in
BlackBerry®
U,lize
Python™
mobile
libraries
e.g.
bblib.py.
3. Write
your
tests
Tests
are
device
independent
and
reusable.
4. Run
14

15. Step
1
:
Define
applica,on
interface
class
AppInterface:
"""Applica,on
Interface"""
def
launch(self):
"""Launch
app"""
def
add(self,
contact):
"""Add
a
contact"""
def
find(self,
contact):
"""Find
contact"""
def
delete(self):
"""Delete
current
contact"""
15

16. Test
Automa,on
Overview
1. Define
applica<on
interface
This
interface
is
device-­‐agnos,c.
2. Implement
the
interface
Implement
interface
in
BlackBerry®
U,lize
Python™
mobile
libraries
e.g.
bblib.py.
3. Write
your
tests
Tests
are
device
independent
and
reusable.
4. Run
16

17. Step
2
(Storm):
Implement
the
interface
def
StormImpl(appbase.AppInterface):
def
add(self,
contact):
touch(100,
50)
enter(contact.getFirstname()
thumbwheel('down',
1)
enter(contact.getLastname())
#
save
menu()
touch(50,
300)
17

18. Step
2
(Storm):
Implement
the
interface
def
StormImpl(appbase.AppInterface):
def
add(self,
contact):
touch(100,
50)
enter(contact.getFirstname()
thumbwheel('down',
1)
enter(contact.getLastname())
#
save
menu()
touch(50,
300)
18

19. Step
2
(Storm):
Implement
the
interface
def
StormImpl(appbase.AppInterface):
def
add(self,
contact):
touch(100,
50)
enter(contact.getFirstname()
thumbwheel('down',
1)
enter(contact.getLastname())
#
save
menu()
touch(50,
300)
19

20. Step
2
(Pearl):
Implement
the
interface
def
PearlImpl(appbase.AppInterface):
def
add(self,
contact):
enter()
enter(contact.getFirstname()
thumbwheel('down',
1)
enter(contact.getLastname())
#
save
menu()
enter()
20

21. Step
2
(Pearl):
Implement
the
interface
def
PearlImpl(appbase.AppInterface):
def
add(self,
contact):
enter()
enter(contact.getFirstname()
thumbwheel('down',
1)
enter(contact.getLastname())
#
save
menu()
enter()
21

22. Step
2
(Pearl):
Implement
the
interface
def
PearlImpl(appbase.AppInterface):
def
add(self,
contact):
enter()
enter(contact.getFirstname()
thumbwheel('down',
1)
enter(contact.getLastname())
#
save
menu()
enter()
22

23. Step
2
Recap:
Implement
the
interface
def
StormImpl(appbase.AppInterface):
def
PearlImpl(appbase.AppInterface):
def
add(self,
contact):
def
add(self,
contact):
touch(100,
50)
enter()
enter(contact.getFirstname()
enter(contact.getFirstname()
thumbwheel('down',
1)
thumbwheel('down',
1)
enter(contact.getLastname())
enter(contact.getLastname())
#
save
#
save
menu()
menu()
touch(50,
300)
enter()
23

24. Test
Automa,on
Overview
1. Define
applica<on
interface
This
interface
is
device-­‐agnos,c.
2. Implement
the
interface
Implement
interface
in
BlackBerry®
U,lize
Python™
mobile
libraries
e.g.
bblib.py.
3. Write
your
tests
Tests
are
device
independent
and
reusable.
4. Run
24

25. Step
3
:
Wri,ng
tests
class
AddContactTest(uniqest.TestCase):
device
=
testenv.getDeviceClass()
def
addContactWithOnlyFirstnameTest(self):
self.contact.setFirstname(firstname)
self.device.add(self.contact)
def
addContactWithOnlyLastnameTest(self):
self.contact.setLastname(firstname)
self.device.add(self.contact)
25

26. Step
3
:
Run,me
device
binding
def getDeviceClass(self):
""" Returns the device to test """
mobileDevice = self.getMobileDevice()
if mobileDevice == 'pearl':
import pearl
deviceClass = pearl.PearlImpl()
elif mobileDevice == 'storm':
import storm
deviceClass = storm.StormImpl()
else:
import bb
deviceClass = bb.BlackBerry()
return deviceClass
26

27. More
device-­‐independent
tests
Addi<onal
tests
are
easy
to
write
def
addContactWithEmailTest(self):
def
addContactWithAddressesTest(self):
def
addContactWithAllDetailsTest(self):
def
addContactWithLongDetailsTest(self):
def
addContactAddressWithStateZip(self):
def
addContactAddressWithCityStateZip(self):
def
addContactAddressWithNoDataNega,veTest(self):
27

28. Step
4
:
Run
• Basic
run
command
– python
<test.py>
• Python™
test
frameworks
– uniqest
– PyUnit
– python-­‐nose
28

29. Test
Verifica,on
• Server
hosted
apps
– API
asser,ons
– Database
asser,ons
• Image
asser,ons
self.assertTrue(imagelib.compare(self.device,
testname,
'100%x90%‘))
imagelib.py
:
def
compare(device,
imageName,
crop=None,
tolerance=500)
29

30. Logging
• Ini,aliza,on
self.log
=
self.device.initLogger(self._testMethodName,
self.__class__.__name__)
• Usage
self.log.info('Star,ng
test:
'
+
self._testMethodName)
self.log.debug(self.contact)
self.log.error(‘Missing
image
to
compare’)
• Logs
AddressTest.log
:
2010-­‐06-­‐10
15:19:46,773
-­‐
testCreateAddressMethod
-­‐
INFO
-­‐
[Address1]
200
Villa
St
Mountain
View
CA
94040
BUSINESS
ADDRESS
30

31. Login
example
on
BlackBerry®
Pearl
Interface
def
login(username,
passwd)
Test (logintest.py)
login(‘user100’,’helloworld’)
enter(username)
def
enter(str)
scroll(‘down’)
def
scroll(ac<on)
enter(password)
def
enter()
enter()
pearlapp.py bblib.py
31

32. Login
example
on
BlackBerry®
Storm
Interface
def
login(username,
passwd)
Test (logintest.py)
login(‘user100’,’helloworld’)
touch(100,100)
enter(username)
def
enter()
touch(100,200)
def
touch(x,y)
enter(password)
touch(150,
300)
stormapp.py bblib.py
32

33. Mobile
Tes,ng
Framework
Mobile
Applica<on
Interface
Python™ Test Framework
Device
Independent
Tests
Runtime device binding
Simulator libraries
BB
app
library
Mobile
app
library
BB
device
library
Device
library
33

34. Advantages
• Zero
cost
to
use
• No
device
sharing/scheduling
• Bigger
device
pool
• Reduce
manual
tes,ng
,me
• Run
on
developers
machines
• Debugging
capabili,es
34

35. Limita,ons
• Requires
ethernet
or
internet
connec,vity
• Does
not
simulate
network
performance
• Does
not
support
hardware
controls
tes,ng
• Dependent
on
simulator
reliability
• Limited
peer-­‐to-­‐peer
applica,ons
tes,ng
35

36. Resources
BlackBerry
®
JDE
Downloads
hqp://na.blackberry.com/eng/developers/javaappdev/javadevenv.jsp
Documenta<on
Fledge
controller
hqp://docs.blackberry.com/en/developers/deliverables/15476/Test_B
BSmrtphnSmltr_events_using_script_607587_11.jsp
BlackBerry
®
Simulator
Tes,ng
hqp://docs.blackberry.com/en/developers/deliverables/6338/Tes,ng_apps_using_the
_BBSmrtphnSmltr_607559_11.jsp
Python
™
hqp://docs.python.org/
BlackBerry
® Developers
Forum
hqp://supporLorums.blackberry.com/
36

37. Acknowledgments
• Desiree
Gosby,
Mobile
Architect
• Paul
Hau,
QA
Manager
• Jaron
Jones,
QA
Manager
• Jason
Pugh,
Architect
(definitely
not
least)
37

38. Q
&
A
38

39. Thanks
and
enjoy
the
rest
of
DevCon
2010!
For
more
details
on
the
presenta,on,
contact
devcon@intuit.com
39