1. Addressing Mobile App Testing
Challenges
Notes from webinar by
Lee Barnes
hosted by QAI on
February 13th 2013
This presentation by Maira Bay de Souza is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License

2. Before we begin ...
Items in this font are the notes I took from what the
presenter said
Items in this font are my own comments

3. Mobile testing challenges
Testing in mobile environments has the same
challenges of any testing: you need to prioritize.
I like how he's able to have a big picture view of testing.
I've worked in several industries and in companies from large
to small. I've noticed that the problems in IT projects are
very similar no matter where you go. And the same applies
for the problems in testing projects.

4. Target test environment
When defining the target test environment, consider the
difference between the 2 main OSs:
− Android: installation of a new OS version is
optional and up to the user. There are lots of users
still with very old versions installed. There is a
wide variety of environments.
− iOS: the system auto-updates to new versions. This
means that Apple has complete control over the OS
in their devices.
He didn't mention it, but the Blackberry OS is like the
Android. I know because I have one.

5. What does this mean for testing?
This means that when you test in iOS, you know that the
application will work the same way in all devices (all hardware
versions). But the same cannot be said for Android.
If you want your app to work on Android, you have to set up
several test environments, with all the different OS versions.
That makes sense. Maybe the solution is to target just
one version of Android OS.

6. Other environment considerations

Devices can also vary in terms of:
− How fast it the battery consumed?
− Does it have a camera?
− What is the screen size?

7. Unique mobile scenarios
(while your app is running)

Incoming call

Low battery warning comes to the foreground, sound is played

Power is lost due to drained battery

An alarm from the calendar app comes to the foreground

Loss of mobile network signal

Transition between mobile networks (including roaming)

Transition from mobile network signal to wi-fi network signal

Keyboard slide

Change of screen orientation (landscape/portrait)

... and much more!

8. More on environments

Consider the:
− Type of app. If it's native, it has more/easier access
to the device (integration with camera, GPS, etc). If
it runs on a virtual machine (like Java) things may
be different.
− App usage: if it's used very frequently, it will
require more data, which means more network
demand and faster battery drain.
Wow, there's so many things to consider when testing mobile
apps!

9. Test Case selection

You don't need to write specific steps because the
devices/OS/app changes a lot. It's better to write
high-level test cases/test scenarios.

I was a bit surprised by that approach. But on one
hand, it makes sense.

10. Test environment maturity

Tools for desktop testing are vary mature: we have
tools to automate, to communicate with each other,
to control versions, to capture screenshots, etc.

That is not the case for mobile (yet).

11. Test environment options

Type of environments:
− Remote device control
− Mobile cloud

Both require jailbroken devices.
And for some devices, jailbreaking means loss of
warranty.

12. Continues in Part 2

13. Disclaimer
The notes presented here are what I understood from
what the presenter communicated. They might not be
100% accurate, as I was taking notes and listening to
the presentation at the same time.
All the information I am quoting from the presenter is
their intellectual property. I am reproducing it here
under the fair use policy, for quoting purposes only.