Find out what testing works for your mobile app.
Agile Software Development means we want to maximise progress while minimising waste. Delays cause waste, for instance wasted time and efforts; ineffective work causes waste; poor quality causes waste; and bugs cause waste and delay progress, etc.
Mobile apps and the mobile app ecosystem help determine what sorts of testing will be more valuable for the project. This workshop introduces various key concepts and factors related to testing mobile apps effectively. You will have the opportunity to practice testing mobile apps during the workshop to help reinforce your learning and discovery.
We will cover both interactive and automated testing of mobile apps, and find ways to reduce the Time To Useful Feedback (TTUF) so the project team can make more progress while reducing project waste. We will also cover various ways to gather more and better information about the qualities of our mobile codebase and of the quality of the apps-in-use.
Bring your mobile apps and mobile devices and be prepared to get involved in testing!
More details: http://confengine.com/agile-pune-2014/proposal/861/agile-mobile-testing
Conference: http://pune.agileindia.org/
Leading Mobile App Development Companies in India (2).pdf
Agile Mobile Testing Workshop
1. AGILE MOBILE
TESTING
WORKSHOP
PUNE AGILE CONFERENCE
JULIAN HARTY
Creative Commons License
How to design your mobile apps by Julian Harty is licensed under
a Creative Commons Attribution-ShareAlike 3.0 Unported License.
http://creativecommons.org/licenses/by-sa/3.0/deed.en_US
Contact me: Rev: 22 Nov 2014 julianharty@gmail.com
7. WHERE ARE THE
LATENCIES?
• Build times
• Affects end-to-end Unit Test runtime
• Commissioning run-time environment
• Automated tests
• Deployment
• Installing the app so it can be tested
• App Store Approval
• Feedback from the market
• Feedback from the field
• App Qualities
• Failures & Defects in use
9. TBS
T
B S
T = Testing
B = Bug reporting
S = Setup
We want to maximize T and minimize B & S
10. MINIMIZE SETUP
Email or SMS URLs to phone
Have a configuration workstation with all the drivers installed
Create apps on your build server and make them available
11. MINIMIZE BUG
INVESTIGATION
• Screenshot utilities
• Learn how to access, filter and store device logs
• Good quality camera for close-up screenshots
• Write a bug report that will still be valuable when
the bug will actually be investigated
12. MAXIMIZE TESTING
Use testing heuristics
• I SLICED UP FUN (Jonathan Kohl)
• COP FLUNG GUN (Moolya)
19. WHAT IS
TESTABILITY?
The concept of designing & implementing software
so it is easier to test
• Testing can be automated
• Testing can be interactive
20. SCALES OF
TESTABILITY
easy
interfacing
transparency
challenging
transparent
opaque
There are at least 2 dimensions of Testability:
• ease of interfacing
• transparency into the state & behaviour of the software being tested.
21. DESIGNING FOR
TESTABILITY: HOOKS
Programmatic Hooks
To connect test automation easily
Consider whether to leave them in situ
22. DESIGNING FOR
TESTABILITY: VISIBILITY
“Eyes into the Soul of the machine...”
Expose internal data and state
• Makes some checks easier to confirm
• e.g. Error recovery mechanisms cleaned up the
app’s internal state
Beware:
• Non-test code might start using the data
• If so, consider formalising the access in an API
23. TESTABILITY:
LAYERING OF CODE
Already covered some aspects in the Segmented
Design topic
Ideal to be able to automate the testing of each
layer or component independently
Then testing of the composite software can focus
on testing the composite aspects
Beware of emergent behaviour
• Test the qualities: non-functional-testing (NFT)
24. TESTABILITY: SEPARATION
OF CONCERNS
Separate generic and platform-specific code
Generic code:
• Application logic: What the app does, functionality
Platform-specific code:
• User Interface
• Threading
• Calls to platform-specific APIs
25. TESTABILITY: ISOLATE
COMPLEX CODE
Try encapsulating & isolating complex code
• Provide an interface*
• Have excellent automated tests exercise it
• Warn casual developers (and testers) not to tamper
with it
• Now the rest of our code is easier to understand &
manage
In parallel consider ways to replace complex code
with simpler code
* e.g. See the Facade design pattern
28. FULL LIFECYCLE
COSTS
The initial development effort may be dwarfed by
maintenance work
There are trade-offs between reducing the cost of initial
development and the cost of maintenance work
Code that costs more to modify is undesirable. Well
designed code & good automated tests can reduce the risk
and cost of maintenance work.
Beware of premature aging of your app’s codebase!
30. NOVODA
Costs 60% more to ‘add’ test automation to
Android projects
Who’s willing to sign off on it?
Where and when does the ROI start?
31. THINGS TO CONSIDER
How long do your code bases ‘last’?
Who pays for ‘maintenance’?
Where is the expertise to maintain the code?
Active apps need ongoing nurture & investments
even if you’re not changing the functionality
32. ALTERNATIVES TO
TESTING
Testing is not the only way to obtain useful
feedback. Sometimes it’s not the best way
either.
33. COMPLEMENTING
TESTING WITH OTHER
INFORMATION SOURCES
• Crowd Sourcing
• Log Analysis & Crash Dumps
• Analytics
• In-app feedback
36. USING MOBILE
ANALYTICS
An overview of Mobile Analytics
How they can help augment our testing
37. TOPOLOGY
Data Collector
Database
Filter(s)
Analytics
WebServer
Overview of Mobile Analytics
Each step may be delayed
Business
view
Mobile Apps sending
Analytics data
38. TYPES OF EVENTS
Mobile app Analytics
Library
Analytics
Collector
1:1 App-initiated
event
m:1 App-initiated
event
Library-initiated
event
E1
E
… E4
E
Ea
L
E
Ea Analytics
L
Database
Internet
connection
39. ANALYTICAL QUESTIONS
Engineering Activity,
Benchmarking,
Testing
Trends, Defect
Reports Extrapolation
Software quality
models, bottleneck
analysis
Specification
refinement, asset
reallocation
Failure prediction
models
What’s Happened?
(Reporting)
What’s Happened?
(Alerts)
What will Happen?
(Forecasting)
How and why did it
happen?
(Factor analysis)
What is the next best
action?
(Recommendation)
What’s the best / worst
that can happen?
(Modeling / Simulation)
Information
Insight
Past Present Future