This is the presentation Mobtest's CTO and founder Dirk de Kok gave at the Renaissance iOS app makers' conference. He talked about user testing iOS apps, including concept testing, usability testing, beta testing, QA testing. See also https://github.com/mobtest/renaissance.io/

1. Human Testing iOS apps
Dirk de Kok
dirk@mobtest.com
twitter.com/dirkdk
Renaissance conference
Jan 23rd 2013
© Mobtest 2013

2. App quality
‣ works technically as intended
‣ offers considerable value
‣ intuitive and easy to use

3. Human testing
Manual QA testing
{
Concept testing
Usability testing
User testing
Beta testing
Live testing

4. Test to create better apps
not just bug free apps

5. Test early, before launch
‣ if high quality, app might get featured
‣ initial ratings drive subsequent downloads
‣ reaching a top 25 normally happens
around launch
‣ critical updates take still 1-2 weeks
because of Apple review process

6. Concept testing
Goal
validate app idea with target market
How
‣ find respondents
‣ do interview with factual questions
‣ show mockup or prototype

9. Usability testing
Goal
test how easy it is to use your app
How
‣ in person
‣ remote

10. In person usability testing
‣ meet testers in office or coffee shop
‣ give tasks with only end result
‣ let user think out loud
‣ observe usage, facial expression, posture

12. Record usability test
‣ easier to remember
‣ easier to share with your team
can’t do it on iOS device itself

14. Via Skype DIY camera

15. Remote usability testing
‣ recruit larger group of users
‣ distribute ad hoc build
‣ give tasks with only end result
‣ track task completion in app and
ask questions afterwards by survey

16. What to look for
‣ does it work as user expects?
‣ discoverability
‣ areas of confusion
‣ positive experience

17. Beta testing
Goals
‣ final feedback from users
‣ integration test with real data
‣ get early adopters enthusiastic about
app (reviews and ratings!)

18. Beta testing
How
‣ Recruit testers via own network or service
‣ Get their UDID, create ad hoc build and
distribute
‣ Reward feedback

19. QA testing

21. QA testing
Goals
‣ make sure app works technically fine
How
‣ get a good test engineer
‣ write a detailed plan
‣ execute plan

22. QA methods
‣ structured testing with predefined test
scripts with expected outcomes
‣ “go break it” exploratory testing

23. What to look for
‣ Hardware models x iOS versions
‣ Connectivity (wifi, cell, intermittent)
‣ Gestures, moves, tilting and multitouch
‣ Location/geofencing

24. Live testing
‣ A/B testing
‣ Monitoring usage patterns
‣ Crash log analysis
‣ Feedback options for user

27. Thank you
http://github.com/mobtest/
dirk@mobtest.com