Customer Visual 
How to Break your App - 
Best Practices in Mobile App Testing 
Daniel Knott | @dnlkntt 
1 29. September 2...
Who am I ? 
• Daniel Knott 
• Software Test Manager/ Mobile Tester @AOE 
• Mobile Testing since the end of 2010 
• Before ...
3 29. September 2014 
I Company Profile
Intro 
Current Status 
Mobile Challenges 
The Customer 
Mobile Specific 
Testing 
Mobile Pyramid 
Tools
5 29. September 2014 
I Current Status 
Author: Barry Caruth, https://www.flickr.com/photos/caruth/8401943928/
Current Status | Some Numbers 
2+ millions apps available 
100+ billion downloads 
80% – 90% of apps are deleted after the...
KIFSU – To Remember the important Things 
K I F S U 
Keep It Fast Simple Useable
8 29. September 2014 
I Mobile Challenges
Mobile Challenges | All known!? 
http://opensignal.com/reports/2014/android-fragmentation/
Mobile Challenges | All known!? 
http://opensignal.com/reports/2014/android-fragmentation/
Mobile Challenges | How to handle fragmentation 
Know the target group 
Group your devices, A B C 
Test only on those devi...
Mobile Challenges | How to handle fragmentation 
Group 1, Prio A 
• High End 
Devices 
• Dual/ Quad Core 
CPU 
• RAM >1024...
Mobile Challenges | More… 
Author: Davide Vizzini Limbte, http://www.flickr.com/photos/limbte/4007115435/ 
Author: Juan de...
14 29. September 2014 
I The Customer 
Author: Michael Newman, https://www.flickr.com/photos/mzn37/1226016289/
The Customer | Better know them 
Know the target group 
Will downsize testing effort 
Talk to your customer 
Do usability ...
16 29. September 2014 
I Mobile Specific Testing 
Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/956376714...
Mobile Specific Testing 
Known functional testing is still required 
• Boundary values 
• Equivalence classes 
• Testing a...
Mobile Specific Testing 
Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053 
Author: I...
Mobile Specific Testing 
Author: amateur_photo_bore, http://www.flickr.com/photos/andresv/233667770/ Author: Rain Rabbit, ...
Mobile Specific Testing 
Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4311105154/ Author: Surat Lozo...
Mobile Specific Testing 
Android 
• JouleUnit, to measure battery usage 
• Monkey, to stress test the app 
iOS 
• Energy U...
Mobile Specific Testing 
Author: Molly Holzschlag, http://www.flickr.com/photos/mollyeh11/272434559/ Author: Quinn Dombrow...
Mobile Specific Testing 
Author: minxlj, http://www.flickr.com/photos/minxlj/3270996158/ Author: Ryan B. BulaPhotography, ...
24 29. September 2014 
I Tools 
Author: OZinOH, http://www.flickr.com/photos/75905404@N00/7126147125/
Test Automation Pyramid 
Manual 
Testing 
UI Tests 
Integration Tests 
Unit Tests
Flipped Test Pyramid 
Manual Tests 
UI Tests 
E2E 
Unit Tests
Mobile Test Pyramid 
Unit Tests 
Beta Tests 
End2End Tests 
(UI Tests) 
Manual Tests
Tools 
Do test automation! 
Don’t buy huge test automation tools 
Start simple and discover different tools 
No tool is pe...
Tools | Android 
Robotium (ExtSolo) 
Spoon 
Selendroid 
Calabash for Android 
Robolectric 
Espresso 
Monkey 
MonkeyTalk 
A...
Tools | iOS 
UIAutomation 
Keep It Functional 
MonkeyTalk 
Calabash for iOS 
ios-driver 
Kiwi 
XCTest 
Appium
31 29. September 2014 
I Security 
Author: Alexandre Dulaunoy, http://www.flickr.com/photos/adulau/7712545428/
Security 
Author: Anthony Catalano, http://www.flickr.com/photos/badwsky/34164244/ 
Author: Dennis Wong, http://www.flickr...
Security 
Security testing is not easy 
Get help from experts 
Think twice about app permissions 
Check the OWASP mobile s...
34 29. September 2014 
I Summary
Mobile Specific Testing | Summary 
Be creative and on the move while testing mobile apps 
Combine your in house testing wi...
Thanks for your attention! 
More information about mobile testing 
can be found in my book 
„Hands-On Mobile App Testing“ ...
AOE GmbH 
LuisenForum, Kirchgasse 6 
65185 Wiesbaden 
Germany 
Phone: +49 6122 70 70 7 - 0 
Fax: +49 6122 70 70 7 - 199 
E...
Links & References 
• World Quality Report [http://www.sogeti.com/en-us/looking-for-solutions/Services/Software- 
Control-...
Links & References 
• MonkeyTalk [https://www.cloudmonkeymobile.com/monkeytalk] 
• UIAutomation[https://developer.apple.co...
How to Break your App - Best Practices in Mobile App Testing
Nächste SlideShare
Wird geladen in …5
×

How to Break your App - Best Practices in Mobile App Testing

5.239 Aufrufe

Veröffentlicht am

These are the slides from my keynote talk at the Mobile App Europe conference 2014 in Potsdam. Unfortunately, I was not able to give the talk because of sickness, but nevertheless I want to share the slides with you guys. If you have any questions, don't hesitate to get in contact with me.

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

How to Break your App - Best Practices in Mobile App Testing

  1. 1. Customer Visual How to Break your App - Best Practices in Mobile App Testing Daniel Knott | @dnlkntt 1 29. September 2014
  2. 2. Who am I ? • Daniel Knott • Software Test Manager/ Mobile Tester @AOE • Mobile Testing since the end of 2010 • Before AOE, Team Lead QA @XING Mobile Team • Blogger and Speaker http://www.adventuresinqa.com http://www.linkedin.com/pub/daniel-knott/1a/925/993 https://www.xing.com/profile/Daniel_Knott @dnlkntt
  3. 3. 3 29. September 2014 I Company Profile
  4. 4. Intro Current Status Mobile Challenges The Customer Mobile Specific Testing Mobile Pyramid Tools
  5. 5. 5 29. September 2014 I Current Status Author: Barry Caruth, https://www.flickr.com/photos/caruth/8401943928/
  6. 6. Current Status | Some Numbers 2+ millions apps available 100+ billion downloads 80% – 90% of apps are deleted after the first usage 56% of apps that requires a registration gets deleted Average user checks device 150 times per day 45% are not well tested
  7. 7. KIFSU – To Remember the important Things K I F S U Keep It Fast Simple Useable
  8. 8. 8 29. September 2014 I Mobile Challenges
  9. 9. Mobile Challenges | All known!? http://opensignal.com/reports/2014/android-fragmentation/
  10. 10. Mobile Challenges | All known!? http://opensignal.com/reports/2014/android-fragmentation/
  11. 11. Mobile Challenges | How to handle fragmentation Know the target group Group your devices, A B C Test only on those devices Test on real device Don’t use simulators or emulators
  12. 12. Mobile Challenges | How to handle fragmentation Group 1, Prio A • High End Devices • Dual/ Quad Core CPU • RAM >1024MB • Retina, Full HD Display >= 4,5“ Group 2, Prio B • Mid-range devices • Dual Core CPU • RAM <1024MB • Screen size <= 4“ • Software not older than 1 year Group 3, Prio C • Small devices • Single Core • RAM < 512MB • Low screen size and resolution • Older OS, older browser Based on your target group
  13. 13. Mobile Challenges | More… Author: Davide Vizzini Limbte, http://www.flickr.com/photos/limbte/4007115435/ Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053 Author: Rain Rabbit, http://www.flickr.com/photos/37996583811@N01/8033259053/
  14. 14. 14 29. September 2014 I The Customer Author: Michael Newman, https://www.flickr.com/photos/mzn37/1226016289/
  15. 15. The Customer | Better know them Know the target group Will downsize testing effort Talk to your customer Do usability testing with them Author: Grzegorz Lobinski, http://www.flickr.com/photos/gregloby/3515990945/ Invite them to a beta testing program (Sonos, FB, Twitter)
  16. 16. 16 29. September 2014 I Mobile Specific Testing Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/9563767145/
  17. 17. Mobile Specific Testing Known functional testing is still required • Boundary values • Equivalence classes • Testing against acceptance criteria Test the app as black box • Try to break it • Be creative and think in edge cases
  18. 18. Mobile Specific Testing Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053 Author: Ian Britton, http://www.flickr.com/photos/freefoto/728651045/ Author: @Doug88888, http://www.flickr.com/photos/doug88888/7149187135/ Author: Moyan Brenn, https://www.flickr.com/photos/aigle_dore/5626341059/ Author: Dan Levy, http://www.flickr.com/photos/danlevy/5607259711/
  19. 19. Mobile Specific Testing Author: amateur_photo_bore, http://www.flickr.com/photos/andresv/233667770/ Author: Rain Rabbit, http://www.flickr.com/photos/37996583811@N01/8033259053/ Author, Daniel Sancho, http://www.flickr.com/photos/teclasorg/2852716491/
  20. 20. Mobile Specific Testing Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4311105154/ Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4464587302/ Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/9563767145/
  21. 21. Mobile Specific Testing Android • JouleUnit, to measure battery usage • Monkey, to stress test the app iOS • Energy Usage in Instruments • UI AutoMonkey, to stress test the app
  22. 22. Mobile Specific Testing Author: Molly Holzschlag, http://www.flickr.com/photos/mollyeh11/272434559/ Author: Quinn Dombrowski, http://www.flickr.com/photos/quinnanya/4718635484/
  23. 23. Mobile Specific Testing Author: minxlj, http://www.flickr.com/photos/minxlj/3270996158/ Author: Ryan B. BulaPhotography, http://www.flickr.com/photos/nayralub/3763581778/ Author: Sterling Coffey, http://www.flickr.com/photos/n0ssc/7046915929/
  24. 24. 24 29. September 2014 I Tools Author: OZinOH, http://www.flickr.com/photos/75905404@N00/7126147125/
  25. 25. Test Automation Pyramid Manual Testing UI Tests Integration Tests Unit Tests
  26. 26. Flipped Test Pyramid Manual Tests UI Tests E2E Unit Tests
  27. 27. Mobile Test Pyramid Unit Tests Beta Tests End2End Tests (UI Tests) Manual Tests
  28. 28. Tools Do test automation! Don’t buy huge test automation tools Start simple and discover different tools No tool is perfect See which tool fits in your development environment
  29. 29. Tools | Android Robotium (ExtSolo) Spoon Selendroid Calabash for Android Robolectric Espresso Monkey MonkeyTalk Appium Great mobile tools provided by the company square: http://square.github.io/
  30. 30. Tools | iOS UIAutomation Keep It Functional MonkeyTalk Calabash for iOS ios-driver Kiwi XCTest Appium
  31. 31. 31 29. September 2014 I Security Author: Alexandre Dulaunoy, http://www.flickr.com/photos/adulau/7712545428/
  32. 32. Security Author: Anthony Catalano, http://www.flickr.com/photos/badwsky/34164244/ Author: Dennis Wong, http://www.flickr.com/photos/denniswong/2406135310/ Âuthor: orkomedix, http://www.flickr.com/photos/orkomedix/5211437958/
  33. 33. Security Security testing is not easy Get help from experts Think twice about app permissions Check the OWASP mobile security page https://www.owasp.org/index.php/OWASP_Mobile_ Security_Project
  34. 34. 34 29. September 2014 I Summary
  35. 35. Mobile Specific Testing | Summary Be creative and on the move while testing mobile apps Combine your in house testing with crowd testing Be up to date! Know your target group Keep the mobile features in mind Keep the mobile test pyramid in mind Do test automation and know the right tools
  36. 36. Thanks for your attention! More information about mobile testing can be found in my book „Hands-On Mobile App Testing“ which will be available very soon. www.handsonmobileapptesting.com
  37. 37. AOE GmbH LuisenForum, Kirchgasse 6 65185 Wiesbaden Germany Phone: +49 6122 70 70 7 - 0 Fax: +49 6122 70 70 7 - 199 E-Mail: sales@aoe.com Web: www.aoe.com 37 29. September 2014 I in Germany
  38. 38. Links & References • World Quality Report [http://www.sogeti.com/en-us/looking-for-solutions/Services/Software- Control-Testing/The-World-Quality-Report-2013--2014/] • Robotium [https://code.google.com/p/robotium/] • Spoon [https://github.com/square/spoon] • Robolectric [http://robolectric.org/] • Calabash for Android [https://github.com/calabash/calabash-android] • Selendroid [http://selendroid.io/] • ExtSolo [http://docs.testdroid.com/_static/extSolodocs/com/bitbar/recorder/extensions/ExtSolo.html] • Espresso [https://code.google.com/p/android-test-kit/wiki/Espresso] • Monkey [http://developer.android.com/tools/help/monkey.html] • UI AutoMonkey [https://github.com/jonathanpenn/ui-auto-monkey]
  39. 39. Links & References • MonkeyTalk [https://www.cloudmonkeymobile.com/monkeytalk] • UIAutomation[https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/ InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html] • Keep It Functional [https://github.com/kif-framework/KIF] • XCTest [https://developer.apple.com/library/ios/documentation/ToolsLanguages/Conceptual/Xcode_Overvi ew/UnitTestYourApp/UnitTestYourApp.html] • Calabash for iOS [https://github.com/calabash/calabash-ios] • ios-driver [http://ios-driver.github.io/ios-driver/?page=home] • Kiwi [https://github.com/allending/Kiwi] • Appium [http://appium.io/] • JouleJunit [https://code.google.com/p/jouleunit/] • Energy Usage Instruments [https://developer.apple.com/library/ios/documentation/AnalysisTools/Reference/Instruments_User _Reference/EnergyUsageInstrument/EnergyUsageInstrument.html]

×