1 26.März2015
Customer Visual
How to Break your App -
Best Practices in Mobile App Testing
Workshop
Daniel Knott | @dnlkntt
Who am I ?
• Daniel Knott
• Software Test Manager/ Mobile Tester @AOE
• Software Tester since 2008
• Mobile Testing since ...
3 26.März2015
I Company Profile
Agenda
1. Current Status
3. How to break your app
2. Exercise I
5. Summary and
Close Up
4. Exercise II
5 26.März2015
I Current Status
Author: Barry Caruth, https://www.flickr.com/photos/caruth/8401943928/
Current Status | Some Numbers
2+ millions apps available
100+ billion downloads
+19,000 Android Devices
16 iOS Devices
Current Status | Some Numbers
80% – 90% of apps are deleted after the first usage
56% of apps that requires a registration...
Current Status | Mobile Testing is different
High User expectation
Lot’s of different devices with HW and SW
Users are on ...
9 26.März2015
I Exercise I
Exercise I
Build up groups 2-5 person
Mind Map your challenges in mobile testing
20 minutes
Group discussion about the min...
Exercise I
Challenges| How to handle fragmentation
Group your devices, A B C
Test only on those devices
Test on real device
Don’t use...
Challenges | How to handle fragmentation
Group 1, Prio A
• High End
Devices
• Dual/ Quad Core
CPU
• RAM >2048MB
• Retina, ...
Challenges | The User - KIFSU
Simple UseableFastItKeep
K I F S U
16 26.März2015
I How to break your app
Author: OZinOH, http://www.flickr.com/photos/75905404@N00/7126147125/
Mobile Specific Testing
Known functional testing is still required
• Boundary values
• Equivalence classes
• Testing again...
Mobile Specific Testing
Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053
Author: Ian...
Author: Davide Vizzini Limbte, http://www.flickr.com/photos/limbte/4007115435/
Mobile Specific Testing
• Logout (English)
...
Mobile Specific Testing
Author: amateur_photo_bore, http://www.flickr.com/photos/andresv/233667770/ Author: Rain Rabbit, h...
Mobile Specific Testing
Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4311105154/ Author: Surat Lozow...
Mobile Specific Testing
Author: Molly Holzschlag, http://www.flickr.com/photos/mollyeh11/272434559/ Author: Quinn Dombrows...
Mobile Specific Testing
Author: minxlj, http://www.flickr.com/photos/minxlj/3270996158/ Author: Ryan B. BulaPhotography, h...
Author: Anthony Catalano, http://www.flickr.com/photos/badwsky/34164244/
Author: Dennis Wong, http://www.flickr.com/photos...
30 26.März2015
I Exercise II
Exercise II
Build up groups 2-5 person
Download a mobile app of your choice
Test it with provided scenarios
Note down the ...
Exercise II – My Bugs – Android Wordpress App
Using the app in bad network
conditions
Don’t show technical errors
Exercise II – My Bugs – Android Wordpress App
Provide user friendly error messages
Test app without backend
availability
Exercise II – My Bugs – Android Wordpress App
Language size
Test your app in different languages
Exercise II – My Bugs – Android Wordpress App
Localization problems
Exercise II – My Bugs – Android Wordpress App
Localization problems
37 26.März2015
I Summary
Mobile Specific Testing | Summary
Mobile Testing is different compared to other software
Be creative and on the move while...
Thanks for your attention!
www.handsonmobileapptesting.com
More information about mobile testing
can be found in my book
„...
40 26.März2015
AOE GmbH
LuisenForum, Kirchgasse 6
65185 Wiesbaden
Germany
Phone: +49 6122 70 70 7 - 0
Fax: +49 6122 70 70 ...
Links & References
• World Quality Report [http://www.sogeti.com/en-us/looking-for-solutions/Services/Software-
Control-Te...
Links & References
• MonkeyTalk [https://www.cloudmonkeymobile.com/monkeytalk]
• UIAutomation[https://developer.apple.com/...
How to Break your App - Workshop - Testbash 2015
Nächste SlideShare
Wird geladen in …5
×

How to Break your App - Workshop - Testbash 2015

1.837 Aufrufe

Veröffentlicht am

Mobile phones are available since the middle of the 1980s. Since then, the devices changed savagely but the biggest change happened in 2007, when the first iPhone was presented by Apple. Since then, the mobile smartphone market knows only one direction – UP! Since 8 years touch devices are everywhere, from smartphones to tablets.

More than 2 millions apps are available for download in the stores of the biggest vendors and this number is still increasing. There are apps for photos, music, games, office and many more categories just to name some of them.

But what about the quality of those apps? Are those apps reliable, trust worthy, easy to use, well developed and tested? The latest world quality report from Sogeti shows that almost half (45 percent) of mobile apps are not well tested in terms of functionality, performance and security.

This workshop includes insights into the challenging job of mobile testing from native to web apps. Best practices will be provided to become a better mobile tester. Besides that, this workshop will show different test techniques from functional to non functional mobile testing, test automation tools and how to handle the device fragmentation.

The workshop will not include practical mobile test automation. However, the participants will be able to test different mobile apps manually based on the content of the workshop.

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

How to Break your App - Workshop - Testbash 2015

  1. 1. 1 26.März2015 Customer Visual How to Break your App - Best Practices in Mobile App Testing Workshop Daniel Knott | @dnlkntt
  2. 2. Who am I ? • Daniel Knott • Software Test Manager/ Mobile Tester @AOE • Software Tester since 2008 • Mobile Testing since the end of 2010 • Author, Blogger and Speaker http://www.adventuresinqa.com @dnlkntt http://www.linkedin.com/pub/daniel-knott/1a/925/993 https://www.xing.com/profile/Daniel_Knott
  3. 3. 3 26.März2015 I Company Profile
  4. 4. Agenda 1. Current Status 3. How to break your app 2. Exercise I 5. Summary and Close Up 4. Exercise II
  5. 5. 5 26.März2015 I Current Status Author: Barry Caruth, https://www.flickr.com/photos/caruth/8401943928/
  6. 6. Current Status | Some Numbers 2+ millions apps available 100+ billion downloads +19,000 Android Devices 16 iOS Devices
  7. 7. Current Status | Some Numbers 80% – 90% of apps are deleted after the first usage 56% of apps that requires a registration gets deleted Average user checks device 150 times per day 45% are not well tested
  8. 8. Current Status | Mobile Testing is different High User expectation Lot’s of different devices with HW and SW Users are on the move Mobile Tools and Approaches
  9. 9. 9 26.März2015 I Exercise I
  10. 10. Exercise I Build up groups 2-5 person Mind Map your challenges in mobile testing 20 minutes Group discussion about the mind map
  11. 11. Exercise I
  12. 12. Challenges| How to handle fragmentation Group your devices, A B C Test only on those devices Test on real device Don’t use simulators or emulators Know the devices of your target group
  13. 13. Challenges | How to handle fragmentation Group 1, Prio A • High End Devices • Dual/ Quad Core CPU • RAM >2048MB • Retina, Full HD Display >= 5“ Group 2, Prio B • Mid-range devices • Dual Core CPU • RAM <=1024MB • Screen size <= 4“ • Software not older than 1 year Group 3, Prio C • Small devices • Single Core • RAM < 512MB • Low screen size and resolution • Older OS, older browser Based on your target group
  14. 14. Challenges | The User - KIFSU Simple UseableFastItKeep K I F S U
  15. 15. 16 26.März2015 I How to break your app Author: OZinOH, http://www.flickr.com/photos/75905404@N00/7126147125/
  16. 16. Mobile Specific Testing Known functional testing is still required • Boundary values • Equivalence classes • Testing against acceptance criteria Test the app as black box • Try to break it • Be creative and think in edge cases
  17. 17. Mobile Specific Testing Author: Juan de Dios Santander Vela, http://www.flickr.com/photos/juandesant/209098053 Author: Ian Britton, http://www.flickr.com/photos/freefoto/728651045/ Author: Moyan Brenn, https://www.flickr.com/photos/aigle_dore/5626341059/ Author: @Doug88888, http://www.flickr.com/photos/doug88888/7149187135/ Author: Dan Levy, http://www.flickr.com/photos/danlevy/5607259711/
  18. 18. Author: Davide Vizzini Limbte, http://www.flickr.com/photos/limbte/4007115435/ Mobile Specific Testing • Logout (English) • Ausloggen (German) • Déconnexion (French) • Çıkış yap (Turkish) • Выйти (Russian)
  19. 19. Mobile Specific Testing Author: amateur_photo_bore, http://www.flickr.com/photos/andresv/233667770/ Author: Rain Rabbit, http://www.flickr.com/photos/37996583811@N01/8033259053/ Author, Daniel Sancho, http://www.flickr.com/photos/teclasorg/2852716491/
  20. 20. Mobile Specific Testing Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4311105154/ Author: Surat Lozowick, http://www.flickr.com/photos/suratlozowick/4464587302/ Author: Theus Falcao, http://www.flickr.com/photos/theusfalcao/9563767145/
  21. 21. Mobile Specific Testing Author: Molly Holzschlag, http://www.flickr.com/photos/mollyeh11/272434559/ Author: Quinn Dombrowski, http://www.flickr.com/photos/quinnanya/4718635484/
  22. 22. Mobile Specific Testing Author: minxlj, http://www.flickr.com/photos/minxlj/3270996158/ Author: Ryan B. BulaPhotography, http://www.flickr.com/photos/nayralub/3763581778/ Author: Sterling Coffey, http://www.flickr.com/photos/n0ssc/7046915929/
  23. 23. Author: Anthony Catalano, http://www.flickr.com/photos/badwsky/34164244/ Author: Dennis Wong, http://www.flickr.com/photos/denniswong/2406135310/ Âuthor: orkomedix, http://www.flickr.com/photos/orkomedix/5211437958/ Mobile Specific Testing
  24. 24. 30 26.März2015 I Exercise II
  25. 25. Exercise II Build up groups 2-5 person Download a mobile app of your choice Test it with provided scenarios Note down the bugs 30 Minutes Let’s talk about the bugs
  26. 26. Exercise II – My Bugs – Android Wordpress App Using the app in bad network conditions Don’t show technical errors
  27. 27. Exercise II – My Bugs – Android Wordpress App Provide user friendly error messages Test app without backend availability
  28. 28. Exercise II – My Bugs – Android Wordpress App Language size Test your app in different languages
  29. 29. Exercise II – My Bugs – Android Wordpress App Localization problems
  30. 30. Exercise II – My Bugs – Android Wordpress App Localization problems
  31. 31. 37 26.März2015 I Summary
  32. 32. Mobile Specific Testing | Summary Mobile Testing is different compared to other software Be creative and on the move while testing mobile apps Be up to date! Know your target group Keep the mobile features in mind Keep the high user expectations in mind
  33. 33. Thanks for your attention! www.handsonmobileapptesting.com More information about mobile testing can be found in my book „Hands-On Mobile App Testing“ Soon available in a printed copy, too.
  34. 34. 40 26.März2015 AOE GmbH LuisenForum, Kirchgasse 6 65185 Wiesbaden Germany Phone: +49 6122 70 70 7 - 0 Fax: +49 6122 70 70 7 - 199 E-Mail: sales@aoe.com Web: www.aoe.com I in Germany
  35. 35. Links & References • World Quality Report [http://www.sogeti.com/en-us/looking-for-solutions/Services/Software- Control-Testing/The-World-Quality-Report-2013--2014/] • Robotium [https://code.google.com/p/robotium/] • Spoon [https://github.com/square/spoon] • Robolectric [http://robolectric.org/] • Calabash for Android [https://github.com/calabash/calabash-android] • Selendroid [http://selendroid.io/] • ExtSolo [http://docs.testdroid.com/_static/extSolodocs/com/bitbar/recorder/extensions/ExtSolo.html] • Espresso [https://code.google.com/p/android-test-kit/wiki/Espresso] • Monkey [http://developer.android.com/tools/help/monkey.html] • UI AutoMonkey [https://github.com/jonathanpenn/ui-auto-monkey]
  36. 36. Links & References • MonkeyTalk [https://www.cloudmonkeymobile.com/monkeytalk] • UIAutomation[https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/ InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html] • Keep It Functional [https://github.com/kif-framework/KIF] • XCTest [https://developer.apple.com/library/ios/documentation/ToolsLanguages/Conceptual/Xcode_Overvi ew/UnitTestYourApp/UnitTestYourApp.html] • Calabash for iOS [https://github.com/calabash/calabash-ios] • ios-driver [http://ios-driver.github.io/ios-driver/?page=home] • Kiwi [https://github.com/allending/Kiwi] • Appium [http://appium.io/] • JouleJunit [https://code.google.com/p/jouleunit/] • Energy Usage Instruments [https://developer.apple.com/library/ios/documentation/AnalysisTools/Reference/Instruments_User _Reference/EnergyUsageInstrument/EnergyUsageInstrument.html]

×