Watch a live presentation at http://offer.bitbar.com/ensure-your-mobile-channels-generate-revenue-on-holiday-peak-days Are you – and especially your mobile apps and web channels – ready to make the most of the upcoming holiday season? Being unplugged few minutes or mobile app not being able to deliver means instant loss in revenue. To succeed in this critical time of the year, ‘Sorry, not available’ is not an option for a second during that hectic 6 weeks. Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/

1. 30 September 2015
Ville-Veikko Helppi
Technical Product Manager
ville-veikko.helppi@bitbar.com
How to Ensure Your Mobile
Channels Create Revenue on
Holiday Peak Days
W E B I N A R
Dave Horner
Director of Sales
dave.horner@bitbar.com

2. Agenda
• Brief Testdroid update + Q3 statistics
• The importance of sales readiness &
what are the top things to prepare for
• How to prepare mobile apps and web for
holiday shopping season/peak days
• How to get started – what are the most
important things to get ready
• Q&A

3. Q3'2015 – Test Run Stats

4. Why Android Apps Fail?

5. MOBILE APP AND WEB READINESS

6. 5 E-Commerce Trends in 2015
• Mobile is More Important Than Ever:
28% growth from 2014 to $31.6B
• Branded Shopping Days – Cyber Monday,
Black Friday, Thanksgiving, Christmas
• Search & Purchase – Now More in Mobile
• Forrester Research: Ubiquitous Free
Shipping Offers, Aggressive Email
Marketing and Effective Mobile Presence
* = http://www.lyonscg.com/insights/5-ecommerce-trends-2015-holiday-season/
** = https://www.forrester.com/Forrester+Research+eCommerce+Forecast+2014+To+2019+US/fulltext/-/E-res116713
***= http://www.marketwired.com/press-release/survey-mobile-broadband-performance-expectations-higher-than-ever-
even-as-more-operators-2052324.htm

7. Why Mobile Web?
• 85% of survey respondents said mobile
devices are a central part of everyday life *
• > 50% of US e-commerce web traffic is
done via mobile **
• Google: >50% of searches come from
smartphones ***
* = https://www.salesforce.com/blog/2014/02/mobile-behavior-report.html
** = http://marketingland.com/mobile-drives-nearly-half-e-commerce-traffic-pc-still-rules-sales-report-118629 &
***= http://searchengineland.com/its-official-google-says-more-searches-now-on-mobile-than-on-desktop-220369

8. Holiday Season + Mobile Web
• In 2014, over 60% of shoppers using
Amazon and Target with a mobile device *
• The US e-commerce will top $300 billion
this year **
• Study: Top concern by consumers is the
speed of (mobile internet) service ***
• "web pages loading slowly or not at all" is the
single most frustrating experience
* = http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=2002024
** = https://www.forrester.com/Forrester+Research+eCommerce+Forecast+2014+To+2019+US/fulltext/-/E-res116713
***= http://www.marketwired.com/press-release/survey-mobile-broadband-performance-expectations-higher-than-ever-
even-as-more-operators-2052324.htm

9. Holiday Season + Mobile Apps
• All forecasts show double-digit growth for
m-commerce/mobile app purchases *
• IBM: Larger screens are still preferred when
buying online: 40% Mobile/Tablets **
• Mobile purchases: 45% Android – 55% iOS
• Distractions: Mobile users are more focused
on buying, More distraction on desktops
(shopping cart abandonment higher)
* = http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=2002024
** = http://www.slideshare.net/IBMExpOne/uk-holiday-readiness-2015-infographic
***= http://www.marketwired.com/press-release/survey-mobile-broadband-performance-expectations-higher-than-ever-
even-as-more-operators-2052324.htm

10. Top Failures During Peak Days
• Web: "Service is currently unavailable."
• Instant loss of revenue
• App: Crash or simply 'Do Not Work'
• Instant loss of revenue
• General Slowness of Both – App & Web
• Delayed revenue generation, Frustration
• Web Experience is Bad
• Causes A Hit on Brand -> People go to other
sites to buy (and most likely will stay there)
* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season

11. Android Fragmentation 2015
by Open Signal report August 2015
Testdroid Cloud
>95%

12. Android Fragmentation 2015
by Open Signal report August 2015
Testdroid Cloud
>90%

13. Correct behaviour across
platforms and browsers
Integration with web back-
ends
Typically need to
fully utilize HW
(CPU+GPU)
Resource (e.g.
battery)
consumption
OpenGL ES 2/3
Performance!
Functionality and
usability
Screen
orientations,
connectivity, user
profiles
Robustness
Robustness and security!
Brand
Compliances, verification
with back-ends and data
Different 'Verticals'

14. Recent Changes – Prepare!
• New OS versions with Android & iOS
• Android 6.0 Marshmallow
• iOS 9 and 9.0.1
iOS 8.0
iOS 8.0.2
iOS 8.1
iOS 8.1.1
iOS 8.2
iOS 8 ALL
KitKat 4.4
KitKat 4.4.2
KitKat 4.4.3
KitKat 4.4.4
KitKat ALL
Lollipop 5.0
Lollipop 5.0.1
Lollipop 5.0.2
Lollipop ALL
Android 'M'
0 5 10 15 20 25 30 35
Failed test runs. Percentage (%).

15. HOW CAN YOU PREPARE?

16. First, Get Experts to Help You!
• Testdroid team offers FREE
consultation to get your
mobile app and web ready
• Get in touch with us at
sales@bitbar.com
FREE Ebook available at http://www2.testdroid.com/mobile-success-guide-for-holiday-shopping-season

17. Test Early, Test Often

18. When Revenue is Generated
Thanksgiving
week + weekend
Cyber MondayBlack Friday
Shopping for
Christmas

19. Three Stages in Preparation
• Stage One: Functional & Compatibility
• The basic functionality, compatibility and
thorough test automation across real devices
• Stage Two: Performance & Load
• Performance, Load, Stress, Peak and all other
forms of performance testing
• Stage Three: Production Monitoring
• Monitoring the behavior, speeds, robustness
of the app, web and back-end services
* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season

20. STAGE ONE:
FUNCTIONAL AND COMPATIBILITY

21. Stage #1:
Functional & Compatibility
• Typically well ahead of peak days (10 wks)
• Prepare the functional test suite
• 10 steps in app testing: Install, uninstall, upgrade,
create new account, make payment,
upgrade/downgrade service, integrations,
navigation/behavior flows, visual correctness,
working with back-end
• Select the most used devices & browsers
• Top Android devices from last and this year
• Latest iOS devices, but include also some
older ones as well* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season

22. Stage #1:
Functional & Compatibility
• Good Mix of 'Users'
• Phones and Tablets
• OS versions
• Display sizes/resolutions
• Devices with different memory size
• Use variety of browsers
• Safari, Chrome, Firefox, Webkit, others...
• Use recommended devices (not the most
robust ones, but good combo of users)
* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season

23. Top Prerequisites for Testing
• Real Devices and Real Browsers, as
emulators cannot help you to test...
* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season
• User Experience
• Usability
• Hardware
• Software
• Infrastructure
0 % = the percentage of your app users
that use emulator to run your app!

24. How to Prepare?

25. STAGE TWO:
PERFORMANCE AND LOAD

26. Stage #2:
Performance & Load
• Typically 2-6 weeks ahead of peak days
• Prepare the performance tests
• Measure the launch time (time to revenue), checkout,
graphical content optimization, where is content
hosted, performance w/wo internet, different networks
• Prepare the load tests
• Run significant amount of concurrent views, checkouts,
purchases, web requests, and make tests fail
• Check these useful tools for generating server-side
load for your system: JMeter and Blazemeter
* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season

27. Stage #2:
Performance & Load
• Build in performance tests with real devices
and browsers – setup alerts when low perf.
• Use additional tools – e.g. AT&T ARO,
Qualcomm Trepn profiler, Gamebench
• Measure mobile website traffic, e.g. Chrome
Dashboard & Safari Web Inspector
* = http://testdroid.com/news/is-your-mobile-channel-ready-for-2015-holiday-shopping-season

28. Stage #2:
Performance & Load
• Performance testing in production should be
everyone's goal. There is nothing like the real
world to get true performance metrics.
• Depending on load levels, we recommend
communicating with your 3rd party partners about
when load tests will be performed to forego bad
situations.
• Your performance tests in production can also be
used to "pre-heat" your CDNs and caching
systems. This is a great thing to do before
anticipated spikes.

29. Stage #2:
Performance & Load
• Do you know the behavior of your web users?
100,000 unique visitors
8,250 sign-ups/convers. (8.25%)
3,105 billing pages (37.6%)
2,888 payment pages (93.0%)
2,790 payments done (96.6%)

30. Connectivity & Interdependencies
Page load time is impacted by various different things, from
the user to external systems integrated, and supported by
the back-end systems, servers and other infrastructure.

31. Page Load Time

32. STAGE THREE:
PROACTIVE PRODUCTION MONITORING

33. Stage #3:
Proactive Production Monitoring
• Typically 2 weeks before and 2-3 months
after the launch of peak day critical 'tools'
• Prepare the infrastructure for monitoring
• Ensure your app is downloadable/accessible, utilizing
the latest versions, create accounts, loggings,
purchasing, preferably every 10-15 min intervals
• Prepare efficient comparison of results
• Lots of test data -> compare changes to spot if there
are any weaknesses, vulnerabilities or non-robustness
• Check these useful tools/dashboards for generic
monitoring Nagios and Zabbix

34. Stage #3:
Proactive Production Monitoring
• Good monitoring should always include:
• Alarms, Dashboards, Data Visualization
• Use multiple devices and infrastructure to
rule out false positives and seek realistic
averages
• Understanding the performance on different
devices, GEOs, networks, different steps of
engagement and payments

35. MANAGED SERVICES

38. THANK YOU!
More information about frameworks, devices
and testing at www.testdroid.com