Top 10 Mobile and Web Perf Lessons

3. © 2014 SOASTA. All rights reserved. May 27, 2015 3 CONFIDENTIAL – Not for Distribution According to the IBM Black Friday Report 2014 o  Thanksgiving Day traffic from mobile (smartphones and tablets) was 52% of all online traffic •  First time mobile devices have outpaced PC for online browsing •  People don’t typically bring their PC to Thanksgiving dinner but they don’t want to miss a sale! o  Black Friday mobile traffic was 49.6% of all online traffic •  Increased 25% over last year o  Black Friday mobile sales were 27.9% of all online sales •  Increased 28.2% over last year

4. © 2014 SOASTA. All rights reserved. May 27, 2015 4 CONFIDENTIAL – Not for Distribution o  Thanksgiving online sales were up 14.3% compared to 2013 o  Black Friday online sales were up 9.5% YoY •  Mobile devices accounted for 25% of all online purchases

6. © 2014 SOASTA. All rights reserved. May 27, 2015 6 CONFIDENTIAL – Not for Distribution o  BestBuy site failed:    “A concentrated spike in mobile  traﬃc triggered issues that led us  to shut down BestBuy.com in  order to take proactive measures  to restore full performance.”

8. © 2014 SOASTA. All rights reserved. May 27, 2015 8 CONFIDENTIAL – Not for Distribution o  WiFi is Everywhere, too •  Xﬁnity Connect modems •  Every airport, hotel, even stores (e.g. Kohl’s, Target) •  Levi’s Stadium, home of the 49ers •  1200 Access Points, 1 for every 100 seats •  40 Gb/s capacity •  Mobile app for video replays,   food ordering, wayﬁnding o  More access from mobile web,  mobile apps

9. © 2014 SOASTA. All rights reserved. May 27, 2015 9 CONFIDENTIAL – Not for Distribution o  Number of users accessing from mobile devices will continue to grow o  A customer can be in a store and in multiple mobile apps and sites

12. © 2014 SOASTA. All rights reserved. May 27, 2015 12 CONFIDENTIAL – Not for Distribution o  Customer Service burden o  Customer Retention Cost •  Email: “Our apologies and $10 oﬀ.  How lucky can you get if the site is  down?”

13. © 2014 SOASTA. All rights reserved. May 27, 2015 13 CONFIDENTIAL – Not for Distribution o  DSW site has clean history for site failures Except: o  Coupon code processor was not able to scale to traﬃc •  For second year in a row o  It’s not enough to load test the site without testing the scenarios

14. © 2014 SOASTA. All rights reserved. May 27, 2015 14 CONFIDENTIAL – Not for Distribution o  Include testing time in your schedule •  Seasonal events don’t move! o  Make time for testing twice, or thrice! •  If ﬁrst test fails, when will you test again? •  Can you tweak things quickly to run more than one test per window? o  Even with Continuous Testing, you should leave extra time for load testing. Passing functional tests doesn’t mean your app will pass load tests

15. © 2014 SOASTA. All rights reserved. May 27, 2015 15 CONFIDENTIAL – Not for Distribution Don’t let your site go down in your high season! o  Super Bowl Sunday – commercials drives high demand to your site o  Chinese New Year – retail load for new clothes o  Valentine’s Day – candy, gifts, flowers o  Tax Day – most online filers submit on April 15 o  Healthcare enrollment – many filing for coverage in November o  Singles Day – Largest online shopping day in the world! o  Christmas season – Thanksgiving Day, Black Friday, Cyber Monday

17. © 2014 SOASTA. All rights reserved. May 27, 2015 17 CONFIDENTIAL – Not for Distribution o  Reduce the amount of content your are trying to deliver •  Lower bytecount means faster delivery o  Simplify Your Pages •  Remove extra third-party features that slow your site down o  Optimize for diﬀerent devices •  Don’t send high-res images to low-res devices •  Users don’t want their data allowance hogged by your site o  Add capacity when you know you’ll need it

19. © 2014 SOASTA. All rights reserved. May 27, 2015 19 CONFIDENTIAL – Not for Distribution o  See how Target monitored their site from the users’ perspective on Black Friday: o  Synthetic monitoring doesn’t collect real users’ experience

20. © 2014 SOASTA. All rights reserved. May 27, 2015 20 CONFIDENTIAL – Not for Distribution o  Possible areas for failure: •  Payment processing engine •  Social media content plug-in •  CDN •  Analytics plug-in •  Synchronus vs. Asynchronus calls o  Build your site to handle third party content failures •  Why let another site’s problem ruin your user’s experience?

21. © 2014 SOASTA. All rights reserved. May 27, 2015 21 CONFIDENTIAL – Not for Distribution o  Test in pre-production but don’t only test there o  Test in production – the only way to know the true capacity •  Test off-hours •  Better to find issues before the user volume •  Test at 100% anticipated load, then 125%, then 150%, even 200% •  Good way to find weak spots to put in budget for next season •  Maybe your competitors site will fail and your site will get extra load o  Monitor the production site at system and user level •  Watch CPU, memory, IO, throughput, etc. •  Measure real user metrics to determine a real user’s experience •  Monitor in real-time so you can modify the test, down or up

22. © 2014 SOASTA. All rights reserved. May 27, 2015 22 CONFIDENTIAL – Not for Distribution o  Best Buy failed on Black Friday after running an online promotion o  Cost of Downtime calculator •  https://www.serverdensity.com/cost-of-downtime

24. © 2014 SOASTA. All rights reserved. May 27, 2015 24 CONFIDENTIAL – Not for Distribution o  Facebook and Instagram go down on January 26 •  Facebook said: “Earlier today many people had trouble accessing Facebook and Instagram. This was not the result of a third party attack but instead occurred after we introduced a change that affected our configuration systems. We moved quickly to fix the problem, and both services are now back to 100% for everyone.” •  The dating app Tinder, which relies on Facebook to provide its service, was also affected.

26. © 2014 SOASTA. All rights reserved. May 27, 2015 26 CONFIDENTIAL – Not for Distribution o  Sign up for mPulse Lite, CloudTest Lite, TouchTest Lite o  Always Free, no time-out •  http://goo.gl/jWEX56 or Soasta.com/free •  Free access to CloudLink forums, videos, docs: http://cloudlink.soasta.com o  Read our blog •  Soasta.com/blog o  Third-party sites •  PerfBytes: perfbytes.com podcast •  PerfPlanet: perfplanet.com o  Keep in touch: @TomChavez

27. © 2014 SOASTA. All rights reserved. May 27, 2015 27 CONFIDENTIAL – Not for Distribution o  mPulse: Real User Monitoring (RUM) o  CloudTest: Load testing from the public cloud o  Data Science Workbench: analytics Also Available: o  TouchTest: functional testing of mobile apps and websites

Top 10 Mobile and Web Perf Lessons

Tom Chavez

Top 10 Mobile and Web Perf Lessons