While 2014 did not have large site failures as the year before did with sites like healthcare.gov, it was not all smooth sailing in the web and mobile performance space. Take a look back at 2014 and learn from some of the failures so we don't repeat them in 2015.

1. © 2014 SOASTA. All rights reserved.
January 30, 2015
1
CONFIDENTIAL – Not for Distribution

2. © 2014 SOASTA. All rights reserved.
January 30, 2015
2
CONFIDENTIAL – Not for Distribution
o Overview
o Top 10 Mobile and Web Performance Lessons from 2014
o Demo
o Q&A

3. © 2014 SOASTA. All rights reserved.
January 30, 2015
3
CONFIDENTIAL – Not for Distribution
According to the IBM Black Friday Report 2014
o Thanksgiving Day traffic from mobile (smartphones and tablets) was
52% of all online traffic
• First time mobile devices have outpaced PC for online browsing
• People don’t typically bring their PC to Thanksgiving dinner but they don’t
want to miss a sale!
o Black Friday mobile traffic was 49.6% of all online traffic
• Increased 25% over last year
o Black Friday mobile sales were 27.9% of all online sales
• Increased 28.2% over last year

4. © 2014 SOASTA. All rights reserved.
January 30, 2015
4
CONFIDENTIAL – Not for Distribution
o Thanksgiving online sales were up 14.3% compared to 2013
o Black Friday online sales were up 9.5% YoY
• Mobile devices accounted for 25% of all online purchases

5. © 2014 SOASTA. All rights reserved.
January 30, 2015
5
CONFIDENTIAL – Not for Distribution
o And Tablets lead SmartPhones

6. © 2014 SOASTA. All rights reserved.
January 30, 2015
6
CONFIDENTIAL – Not for Distribution
o BestBuy site failed:
“A concentrated spike in mobile
traffic triggered issues that led us
to shut down BestBuy.com in
order to take proactive measures
to restore full performance.”

7. © 2014 SOASTA. All rights reserved.
January 30, 2015
7
CONFIDENTIAL – Not for Distribution
o 4G Coverage is covering the map

8. © 2014 SOASTA. All rights reserved.
January 30, 2015
8
CONFIDENTIAL – Not for Distribution
o WiFi is Everywhere, too
• Xfinity Connect modems
• Every airport, hotel, even stores (e.g. Kohl’s, Target)
• Levi’s Stadium, home of the 49ers
• 1200 Access Points, 1 for every 100 seats
• 40 Gb/s capacity
• Mobile app for video replays,
food ordering, wayfinding
o More access from mobile web,
mobile apps

9. © 2014 SOASTA. All rights reserved.
January 30, 2015
9
CONFIDENTIAL – Not for Distribution
o Number of users accessing from mobile devices will continue to grow
o A customer can be in a store and in multiple mobile apps and sites

10. © 2014 SOASTA. All rights reserved.
January 30, 2015
10
CONFIDENTIAL – Not for Distribution
o 2.5M Likes on Facebook
o 75k followers on Twitter
o Flash sale on Nov. 11
• Just three hours long!

11. © 2014 SOASTA. All rights reserved.
January 30, 2015
11
CONFIDENTIAL – Not for Distribution
o Site down

12. © 2014 SOASTA. All rights reserved.
January 30, 2015
12
CONFIDENTIAL – Not for Distribution
o Customer Service burden
o Customer Retention Cost
• Email: “Our apologies and $10 off.
How lucky can you get if the site is
down?”

13. © 2014 SOASTA. All rights reserved.
January 30, 2015
13
CONFIDENTIAL – Not for Distribution
o DSW site has clean history for site failures
Except:
o Coupon code processor was not able to scale to traffic
• For second year in a row
o It’s not enough to load test the site without testing the scenarios

14. © 2014 SOASTA. All rights reserved.
January 30, 2015
14
CONFIDENTIAL – Not for Distribution
o Include testing time in your schedule
• Seasonal events don’t move!
o Make time for testing twice, or thrice!
• If first test fails, when will you test again?
• Can you tweak things quickly to run more than one test per window?
o Even with Continuous Testing, you should leave extra time for load
testing. Passing functional tests doesn’t mean your app will pass
load tests

15. © 2014 SOASTA. All rights reserved.
January 30, 2015
15
CONFIDENTIAL – Not for Distribution
Don’t let your site go down in your high season!
o Super Bowl Sunday – commercials drives high demand to your site
o Chinese New Year – retail load for new clothes
o Valentine’s Day – candy, gifts, flowers
o Tax Day – most online filers submit on April 15
o Healthcare enrollment – many filing for coverage in November
o Singles Day – Largest online shopping day in the world!
o Christmas season – Thanksgiving Day, Black Friday, Cyber Monday

16. © 2014 SOASTA. All rights reserved.
January 30, 2015
16
CONFIDENTIAL – Not for Distribution
o At least they could have been honest!

17. © 2014 SOASTA. All rights reserved.
January 30, 2015
17
CONFIDENTIAL – Not for Distribution
o Add capacity when you know you’ll need it
o Reduce the amount of content your are trying to deliver
• Lower bytecount means faster delivery
o Simplify Your Pages
• Remove extra third-party features that slow your site down
o Optimize for different devices
• Don’t send high-res images to low-res devices
• Users don’t want their data allowance hogged by your site

18. © 2014 SOASTA. All rights reserved.
January 30, 2015
18
CONFIDENTIAL – Not for Distribution
o Valentine’s spending:
o Test from the same locations as your customers

19. © 2014 SOASTA. All rights reserved.
January 30, 2015
19
CONFIDENTIAL – Not for Distribution
o See how Target monitored their site from the users’ perspective on
Black Friday:
o Synthetic monitoring doesn’t collect real users’ experience

20. © 2014 SOASTA. All rights reserved.
January 30, 2015
20
CONFIDENTIAL – Not for Distribution
o Possible areas for failure:
• Payment processing engine
• Social media content plug-in
• CDN
• Analytics plug-in
• Synchronus vs. Asynchronus calls
o Build your site to handle third party content failures
• Why let another site’s problem ruin your user’s experience?

21. © 2014 SOASTA. All rights reserved.
January 30, 2015
21
CONFIDENTIAL – Not for Distribution
o Test in pre-production but don’t only test there
o Test in production – the only way to know the true capacity
• Test off-hours
• Better to find issues before the user volume
• Test at 100% anticipated load, then 125%, then 150%, even 200%
• Good way to find weak spots to put in budget for next season
• Maybe your competitors site will fail and your site will get extra load
o Monitor the production site at system and user level
• Watch CPU, memory, IO, throughput, etc.
• Measure real user metrics to determine a real user’s experience
• Monitor in real-time so you can modify the test, down or up

22. © 2014 SOASTA. All rights reserved.
January 30, 2015
22
CONFIDENTIAL – Not for Distribution
o Best Buy failed on Black Friday after running an online promotion
o Cost of Downtime calculator
• https://www.serverdensity.com/cost-of-downtime

23. © 2014 SOASTA. All rights reserved.
January 30, 2015
23
CONFIDENTIAL – Not for Distribution
o

24. © 2014 SOASTA. All rights reserved.
January 30, 2015
24
CONFIDENTIAL – Not for Distribution
o Facebook and Instagram go down on January 26
• Facebook said: “Earlier today many people had trouble accessing Facebook
and Instagram. This was not the result of a third party attack but instead
occurred after we introduced a change that affected our configuration
systems. We moved quickly to fix the problem, and both services are now back
to 100% for everyone.”
• The dating app Tinder, which relies on Facebook to provide its service, was
also affected.

25. © 2014 SOASTA. All rights reserved.
January 30, 2015
25
CONFIDENTIAL – Not for Distribution
Expertise

26. © 2014 SOASTA. All rights reserved.
January 30, 2015
26
CONFIDENTIAL – Not for Distribution
o mPulse: Real User Monitoring (RUM)
o CloudTest: Load testing from the public cloud
o Data Science Workbench: analytics
Also Available:
o TouchTest: functional testing of mobile apps and websites

27. © 2014 SOASTA. All rights reserved.
January 30, 2015
27
CONFIDENTIAL – Not for Distribution
o Sign up for mPulse Lite, CloudTest Lite, TouchTest Lite
o Always Free, no time-out
• http://goo.gl/jWEX56 or Soasta.com/free
• Free access to CloudLink forums, videos, docs: http://cloudlink.soasta.com
o Read our blog
• Soasta.com/blog
o Third-party sites
• PerfBytes: perfbytes.com podcast
• PerfPlanet: perfplanet.com
o Keep in touch: @TomChavez