1. eBook
Add to Cart:
How One Simple Purchase
Led to Four Unique Stories
of E–Commerce Success
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

2. eBook
Add to Cart
Table of Contents
INTRODUCTION
03
CHAPTER 1 : With Visibility Comes Insight
04
CHAPTER 2: The Beauty of a Single Pane of Glass
09
CHAPTER 3: Fixing Problems Before They Happen
12
CHAPTER 4: Up and Running in No Time
16
CONCLUSION
19
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

3. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
Introduction
On a spring day in May 1984, Jane Snowball, a 72-year-old grandmother
poor user experiences, time lost working on strategic objectives. And
living in Gateshead, England, went about her typical weekly errands, buying
it’s not just the smaller companies that are suffering; these issues are
groceries from her local Tesco supermarket. Unlike the rest of the world,
impacting the e-commerce giants as well.
however, Mrs. Snowball didn’t travel to the store and make the purchase
in person; she did it from home using a modified domestic television
In fact, a March 2013 report found that the world’s top e-commerce sites are
connected by a phone line to the Tesco store. This experiment—conducted
22 percent slower than they were the previous year.4 To make matters worse,
in partnership between the city’s Social Services department, Tesco and
the three most widely used browsers—Firefox, Chrome, and Internet Explorer—
all showed slower load times as well. Yet in the face of all this, users are
and it led the way for companies like Amazon and eBay that spearheaded
becoming increasingly impatient, with research suggesting that the average
the industry a decade later.
Internet user has a wait time threshold of three seconds or less.
Fast-forward to today, and you’ll find an estimated 162,884 e-commerce
In an industry where the name of the game is speed, the way you manage
companies 1 selling goods online every day. In 2012, global e-commerce
and monitor your e-commerce site can potentially make or break you. Some
sales reportedly topped $1 trillion.2 And the American e-commerce market
of the most common application management and monitoring challenges
is expected to grow 13 percent this year, with Forrester predicting web
that e-commerce teams are facing include:
shoppers will spend $262 billion by the end of 2013. 3
• Lack of deep visibility. Many e-commerce teams may feel they have
With numbers like these, it’s safe to say the e-commerce industry is alive
their applications, they never reach their potential in terms of speed and
challenges. Performance bottlenecks, unplanned outages, and massive
traffic spikes can all cause downtime, not to mention the additional
responsiveness. This also means they lack valuable insights that can help
remove performance bottlenecks and improve user experiences.
fallout that ensues: lengthy and expensive maintenance, faulty checkouts,
1
http://clients1.ibisworld.com/reports/us/industry/ataglance.aspx?entid=1930
2
http://www.emarketer.com/Article/E-Commerce-Sales-Topped-1-Trillion-First-Time-2012/1009649
3
http://www.internetretailer.com/2013/03/13/us-e-commerce-grow-13-2013
4
http://www.webperformancetoday.com/2013/03/27/top-e-commerce-sites-are-slower-than-they-were-last-year/
03
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

4. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
• Disparate and disconnected tools. You may have one tool that tells you the
size—and with any budget—to improve application performance, and in turn,
network is down. Another tool that tells you the site is slow. And yet another
customer satisfaction. These tools make it possible to monitor every detail of
tool that tells you how your servers are doing. Having to manage and monitor
your e-commerce application—from the end user experience, through servers,
all these separate tools can make performance monitoring much more time-
down to the line of code—all through a single user interface. And because
SaaS-based application performance management (APM) provides added
• Lack of foresight. It’s common for e-commerce teams to be stuck in
costs and obstacles of traditional on-premise options.
constant reactive mode—often learning about an outage or problem through
This is because they’re not getting a true understanding of site performance
in real time, which in turn, limits their predictive analysis.
and Skullcandy to CanvasPop and Warby Parker. That’s because these
solutions give them:
• Accelerated problem resolution
• Limited deployment time. This is likely true across all IT functions, and it
• Anytime, anywhere app management
• Enhanced user satisfaction
which not only carry a hefty price tag, but are also prohibitive when it comes
you’ve got an e-commerce team desperately looking for an alternative solution.
• Reduced infrastructure and maintenance costs
In this e-book, you’ll learn how e-commerce companies of all shapes and sizes
are tackling application performance challenges and staying ahead of the
The good news is you don’t need to have the budget of an Amazon or eBay
to solve these common challenges. Software-as-a-service (SaaS)-based
competition. Read on to explore four unique stories of e-commerce success
that resulted from one simple purchase: SaaS-based APM.
web application performance tools make it possible for a company of any
04
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

5. CHAPTER 1
With Visibility
Comes Insight
The need for deep visibility into an e-commerce application is typically
a no-brainer: without it, you can’t see what’s going on, and in turn, aren’t
this capability, many e-commerce teams tend to settle for good-enough
Although this had long been the thinking at Warby Parker, eventually the
e-commerce team realized that if it were to improve the performance of
its site, then gaining deep visibility would need to be a strategic priority.
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

6. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
How Warby Parker Cut its Average
Load Time by 50%
Despite this importance of fast load times, Warby Parker went for two
years without access to in-depth, up-to-the-minute information on the
site’s performance. “We had some basic data, but very little of it was
Warby Parker offers boutique quality, classically crafted eyewear at
affordable prices. Founded in 2010, the company engages with many
customers directly through its website, avoiding traditional overhead
$95—including the cost of polycarbonate prescription lenses and anti-
so for a long time we saw no reason to change. But the fact remains
that we didn’t have deep visibility into our environment. As a result, we
weren’t reaching our potential in term of speed and responsiveness.”
to get their hands on a pair (or two, or three) of Warby Parker’s stylish
eyewear. But with the number of unique site visitors increasing by 3040% every month, the company’s rapidly growing engineering team
was under mounting pressure to keep pace with runaway success.
As a metrics-driven company generating a huge proportion of its sales
on the web, Warby Parker recognizes that website performance has
a direct impact on revenue. “For online consumers, a huge part of the
happiness quotient depends on fast load times,” explains Ali Khan,
the company’s Director of Technology. “We subscribe to the idea that
the number of people who will proceed to checkout.”
06
05
© 2013 New Relic, Inc
US +888-643-8776
Warby Parker uses New Relic to analyze website performance by state or region.
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

7. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
That all changed, however, when Khan and his team learned about
SaaS-based APM solution New Relic. They installed the software in less
Measuring customer happiness with Apdex
than an hour, and quickly realized how much it could change Warby
Parker’s whole approach to measuring the success of each deployment.
Apdex is an industry standard to measure users’ satisfaction with the response
“New Relic enables us to measure like we’ve never measured before,”
explains Kahn. “Now, every time we deploy new code—which we do
users are, in contrast to traditional metrics like average response time, which
two or three times each week—we can see, in near real time, if load
can be skewed by a few very long responses.
times increase or decrease. We’re able to measure performance in
terms of geography, so we can monitor our Canadian website with far
Warby Parker relies heavily on New Relic’s Apdex score to track load times and
more precision than before. We can even analyze data on a state-by-
conversion rates, thus helping ensure a higher level of customer happiness.
state or region-by-region level.”
With help from New Relic, Warby Parker cut average load times by
50% in just three months. “We’ve been very aggressive in addressing
Kahn’s team, giving them the insight necessary to make better-informed
any weaknesses across our site in the midst of exponential growth,”
decisions about where to focus resources. As a result, Warby Parker is
says Kahn. “You can’t make that kind of improvement in such a short
now in the enviable position of being as proactive as possible, as often
amount of time unless you have the latest information available at your
as possible. “We’re committed to customer happiness, which means
what New Relic gives us.”
New Relic gives us the data we need to stay far ahead of the curve—and
keep those customers coming back for more.”
Learn more about Warby Parker’s online success:
newrelic.com/case-studies/warby-parker
07
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

8. CHAPTER 2
The Beauty of a
Single Pane of Glass
When business is good, it can be a double-edged sword: on the one
hand, you see revenues skyrocket; yet at the same time, massive spikes
in demand leave the engineering team scrambling to keep pace. This
is especially true when you’re a monitoring-centric company like
swiftly and release software quickly. Understanding the importance
combine metrics from many technologies into a single platform, and
simplify its overall testing process.
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

9. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
unresponsive systems in any part of the process. So when Baird joined
Improved Customer Satisfaction
the company in January 2013, he was glad to see that it already had a
SaaS-based APM solution in place. “I was familiar with New Relic from
my time at a previous company,” he says. “It’s been tested through millions
platform for professional event promoters and venues in the United
States and Canada, from general admission clubs to festival grounds
of hours of use, and it’s easily the most mature solution of its kind.”
and theaters with reserved seating. The company works with some of
the most respected venues and promoters in North America, including
Brooklyn Bowl, Austin City Limits Live, the Independent in San
Francisco, the Troubadour in Los Angeles, the Virgin Mobile FreeFest
in Washington, D.C., and the Pitchfork Music Festival in Chicago.
But of course, with that growth, also comes its own share of challenges.
“As we gear up for another summer concert season, we know that our
system will undergo an immense amount of stress,” says Matthew
to achieve a 10x increase in site performance.”
Warby Parker uses New Relic to analyze website performance by state or region.
imperative. Event attendees are looking to spend their discretionary
09
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

10. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
New Relic is the main tool that Baird uses on a day-to-day basis,
One platform to monitor everything
helping him keep a close eye on the frontend ticket-purchasing
application and the backend event-management application. By
dashboard, Baird is able to generate up-to-the-minute reports on
sales performance. He can easily break out how many tickets the
Platform empowers developers. You can easily build your own plugins
to address almost any issue. And if you want to monetize or build another
desktop, and social channels.
channel, you can use this platform for marketing and promotion. The
possibilities are truly endless.”
Ticketfly also uses New Relic to supplement data from other
applications. For example, Baird and his team rely on BlazeMeter
to determine rate-limiting reactions in large tests—a practice that
rates. In just one 12-week period, errors in the Wordpress stack dropped
initially forced them to keep multiple windows open throughout the
by 98%. The ticket-purchasing application dropped to a zero percent
testing process. But now, with the New Relic Platform, that
error rate. Average response time dropped from 800 milliseconds to
250 milliseconds. And the company achieved all of those improvements
during a time of remarkable growth: during that same 12-week period,
requests made a massive jump—from 3.27 million to 10.2 million.
view, helping the company achieve instant visibility into an increasingly
complex environment. Baird and his team can choose from a number of
“We can see real issues now, and increase performance by 10x through
plugins already created by an open community of developers, or they
dispatching alone,” says Baird. Perhaps most important of all, these
can create their own plugins to solve unique problems.
period of error reduction corresponded, not coincidentally, with
company is achieving a deeper understanding of a broader range of data.
“It’s like having an MRI of our applications, showing us a slice of data across
Learn more about
all systems and making it easier for us to draw correlations,” says Baird.
10
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com
:

11. CHAPTER 3
Fixing Problems
Before They Happen
Whether it’s budget constraints, a growing priorities list, or the fact that
you’re operating as a lean and mean IT team, there are a number of
reasons why e-commerce companies can get stuck in a constant reactive
frame of mind. Looking to shift towards more proactive operations,
CanvasPop decided to overhaul its entire web programming system,
and in the process, realized how critical a role SaaS-based APM plays
in helping improve overall site performance.
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

12. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
How CanvasPop Cut Application Response
Time to Microseconds
After a six-month process of building and testing, CanvasPop went
live with its new PHP-based system on August 1, 2012. The New Relic
PHP client was with them every step of the way. “Deploying New
Since 2009, CanvasPop has delivered more than 280,000 highquality, custom-made canvas photo prints to 100,000+ customers
giving us actionable information within minutes.”
across North America. Like any online retail business, CanvasPop
It wasn’t long before the CanvasPop IT team realized that they
could use SaaS-based APM to monitor not just the new PHP system,
holiday season, when average page views rise from about 250,000
but legacy systems as well. Today, New Relic plays a crucial role in
per day to as many as 700,000 per day. At other times of the year,
CanvasPop’s continuous deployment strategy, providing the real-
heavy media coverage may cause similar jumps in demand. “We
time data necessary for engineers to deploy changes to production
love the exposure,” says Paul Brohman, Lead Software Developer
an average of three to four times every day.
at CanvasPop. “But the exposure won’t do us any good if our site
can’t handle those unexpected visitors.”
A better way to view Transaction Traces
In late 2011, customer demand was threatening to exceed the scale of
the CanvasPop infrastructure. That’s when the IT team determined
Although Google Analytics can be a helpful tool for viewing transaction
that unless they performed a major site overhaul, the customer
experience would suffer. In order to build a more scalable infrastructure
and keep a closer eye on system performance, CanvasPop decided
to migrate their entire system from ASP to PHP.
12
© 2013 New Relic, Inc
traces, the CanvasPop IT team considers it “more of a marketing
dashboard.” The web transactions in New Relic, however, are more
engineering-focused, allowing them to see the slow points in terms
of database memcache or rendering.
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

13. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
“This is a single, unified solution that gives us the same interface
and reports whether we’re in dev, test, or prod,” says Patrick Leckey,
Senior Developer and Lead Systems Architect at CanvasPop.
“And that helps anticipate many production issues that otherwise
wouldn’t show up in testing.” Whether monitoring the dot-com
frontend or the manufacturing backend, New Relic gives CanvasPop
the tools and features necessary to keep all systems on track.
With SaaS-based APM, CanvasPop’s application performance isn’t just
faster—it’s more consistent, too. New Relic proved especially valuable
when Amazon Web Services experienced a major outage on the East
Coast of the United States in October 2012. “New Relic alerted us to
the Amazon outage before Amazon did,” says Leckey.
CanvasPop now has application performance that is faster and more consistent.
In addition to providing up-to-the-minute metrics for proactive
problem solving, having SaaS-based APM also helped CanvasPop IT
address longstanding issues that had simply gone unnoticed. With
faster, easier access to performance metrics—and with all metrics
generated by a single solution—New Relic contributes to a major
CanvasPop teams,” says Leckey. “And I credit New Relic with giving us
the most accurate, most current data we’ve ever had for moving this
business forward.”
boost in productivity, not to mention, knowledge-sharing across the
Learn more about the CanvasPop story:
newrelic.com/case-studies/canvaspop
organization. “Broader awareness means better alignment across all
13
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

14. CHAPTER 4
Up and Running
in No Time
or another from using a SaaS-based solution. These products are affordable
and they’re quick to deploy, making them perfect for e-commerce
environments that need APM—especially when they need the service
up and running immediately. Skullcandy saw the time and cost savings
that would come from using SaaS-based APM compared to a traditional
on-premise solution. And thanks to its decision to go with SaaS, the
e-commerce team is seeing measurable improvements.
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

15. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
How Skullcandy Successfully Launched a
Brand New E-Commerce Site
Director of Interactive Technology, attributes much of that success
to the fact that New Relic is a SaaS-based solution.
Skullcandy became one of the world’s most distinct audio brands
by bringing color, character and performance to an otherwise
once we added it to our provisioning, all our servers get New Relic
monochromatic space and helped revolutionize the audio arena by
introducing headphones, earbuds, and other audio and wireless lifestyle
products that possess unmistakable style and exceptional performance.
that, the enterprise products I’ve used in the past took at least 30 days
Relic, we get everything we need from a single service—alerts, error
When the company launched its e-commerce site in 2008, it took off
immediately. Built on the Magento e-commerce platform with PHP and
Linux, the online channels’ sales growth has challenged the interactive
monitoring, transaction traces, application statistics, etc., and best of all,
we got immediate results and insights that we just had not seen before.”
team to keep up with accelerating demands. And eventually, the site’s
Navigating the App Map
growing pains got worse with a number of performance issues.
Unplanned outages were on the rise, usually occurring more than
An Application Topology map (App Map) illustrates your app and its
related services as an architectural map. It also shows performance
to lost revenue, and potentially, even to lost customers, Skullcandy’s
problems in your app’s end-user experience, your app itself, and any
interactive team immediately sought to identify the solution and
of the internal or external services backing your app.
the problem, Skullcandy still lacked proactive real-time performance
Skullcandy’s use of the App Map feature is interesting. The team wrote
their own internal services to replace parts of Magneto’s system, as
what was going on in real time.
rewrote as an external service. When the system makes an HTTP call
Based on a recommendation, the Skullcandy team decided to give
to the external service, New Relic monitors that performance via the
New Relic a try. They were immediately delighted by the ease of
App Map. This new process helped the e-commerce site’s performance
implementation. And even more delighted with how fast they had
improve by 25%.
actionable data and information. Andrew Spencer, Skullcandy’s
15
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

16. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
Skullcandy has achieved some notable results using New Relic, including
a mean time to resolution improvement of 300%. It’s also made staff’s
would otherwise make them frazzled.
When the company released its new e-commerce site, Skullcandy 2.0, in
2011, having SaaS-based APM especially came in handy. To get ready for
the launch, the team used New Relic to monitor the application, which
New Relic, we had immediate insight into the application, how it was
we needed to move forward even knowing that late in the year is not
the best time to launch a new online site.”
SkullCandy boosts sales with a new online channel.
performance problem that was not revealed by prior tests. But within
minutes, New Relic pinpointed the problem, highlighting both slow
and monitor those issues that quickly, we would have had to roll back
and wait until Q1 of the next year to implement the new site. Thanks
less than 30 minutes to see and fully diagnose the problem. The team
to New Relic, Skullcandy 2.0 went live on time—our internal customers
were pleased and so were we.”
an hour, and after that, the site just hummed.
Learn more about the full scoop on the launch of Skullcandy 2.0:
newrelic.com/case-studies/skullcandy
16
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

17. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
Conclusion
In today’s highly competitive e-commerce landscape, the success of your
To ensure a positive shopping experience for customers, e-commerce
company directly correlates with how quickly you’re able to pinpoint and
companies just like yours are turning to SaaS-based APM to improve
solve performance issues. After all, poor website performance translates
performance and ease application management and monitoring. With
into lost revenue, brand damage and additional support costs for your
these cloud-driven capabilities, you’ll be in the perfect position to increase
company. And when it’s peak shopping season, the pressure can be
revenue and drive business growth.
especially overwhelming.
APM Assessment
QUESTION
Yes
Can we quantify how third-party technology and services, such as ad networks and payment processes, are
If you answered “No” or “Not Sure” to any of the questions above, it may be time for you to consider an APM alternative.
To learn how you can improve your e-commerce site’s performance and boost user satisfaction, visit: www.newrelic.com.
17
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com
No
Not
Sure

18. eBook
Add to Cart: How One Simple Purchase Led
to Four Unique Stories of E–Commerce Success
About New Relic
New Relic is a software analytics company that makes sense of billions of
metrics about millions of applications in real time. Our comprehensive SaaSbased solution provides one powerful interface for web and native mobile
applications and consolidates the performance monitoring data for any
chosen technology in your environment. When your brand and customer
experience depend on the performance of modern software, New Relic
provides insight into your overall performance.
New Relic, San Francisco HQ
New Relic, Portland
New Relic, Seattle
188 Spear Street, Suite 1200
111 SW 5th Avenue, Suite 2800
2101 4th Avenue, 19th Floor
San Francisco, CA 94105
Portland, OR 97204
Seattle, WA 98121
Tel: +1.888.643.8776
support@newrelic.com
www.newrelic.com
18
© 2013 New Relic, Inc
US +888-643-8776
www.newrelic.com
www.twitter.com/newrelic
blog.newrelic.com

19. ©2008-13 New Relic, Inc. All rights reserved.