Challenge - Wanted to transform performance management for digital age - Needed tools that accurately tested and monitored real user scenarios - Not able to find correlations from data collected Result - Tested to scale without additional hardware investments - Empowered performance team with real user data to advise on business - Determined revenue impact of half-second page load improvement - Built model to proactively amend marketing campaigns not meeting revenue targets in real time

1. Keeping pace with digital business
Business changed dramatically in the three decades since Office
Depot was founded. Determined to keep pace, Office Depot
embarked on a digital transformation to address the challenges
in the Internet age with online and omnichannel as the future.
“We have a ton of data. We don’t know what it’s telling us,” said Carl
Brisco, senior vice president of ecommerce at Office Depot. Office
Depot needed a way to correlate all of the metrics it was collecting
with monitoring and marketing tools – and put them to work.
Limited performance monitoring and testing capabilities did not
provide a complete picture of online success. Furthermore, the
team responsible for managing online performance was buried
within quality assurance with limited insight and influence on
business decisions.
“Our performance testing team was embedded deep in the QA
organization. Our testing was basically just push button, really the
analog age for us,” recalled Erick Leon, performance engineering
team lead at Office Depot, of the previous environment. “The
testing team didn’t have much visibility, so we relied heavily on
other teams to tell us what was wrong. For us, this was costly, not
efficient really, because we didn’t have access to their tools. Also, in
those days, the testing tools that we did have could only really test
for one geographic location.”
In today’s fast-moving workplaces, Office Depot could no longer
just take care of business, it needed anticipate and adapt to real
user experiences in real-time – meeting expectations of customers,
wherever and whenever they shopped for office gear.
Boosting productivity with modern
performance technology
Office Depot ran a proof of concept with SOASTA CloudTest®
to
modernize its load testing capabilities. “CloudTest was already using
the cloud to deploy load test generators, so this was awesome for
us. Not only did we not have to finance additional hardware for
testing servers and infrastructure, but we no longer had a limitation
on the amount of users we were able to inject in our sites, really
opening the door for true performance testing. That single item
of user count opened up the opportunity to really simulate, for
instance, Black Friday testing,” said Leon adding that Office Depot
couldn’t generate this kind of volume before CloudTest. “It was
really cost prohibitive with that previous tool.”
With the ability to test real-world scenarios to scale for the first
time, Office Depot turned to SOASTA for solutions that could help
it also understand real user experiences in real time and proactively
manage online performance. The deployment expanded beyond
CloudTest for load testing to incorporate other components of the
SOASTA DPM Platform including mPulse™ real user measurement
(RUM), Data Science™ deep data analytics, and the Digital
Operations Center (DOC™) to monitor and maintain performance
from a central location
“We do have a lot of data – from real-time user metrics, synthetic
and APM tools, core metrics – but until lately we really didn’t have
a central analysis tool that could help us correlate all of that data,”
said Leon.
Performance is Everything
Office Depot Gears Up for Digital
Transformation with SOASTA
Office Depot, Inc. is a leading global provider of products, services, and solutions for every workplace.
The company has annual sales of approximately $14 billion, employs nearly 50,000 associates, and
serves consumers and businesses in 59 countries with about 1,800 retail stores, award-winning
ecommerce sites, and a dedicated business-to-business sales organization. When Office Depot
decided to promote online performance management from a back-office function to the forefront
of business decision making, it turned to the SOASTA Digital Performance Management (DPM)
Platform to start a digital transformation.
Case Study

2. “Now we’re able to take all of that data and see how performance
is working in real-time – how it’s affecting conversion, how our
campaigns are performing: Are they on the right landing pages?
What are those landing pages? Are those landing pages optimized?
What are those landing pages telling us?” added Leon about the DOC
where his team proactively monitors performance. “So it’s all about
taking the data and correlating it all to make better decisions to truly
and positively impact our bottom line.”
One of the SOASTA capabilities that Leon found “very unique
and relevant” was the Data Science correlation of marketing and
performance data to assess campaign effectiveness and make
adjustments in real time. Office Depot used its own historical
data to build a model that it could take action on. “If a campaign
is performing out of the norm, we can reactively amend to inspire
conversion,” said Leon. With this capability, Office Depot set
revenue targets for marketing campaigns, monitored them real
time, and automatically pushed out additional promotions if the
campaign wasn’t achieving targets.
With such innovations, SOASTA continued, “To move forward and
surge ahead with technology,” said Leon. “Any time we bring any
new requirements, SOASTA is always eager to find that solution
and really deliver on that product. With previous vendors, we didn’t
get that. It was pretty much, here’s the tool, and this is what you
have. We really feel that SOASTA’s become a great partner of ours
in that respect, continuing to move forward with our technology to
better increase performance.”
Earning a promotion for performance
management
Online performance management has moved from the back offices
to the forefront of the business, in part thanks to the SOASTA DPM
platform. “Today we are part of the ecommerce operations team
with access to real-time metrics – to front-end server, back-end
server, database server, and all of the other support systems to
our website,” said Leon. “We are now the focal point for complete
round-trip response times, so we tell the other organizations where
they need to look for performance degradations, if there’s any, and
make recommendations as to how to solve those.”
“Our performance testing team has really evolved in that respect.
Now we’re also moving to an automated load test creation by the
use of the Sunburst reports that will model and create better real-
user test cases. This will be molded right from our real user traffic.”
For example, the Sunburst reports illuminated paths Office Depot
hadn’t tested before, including critical pages that affect the success
of promotions, such as campaign landing pages.
Office Depot also used the SOASTA DPM Platform to monitor,
justify, and improve the impact of third parties on its site,
leveraging the RUM data provided to facilitate productive service
level agreement (SLA) conversations. “mPulse is able to really
report daily on how those are performing, and their impact to
conversion,” said Leon. “Any breach in that SLA can be discussed
with those proper parties, or even third parties.”
Performance is Everything
To learn more visit www.soasta.com or email us at info@soasta.com.
“Any time we bring any new requirements, SOASTA is always eager to find that
solution and really deliver on that product. With previous vendors we didn’t get
that. It was pretty much, here’s the tool, and this is what you have. We really
feel that SOASTA’s become a great partner of ours in that respect, continuing
to move forward with our technology to better increase performance.”
- Erick Leon, Performance Engineering Team Lead, Office Depot
Performance is everything
• Aligned ecommerce, marketing, and IT
operations teams with common set of data
• Monitored and boosted marketing campaign
performance with real-time tracking
• Defined conversion gains for a half-second
page load performance improvement
• Saved money by not financing additional
hardware to test at scale for peak loads
• Reduced headcount required with more
efficient performance tools
He added that Office Depot planned to use Data Science to
better understand the impact of synchronous and asynchronous
page loads on conversions. “Some third-party resources are
synchronous, which means that they will wait for the previous
resource to load, while others are asynchronous and load
simultaneously with others,” said Leon. “We really believe that
the asynchronous resource should also load within the page
SLA. I know that most will tell you it does not affect customer
experience, but what we’re doing is we want to take that and
see how it’s really affecting the overall conversion.”
Armed with actionable data, Office Depot was able to show how
online performance impacted the bottom line. “We’re able to
really tell what a half-second performance gain does for our page
in conversion. We’re really able to set that standard,” said Leon.
Office Depot aligned its ecommerce, marketing, and IT operations
with a single view of online performance provided by the SOASTA
DPM platform. It was also able to measure the entire user
experience journey and the impact on revenue.
“The one big thing that we have to understand is it’s not only
about page performance, it’s all about revenue performance,”
said Leon. “With monitoring real-time campaigns in our DOC
and setting up the correlating alerts, we expect to really make
a splash in our financial performance this year.”