Site spect 2015_forrester_wave (1)

The Forrester Wave™: Online Testing Platforms,
Q3 2015
The Eight Providers That Matter Most And How They Stack Up
by James McCormick
September 22, 2015
For Customer Insights Professionals
forrester.com
Key Takeaways
Adobe, Maxymiser, Optimizely, And SiteSpect
Lead The Pack
Forrester’s research uncovered a market in which
Adobe, Maxymiser, Optimizely, and SiteSpect
lead the pack. HP, Monetate, and Webtrends offer
competitive options. Qubit lags behind.
CI Pros Want Self-Service And A Broad Range
Of Supporting Services
The online testing platform market is growing
because more CI professionals see these
solutions as a way to address their top digital
interaction optimization challenges. Growth is
largely due to CI pros’ increasing trust in online
testing platform providers to act as strategic
partners, providing them with advanced digital
optimization capabilities and advice.
Campaign Management, Enterprise Services,
And Multichannel Testing Differentiate
As previous technology becomes outdated and
less effective, improved campaign management,
enterprise services, and multichannel test
deployment dictate which providers lead
the pack. Vendors that deliver on these
critical improvements position themselves to
successfully deliver online testing capabilities that
support continuous optimization practices.
Why Read This Report
In our 40-criteria evaluation of online testing
platform providers, we identified the eight most
significant ones and researched, analyzed,
and scored them. This report shows how each
provider measures up and helps customer
insights (CI) professionals make the right choice.

2
4
7
10
12
© 2015 Forrester Research, Inc. Opinions reflect judgment at the time and are subject to change. Forrester®
,
Technographics®
, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester
Research, Inc. All other trademarks are the property of their respective companies. Unauthorized copying or
distributing is a violation of copyright law. Citations@forrester.com or +1 866-367-7378
Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA 02140 USA
+1 617-613-6000 | Fax: +1 617-613-5000 | forrester.com
Table Of Contents
Keep Up With Customers By Continually
Evolving Interactions
Online Testing Is A Crucial Component Of
Continuous Optimization
Continuous Optimization Needs Drive Online
Testing Criteria
Online Testing Platform Evaluation Overview
Evaluation Criteria Focuses On Scaling Online
Testing Programs
Evaluated Vendors Are Committed To
Supporting Enterprise-Class Online Testing
Vendors Vary In Functionality, Self-Service,
And Blend Of Services
Vendor Profiles
Leaders
Strong Performers
Contenders
Supplemental Material
Notes & Resources
Forrester conducted product evaluations in June
2015 and interviewed eight vendor companies:
Adobe, HP, Maxymiser, Monetate, Optimizely,
Qubit, SiteSpect, and Webtrends. We also
completed 24 phone reference interviews and
surveyed 113 clients of the above vendors.
Related Research Documents
Decipher The Digital Intelligence Technology Code
Supercharge Analytics With Digital Intelligence
Transform Customer Experience With Continuous
Optimization
The Forrester Wave™: Online Testing Platforms, Q3 2015
by James McCormick
with Mary Pilecki, Samantha Ngo, Emily Miller, and Matthew Izzi
September 22, 2015

September 22, 2015
© 2015 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law.
Citations@forrester.com or +1 866-367-7378
2
Keep Up With Customers By Continually Evolving Interactions
In the age of the customer it is the customer, not the brand, who dictates the terms of engagement —
and the growth of digital interactions has facilitated this shift in power.1
The proportion of US online
adults engaging with online ratings and review sites reached 26% in 2015; while at the same time
70% of them own a smartphone.2
What’s more is that today’s customer expectations from digital
interactions are constantly shifting. Companies must respond and keep up with these changing digital
behaviors, needs, and wants by becoming customer-obsessed and maximizing the level of intelligent
digital interaction.3
Professionals responsible for optimizing customer interactions — such as customer
insights pros, marketers, eCommerce professionals, and product managers — find themselves at the
center of initiatives to continually evolve the experiences targeted at customers as they interact with
digital touchpoints.4
Online Testing Is A Crucial Component Of Continuous Optimization
A few top digital intelligence practices are starting to build out continuous optimization strategies
with the goal of applying analytics-driven approaches at every possible opportunity to evolve and
leverage the understanding of customers as they interact across digital touchpoints.5
Online testing is
at the heart of these programs, applying statistical techniques to compare how variations of customer
interactions performed against a control group to determine the best treatment option. Continuous
optimization needs online testing, which provides unremitting and objective data-informed guidance to
help optimize matching the right visitor segments with the right experience options available through
interactions, such as those found within marketing tactics and product engagements.
Continuous Optimization Needs Drive Online Testing Criteria
At the heart of the continuous optimization vision is the desire to refine digital experience wherever
and whenever customers interact. And today’s customers interact digitally throughout the life cycle.6
To support continuous optimization, online testing practices need to scale across the enterprise,
supporting use cases and processes that coordinate multiple stakeholders, multiple types of digital
touchpoints, complex testing, and high test volumes. Today’s top online testing technology vendors
must offer online testing platforms and services that support/provide:
›› Campaign management. Delivering consistently high-value online testing at any scale demands
the seamless cooperation of multiple stakeholders — including the content creators, customer
experience (CX) designers, developers, marketers, and test analysts. Online testing platforms must
support processes such as campaign planning and scheduling, quality assurance, and concurrent
test interaction management to deliver high volumes of tests across multiple digital touchpoints.
And this must be done in a manner that minimizes damaging errors and maximizes the opportunity
of continuous optimization.

September 22, 2015
3
›› Multichannel deployment. Firms interact digitally with customers far beyond their traditional
websites. CI, CX, marketing, eCommerce, and product management pros are seeking to hone
digital interactions within mobile websites, applications, and social media — each requiring its
own experiment design, execution, and measurement capabilities.7
Enterprise-class online testing
platforms must now support the deployment of A/B and multivariate tests, and they must collect
and analyze rich interaction data within — and across — all of these new and emerging channels.
›› Self-service and usability. Centralized digital optimization teams can’t do it all — even with the
help of external partners.8
Without business support, online testing programs remain small or very
expensive to scale. CI pros must leverage the support of nontechnical participants (such as digital
marketers, product managers, and CX professionals) to be active participants in running and
managing much of the testing process. Online testing platforms must make complex, powerful,
self-service functionality safely available through well-designed user interfaces that users find
intuitive and easy to use.
›› Enterprise services. Mastering the practice of online testing is not a one-off project for
implementation and training. CI pros are constantly pressured to evolve their online testing programs
for digital business transformation initiatives, which seek to keep up with the way customers prefer
to interact with existing digital channels while extending interactions into new and emerging digital
channels.9
Many firms tell Forrester they prefer to manage their own online testing programs, but
with very few exceptions, these firms depend extensively on the vendor’s services team for success.
Online testing platform vendors must offer their customers a broad range of enterprise-class
professional services — from implementation to managed (or strategic) services — and help them
accelerate and maintain the adoption of large-scale, high-volume innovative testing programs.
›› Automation. Online testing programs that support continuous optimization can typically run
hundreds of tests per month. Manually intensive processes are just not scalable. Users are already
time and resource constrained, and having to manually run the entire testing cycle severely limits
the number and types of tests that can be run at any one time. Online testing platforms must have
the capability to make testing simpler, quicker, and more scalable. To achieve this, automation
is required across the entire test-and-learn cycle and must include tasks for test construction,
audience discovery, test validation, test execution, test control, data analysis, and acting upon
experiment results.
›› Digital intelligence. Digital intelligence strategies seek to bring together data to deliver a holistic
customer understanding and, ultimately, to optimize customer interactions.10
Delivering a complete
digital intelligence solution demands that practitioners leverage a range of technical components.11
Data-driven optimization platforms such as those for online testing are fundamental — but they are
only one component of digital intelligence. Online testing platform vendors must provide open and
standards-driven solution architectures that allow their solutions to easily work within a broader
ecosystem of digital intelligence technologies, including data management, analytics, and other
optimization capabilities such as behavioral targeting and recommendations tools.

September 22, 2015
4
Online Testing Platform Evaluation Overview
To assess the state of the online testing platform market and see how the vendors stack up against
each other, Forrester evaluated the strengths and weaknesses of top online testing platform vendors.
Evaluation Criteria Focuses On Scaling Online Testing Programs
After examining past research, user needs assessments, and conducting vendor and expert interviews,
we developed a comprehensive set of evaluation criteria. We evaluated vendors against 40 criteria,
which we grouped into three high-level buckets:
›› Current offering. To assess product strength, we evaluated each offering against seven dimensions:
campaign management; application usability, administration, and security; test deployment and data
handling; testing; dashboards, reports, and alerts; integration and native applications; and service
and support. For some criteria, we incorporated quantitative client feedback regarding the level to
which expectations are met, intuitiveness, and ease of use. These criteria include campaign preview,
interface, administration, A/B testing, multivariate testing (MVT) techniques, MVT automation,
dashboards and monitoring, reporting, and reports and alerts distribution.
›› Strategy. To assess the strength of the vendor’s strategy, we evaluated each vendor against
five dimensions: planned product enhancements, deployment and licensing options, customer
satisfaction, executive vision, and customer self-service. We also incorporated quantitative client
feedback when scoring criteria such as customer satisfaction and customer self-service.
›› Market presence. To benchmark the relative size of the vendors’ online testing businesses, we
evaluated each vendor company against six dimensions: global revenues, revenues and profit from
online testing, online testing installed base, online testing market trajectory, online testing target
market, and staff dedicated to online testing. We paid particular attention to the revenue, enterprise
clients, and the number of product development, professional services, and technical support staff
that are specifically assigned to each vendor’s online testing business.
Evaluated Vendors Are Committed To Supporting Enterprise-Class Online Testing
Forrester included eight vendors in the assessment: Adobe, HP, Maxymiser, Monetate, Optimizely,
Qubit, SiteSpect, and Webtrends (see Figure 1). Each of these vendors:
›› Supports a range of data-driven online testing techniques. To meet the requirements of modern
optimization practices, all evaluated vendors offer capabilities that support a range of testing
techniques. Each vendor offers a software platform that delivers both A/B and multivariate testing
capabilities, where both capabilities have a significant existing user base. Vendors that did not
support multivariate testing were excluded.

September 22, 2015
5
›› Offers a standalone online testing platform. When buying an online testing platform, users don’t
want to be locked into purchasing the vendor’s other platforms, such as content management,
eCommerce, or platforms that support other optimization techniques such as behavioral or
recommendations targeting. Each vendor in this evaluation offers a platform that delivers online
testing capabilities available as a standalone offering that can be used separately from other
software products and services.
›› Has a solid enterprise track record. This Forrester Wave online testing platform evaluation identifies
vendors that meet the demanding requirements of enterprise-class businesses, defined as firms
with at least $1 billion in revenue or 1,000 employees. Each vendor offers an online testing software
platform actively used by at least 40 enterprise-class online testing clients in live production.
›› Demonstrates an established online testing business. Online testing programs are long-term
initiatives that will continue to grow in prominence and importance within firms. Users need a partner
with proven business success and a commitment to the ongoing development and evolution of
their online testing business. Revenue is a signal of current success and an indicator of the level of
commitment to online testing in the long term. Each vendor in this evaluation has generated at least
$5 million in revenue from its online testing business within the most recently completed fiscal year.

September 22, 2015
6
FIGURE 1 Evaluated Vendors: Product Information And Selection Criteria
Vendor
Adobe
HP
Maxymiser
Monetate
Optimizely
Qubit
SiteSpect
Webtrends
Product evaluated
Adobe Target Premium
HP Optimost
MaxTest
Monetate for Optimization
Experience Optimization Platform
Deliver
SiteSpect
Webtrends Optimize
Product version
June 2015 release
June 2015 release
9.1
June 2015 release
June 2015 release
Deliver 3
8.3
8.7.1
Inclusion criteria
Each evaluated vendor offers a software platform that delivers both A/B and multivariate testing
capabilities, where both capabilities have a significant existing user base.
Each evaluated vendor offers a platform that delivers online testing capabilities that are available as a
standalone offering, which can be and is used separately from other software products and services
offered by the vendor, such as content management and eCommerce.
Each evaluated vendor offers an online testing software platform actively used by at least 40
enterprise-class online testing clients in live production.
Each evaluated vendor has generated at least $5 million in revenue from its online testing business
within the most recently completed fiscal year.

September 22, 2015
7
Vendors Vary In Functionality, Self-Service, And Blend Of Services
The evaluation uncovered a market in which (see Figure 2):
›› Adobe, Maxymiser, Optimizely, and SiteSpect lead the market. Leaders have achieved a good
balance of self-service and usability while at the same time building out the enterprise-class
functionality and services offerings needed to enable CI pros to deliver sophisticated online testing
campaigns at scale. While all Leaders have consistently strong offerings when compared against
most criteria evaluated, there are areas of significant differentiation among them, including native
applications, professional services, and agency and services partners. This cohort of vendors also
shows significant variation in their scores for market sizing and market trajectory criteria.
›› HP, Monetate, and Webtrends offer competitive options. Strong Performers have robust offerings
when evaluated against criteria such as support for campaign scheduling, campaign preview,
administration, and test deployment and data handling within fixed websites. Areas for improvement
for all vendors in this category include reports and alerts distribution, internationalization, and base
of customers installed. Scores for vendors differentiate significantly for criteria such as campaign
delivery and deploying tests and handling test data within applications.
›› Qubit is a Contender. As the only Contender in this Forrester Wave, Qubit performed well for
campaign access, campaign preview, application interface, overall customer satisfaction, market
trajectory, and staff dedicated to online testing. However, the vendor has a significant opportunity
for improvement across many criteria, including but not limited to multivariate testing, reports,
native applications, and internationalization.
This evaluation of the online testing platform market is intended to be a starting point only. We
encourage clients to view the detailed product evaluations and adapt criteria weightings to fit their
individual needs through the Forrester Wave Excel-based vendor comparison tool.

September 22, 2015
8
FIGURE 2 Forrester Wave™: Online Testing Platforms, Q3’ 15
Challengers Contenders Leaders
Strong
Performers
StrategyWeak Strong
Current
offering
Weak
Strong
Go to Forrester.com to
download the Forrester
Wave tool for more
detailed product
evaluations, feature
comparisons, and
customizable rankings.
Adobe
HP
Monetate
Optimizely
SiteSpect
Webtrends
Qubit
Maxymiser
Market presence

September 22, 2015
9
FIGURE 2 Forrester Wave™: Online Testing Platforms, Q3’ 15 (Cont.)
CURRENT OFFERING
Campaign management
Application usability, administration,
and security
Test deployment and data handling
Testing
Dashboards, reports, and alerts
Integration and native applications
Service and support
STRATEGY
Planned product enhancements
Deployment and licensing options
Customer satisfaction
Executive vision
Customer self-service
MARKET PRESENCE
Global revenues
Revenues from online testing
Online testing installed base
Online testing market trajectory
Online testing target market
Staff dedicated to online testing
Forrester’s
Weighting
50%
20%
10%
20%
15%
15%
10%
10%
50%
35%
10%
5%
30%
20%
0%
5%
40%
35%
5%
5%
10%
3.91
3.80
3.95
4.70
4.44
2.05
4.65
3.80
4.15
4.00
3.00
5.00
4.00
5.00
3.95
4.00
5.00
3.00
1.00
5.00
4.00
2.88
3.10
2.70
2.70
3.00
2.75
2.90
3.00
3.10
3.00
3.00
3.00
4.00
2.00
1.80
5.00
1.00
1.00
1.00
5.00
5.00
3.99
4.40
3.60
4.20
3.00
4.00
4.40
4.20
4.60
5.00
4.00
3.00
5.00
4.00
3.95
2.00
4.00
4.00
4.00
5.00
4.00
3.21
3.30
4.05
4.20
2.44
2.35
3.35
2.50
3.20
3.00
3.00
5.00
4.00
2.00
2.55
2.00
3.00
2.00
3.00
2.00
3.00
3.43
3.30
3.35
3.60
4.40
3.10
2.75
3.10
3.80
3.00
3.00
5.00
4.00
5.00
3.65
2.00
3.00
5.00
5.00
1.00
3.00
1.77
2.00
3.00
2.20
0.88
1.65
1.05
1.50
2.25
2.00
3.00
5.00
2.00
2.00
1.55
1.00
1.00
1.00
5.00
2.00
4.00
3.84
4.20
4.30
3.90
3.40
4.10
4.05
2.60
3.95
4.00
5.00
5.00
4.00
3.00
2.25
1.00
2.00
2.00
3.00
5.00
3.00
2.74
2.90
3.05
3.00
2.40
2.75
2.30
2.50
3.10
3.00
3.00
3.00
4.00
2.00
1.45
3.00
1.00
1.00
2.00
3.00
3.00
All scores are based on a scale of 0 (weak) to 5 (strong).
Adobe
HP
Maxymiser
Monetate
Optimizely
Qubit
SiteSpect
Webtrends

September 22, 2015
10
Vendor Profiles
Leaders
›› Maxymiser. The MaxTest online testing platform is part of a suite of optimization products that
includes testing, insights, and personalization. Maxymiser has solidified its position as a Leader
through a continued and focused commitment to online testing and currently has a competitive
offering with strengths in most criteria. Its key differentiators are support for campaign planning
and control, segment development and execution, professional services, and executive vision. The
platform will appeal to marketing and digital experience teams wanting a close vendor relationship
while being able to manage most of their optimization program via self-service. Maxymiser’s
customer feedback on ease of use and overall satisfaction was good, but not as good as for some
of the other evaluated vendors. With the recent announcement of Oracle’s agreement to acquire
Maxymiser, the vendor must now double up on its efforts to reassure its customers and seek to
meet and exceed customer expectations for platform usability and overall solution delivery.12
›› Adobe. Adobe has integrated its Test&Target platform within its Target solution, now called
Adobe Target Premium. This product brings together online testing functionality with a broader
optimization set, including those for rules- and machine-based targeting and recommendations.
The Target Premium solution remains itself a component of the Adobe Marketing Cloud. The online
testing platform offers strong capabilities within most criteria categories. Adobe’s key differentiators
include capabilities to deploy test on social sites and applications, A/B testing, the availability of
related native applications, and internationalization. The platform will appeal mostly to marketing
and eCommerce teams with strategies to optimize digital experiences through data-driven
personalization — especially those invested in other parts of the Adobe Marketing Cloud. Complete
support for continuous optimization means online testing platforms must support the needs of
optimization tactics outside of the traditional marketing domain, such as within product delivery,
product development, and customer support. To build on its Leader position, Adobe must position
its online testing platform to appeal more to a nonmarketing and eCommerce user base.
›› SiteSpect. SiteSpect’s A/B and multivariate testing platform is part of a suite of customer
experience optimization solutions that includes products for targeting and personalizing
experiences and optimizing site performance. SiteSpect shows strength across most criteria and
attains a leadership position through its continued approach to a tagless deployment model.13
Other areas of differentiation include winning results selection, administration, test deployment and
handling within fixed web domains, reports, and deployment and licensing options. The solution
will appeal to technically astute buyers from all market verticals seeking the benefits of this tagless
deployment model and wishing to run tens if not hundreds of tests per quarter. To retain and build
on its Leader position, SiteSpect must continue to build out the ability for campaign delivery,
improve the deployment of tests and handling of data within applications, and grow its agency and
services partner program.

September 22, 2015
11
›› Optimizely. Since its inception in 2010, Optimizely’s Experience Optimization Platform has quickly
become a disruptive force in the online testing space. This is due to the vendor’s continued
dedication to ease of use and self-service, while at the same time building out sophisticated
enterprise capabilities. Optimizely differentiates itself through above-average feedback from
customers on the usability and ease of use of its A/B testing, as well as its dashboard and
monitoring capabilities. The vendor also boasts the strongest agency and services partner
programs. Firms looking to kickstart their digital optimization practices by delivering quick value
through a self-service model will find this product compelling. To maintain its trajectory, Optimizely
must broaden its appeal to enterprise users by developing a robust professional services practice
and continuing to build out native capabilities for behavioral targeting and personalization.
Strong Performers
›› Monetate. Monetate for Optimization is part of a suite of online experience optimization
products that makes up a large part of the vendor’s complete offering. It includes online testing,
personalization, and recommendations. Monetate attains its position as a Strong Performer due to
its solid leadership team and its understanding of and focus on the retail market. Its offering will be
attractive to retailers with A/B testing requirements wanting to leverage their vendor’s professional
services team for implementing, managing, and running their optimization campaigns. To broaden
its appeal to other market verticals and challenge the Leaders, Monetate must build out its self-
service and multivariate testing capabilities and credentials.
›› HP. HP Optimost is part of the HP Marketing Optimization suite of products, which includes
solutions for digital engagement, customer communications, digital experience, and workforce
optimization. Given HP’s heritage, it’s no surprise that its professional services offering is a strength,
as it is both extensive and widely available. The platform will appeal to marketing and digital
experience teams within most market verticals that want the vendor’s professional services team to
own and run most, if not all, of the online testing program. To catch up to the competition, HP must
quickly develop in critical areas such as self-service, usability, and mobile app testing capabilities.
›› Webtrends. Webtrends’ online testing capabilities are part of the Webtrends Optimize suite, which
also includes modules for targeting and segmentation. Webtrends is a Strong Performer because
its strategy focuses on an automated optimization vision to create relevant and engaging digital
customer experiences. Practices from all market verticals wanting to adopt a managed services
approach to online testing should consider this vendor, as roughly 90% of its existing customers
use the vendor’s services team to implement, run, and managed most of the tests and output. To
advance on its current Strong Performer position and further broaden its appeal, Webtrends must
focus on improving the existing usability and self-service capabilities of its online testing platform.

September 22, 2015
12
Contenders
›› Qubit. While Qubit’s online testing capabilities are its primary revenue earner, it does offer other
marketing related products and services including those for personalization, recommendations,
voice of the customer (VoC), and analytics. The vendor is building out these capabilities on top
of a common data and analytics architecture. Qubit has focused on developing the A/B testing
capabilities along with associated segmentation and analytics. Retailers with browser-based A/B
testing requirements preferring to leverage their vendor’s professional services team to implement,
manage, and run their optimization campaigns may find the offering compelling. Qubit must advance
its capabilities and credibility within critical areas such as self-service, multivariate testing, and
mobile applications to compete with the other online testing vendors in this evaluation.
Supplemental Material
Online Resource
The online version of Figure 2 is an Excel-based vendor comparison tool that provides detailed product
evaluations and customizable rankings.
Data Sources Used In This Forrester Wave
Forrester used a combination of four data sources to assess the strengths and weaknesses of each
solution:
›› Vendor surveys. Forrester surveyed vendors on their capabilities as they relate to the evaluation
criteria. Once we analyzed the completed vendor surveys, we conducted vendor calls where
necessary to gather details of vendor qualifications.
›› Product demos. We asked vendors to conduct demonstrations of their product’s functionality. We
used findings from these product demos to validate details of each vendor’s product capabilities.
›› Customer reference calls. To validate product and vendor qualifications, Forrester also conducted
reference calls with three of each vendor’s current customers.
›› Customer reference online survey. To establish a level of customer satisfaction across a number
of categories including product functionality, flexibility, usability, services, and support, Forrester
also conducted an online survey of a maximum of 15 of each vendor’s current customers.
The Forrester Wave Methodology
We conduct primary research to develop a list of vendors that meet our criteria to be evaluated in this
market. From that initial pool of vendors, we then narrow our final list. We choose these vendors based
on 1) product fit, 2) customer success, and 3) Forrester client demand. We eliminate vendors that have
limited customer references and products that don’t fit the scope of our evaluation.

September 22, 2015
13
After examining past research, user need assessments, and vendor and expert interviews, we develop
the initial evaluation criteria. To evaluate the vendors and their products against our set of criteria,
we gather details of product qualifications through a combination of lab evaluations, questionnaires,
demos, and/or discussions with client references. We send evaluations to the vendors for their review,
and we adjust the evaluations to provide the most accurate view of vendor offerings and strategies.
We set default weightings to reflect our analysis of the needs of large user companies — and/or other
scenarios as outlined in the Forrester Wave document — and then score the vendors based on a
clearly defined scale. These default weightings are intended only as a starting point, and we encourage
readers to adapt the weightings to fit their individual needs through the Excel-based tool. The final
scores generate the graphical depiction of the market based on current offering, strategy, and market
presence. Forrester intends to update vendor evaluations regularly as product capabilities and vendor
strategies evolve. For more information on the methodology that every Forrester Wave follows, go to
http://www.forrester.com/marketing/policies/forrester-wave-methodology.html.
Survey Methodology
Forrester fielded its North American Consumer Technographics® Online Benchmark Survey (Part 1),
2015 in March 2015 of 61,222 US individuals and 6,642 Canadian individuals ages 18 to 88. For results
based on a randomly chosen sample of this size (N = 61,222 in the US and N = 6,642 in Canada),
there is 95% confidence that the results have a statistical precision of plus or minus 0.4% of what they
would be if the entire population of US online adults (defined as those online weekly or more often)
had been surveyed and plus or minus 1.2% of what they would be if the entire population of Canadian
online adults had been surveyed. Forrester weighted the data by age, gender, income, broadband
adoption, and region to demographically represent the adult US and Canadian online populations.
The survey sample size, when weighted, was 61,222 in the US and 6,638 in Canada. (Note: Weighted
sample sizes can be different from the actual number of respondents to account for individuals
generally underrepresented in online panels.)
Forrester fielded its North American Consumer Technographics Online Benchmark Survey (Part 2),
2015 in April 2015 of 32,188 US individuals and 3,067 Canadian individuals ages 18 to 88. For results
based on a randomly chosen sample of this size (N = 32,188 in the US and N = 3,067 in Canada),
there is 95% confidence that the results have a statistical precision of plus or minus 0.6% of what they
would be if the entire population of US online adults (defined as those online weekly or more often)
had been surveyed and plus or minus 1.8% of what they would be if the entire population of Canadian
online adults had been surveyed. Forrester weighted the data by age, gender, income, broadband
adoption, and region to demographically represent the adult US and Canadian online populations.
The survey sample size, when weighted, was 32,055 in the US and 3,051 in Canada. (Note: Weighted
sample sizes can be different from the actual number of respondents to account for individuals
generally underrepresented in online panels.)
Please note that respondents who participate in online surveys generally have more experience with
the Internet and feel more comfortable transacting online.

September 22, 2015
14
Integrity Policy
All of Forrester’s research, including Forrester Wave evaluations, is conducted according to our Integrity
Policy. For more information, go to http://www.forrester.com/marketing/policies/integrity-policy.html.
Engage With An Analyst
Gain greater confidence in your decisions by working with Forrester thought leaders to apply our
research to your specific business and technology initiatives.
Analyst Inquiry
Ask a question related to our research; a
Forrester analyst will help you put it into
practice and take the next step. Schedule
a 30-minute phone session with the analyst
or opt for a response via email.
Learn more about inquiry, including tips for
getting the most out of your discussion.
Analyst Advisory
Put research into practice with in-depth
analysis of your specific business and
technology challenges. Engagements
include custom advisory calls, strategy
days, workshops, speeches, and webinars.
Learn about interactive advisory sessions
and how we can support your initiatives.
Endnotes
1
We have entered the age of the customer, which Forrester defines as a 20-year business cycle in which the most
successful enterprises will reinvent themselves to systematically understand and serve increasingly powerful
customers. Digital business transformation is a key imperative for competing for and retaining this newly empowered
customer. See the “Winning In The Age Of The Customer” Forrester report.
2
Adults engaging with online ratings and review sites reached 26% in 2015. Source: Forrester’s North American
Consumer Technographics Online Benchmark Survey (Part 1), 2015.
Smartphone ownership has jumped to 70%. Source: Forrester’s North American Consumer Technographics Online
Benchmark Survey (Part 2), 2015.
3
To keep up with the customer digital intelligence analytics and optimization practices must capture, manage, and
analyze customer data to deliver a holistic view of the digital customer experience that drives the measurement,
optimization, and execution of digital customer interactions. See the “Supercharge Analytics With Digital Intelligence”
Forrester report.
4
Competing for today’s empowered customer with continually shifting expectations demands that firms continually
evolve and refine online experiences in order to consistently meet and exceed expectations. See the “Transform
Customer Experience With Continuous Optimization” Forrester report.

September 22, 2015
15
5
Forrester defines continuous optimization as “an analytics-driven approach that seeks to leverage every customer
interaction to evolve the understanding of the customer, which is in turn used to evolve and optimize current and
future customer experiences.” See the “Transform Customer Experience With Continuous Optimization” Forrester
report.
6
Firms must replace traditional funnel based approaches with customer life cycle-based approach. Doing so across
the enterprise guides changes to marketing strategy, organizational alignment, analytics, measurement, and marketing
technology. The customer life cycle describes six phases of customer engagement with firms: discover, explore, buy,
use, ask, and engage. See the “Use The Customer Life Cycle To Power Your Transformation To Customer Obsession”
Forrester report.
7
Even previously nascent optimization practices such as mobile online testing are gaining traction. When Forrester
surveyed 182 mobile executives, 19% of them stated that their firms were applying online testing tools and
techniques. See the “Win Mobile Moments With Digital Intelligence” Forrester report.
8
During Forrester’s research into continuous optimization practices, we found that the most successful large-scale
online testing programs were the result of a strong partnership with nontechnical users who were response for
different parts of customer journey. See the “Transform Customer Experience With Continuous Optimization” Forrester
report.
9
The pressure from increasing digital customer interactions is continually transforming digital channels and disrupting
business and operations functions. Digital not only accelerates the pace of change but also brings new opportunities
for firms that can embrace the technology fast enough. See the “The Digital Business Imperative” Forrester report.
10
Digital intelligence isn’t available through an all-in-one product suite. CI professionals must employ a combination
of technologies to collect, process, store, analyze, distribute, and act upon digital insights. See the “Supercharge
Analytics With Digital Intelligence” Forrester report.
11
A complete digital intelligence solution requires a range of technical capabilities spanning infrastructure components
such as databases, software applications for analysis and action, and integrations to cross-pollinate data and
functionality between systems. See the “Decipher The Digital Intelligence Technology Code” Forrester report.
12
In August 2015, Oracle announced that it has signed an agreement to acquire Maxymiser and the transaction has
closed. Source: “Oracle Buys Maxymiser,” Oracle press release, August 20, 2015 (https://www.oracle.com/corporate/
acquisitions/maxymiser/index.html).
13
Most online tests implemented by the evaluated vendors are deployed via browser script tags that typically refer to
a JavaScript files, which the browser then fetches, compiles, and executes. For the purpose of this report, Forrester
refers to this scenario as a tag-based deployment. Solutions that are not tag based are referred to as a tagless
solution with a tagless deployment model.

We work with business and technology leaders to develop
customer-obsessed strategies that drive growth.
Products and Services
›› Core research and tools
›› Data and analytics
›› Peer collaboration
›› Analyst engagement
›› Consulting
›› Events
Forrester Research (Nasdaq: FORR) is one of the most influential research and advisory firms in the world. We work with
business and technology leaders to develop customer-obsessed strategies that drive growth. Through proprietary
research, data, custom consulting, exclusive executive peer groups, and events, the Forrester experience is about a
singular and powerful purpose: to challenge the thinking of our clients to help them lead change in their organizations.
For more information, visit forrester.com.
Client support
For information on hard-copy or electronic reprints, please contact Client Support at
+1 866-367-7378, +1 617-613-5730, or clientsupport@forrester.com. We offer quantity
discounts and special pricing for academic and nonprofit institutions.
Forrester’s research and insights are tailored to your role and
critical business initiatives.
Roles We Serve
Marketing & Strategy
Professionals
CMO
B2B Marketing
B2C Marketing
Customer Experience
›› Customer Insights
eBusiness & Channel
Strategy
Technology Management
Professionals
CIO
Application Development
& Delivery
Enterprise Architecture
Infrastructure & Operations
Security & Risk
Sourcing & Vendor
Management
Technology Industry
Professionals
Analyst Relations
116868

Site spect 2015_forrester_wave (1)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (14)

Andere mochten auch

Andere mochten auch (15)

Ähnlich wie Site spect 2015_forrester_wave (1)

Ähnlich wie Site spect 2015_forrester_wave (1) (20)

Mehr von CMR WORLD TECH

Mehr von CMR WORLD TECH (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Site spect 2015_forrester_wave (1)