Marketing Mashup: Top takeaways from Web Opt Summit 2014

Ask questions and tell us what
you learned on Twitter!
#WebOpt14

Marketing Mashup: Top Takeaways
from Web Opt Summit 2014
#WebOpt14

Allison Banko
Reporter
MECLABS
@AllisonBanko
Daniel Burstein
Director of Editorial Content
MECLABS
@DanielBurstein
Today’s speakers

Related resources
• Web Optimization: Can you repeat your test results?
• Customer-centric Marketing: How transparency translates into trust
• Web Optimization: Ancestry.com improves conversion 20% by reducing
choice barriers
• Web Optimization: VacationRoost implements 2 testing methodologies to
boost total conversion rates by 12%
• The Boston Globe: Discovering and optimizing a value proposition for
content

Capture transferable insights
Web & Product
Marketing Manager
Accellos
Kait Vinson

Web & Product
Marketing Manager
Accellos
Kait Vinson
Need headline
with value
Intensify the
value of copy
Images of
product
CTA needs
more value
Need to focus
the two CTAs
“Register
Now”
implies no
value

Original

Original
New Design

Original
New Design
56%Increase in conversion

Testing and optimization cycle boosts conversion
Source: MarketingSherpa.com/ecommerce

Poll: How do you want this
webinar to go?
A. Snapshot: Let’s go fast and get through a lot
B. Take it Slow: Let’s dive deep into a few key discoveries

Michael Norton
Harvard Business School
Trust Through Transparency

Transparency
• Expressed a desire for larger government with more services
• Perceived the government to be doing a better job
• And even: Government often does a better job than it’s given
credit for

Use transparency to:
• Get more credit from your partner
• Get more credit from your coworkers
• Help your customers understand all of the services you provide
them

Takeaway: Times haven’t changed that much since third grade math class –
show your work to get full credit.

Ryan Hutchings
Director of Marketing
VacationRoost
How one company implements an entire testing
methodology every day
The Nuts and Bolts

What’s a good goal for bounce rates?
> 50%
40%-50% 30%-40%
< 30%

What’s a good goal for CTR to the next page
or step?
> 60%
50%-60%30%-50%
< 30%

Determining a “good” conversion rate
Source: MarketingSherpa.com/ecommerce
Conversion rate was most
frequently less than 15%, with
many responses below 1%

Control Treatment
Large Test: Lead forms 1
Security seals

Control
Fail – no lift.
Treatment

Control Treatment
Shorter form

Control
Fail – no lift.
Treatment

Control Treatment
Option to call

Control Treatment
Fail – no lift.

Large Test: Lead forms
19% Lift

Small Test: PPC landing pages
39

40

41
How can we
reduce friction and
anxiety on this
page?

42

43

44
427% increase in CTR

Takeaway: Testing your webpages is like golf – you wonder why you put yourself
through all of the “no lifts” until you finally get that win.

Test Snapshot: VMware
Background: SMB Solutions Page features multiple products.
Objective: To increase Try, Buy and engagement clicks.
Primary Research Question: Will elevating vSphere Essentials and vCenter
Log Insight above the fold with CTAs increase engagement?
Test Design: A/B test (Control vs. Variant B vs. Variant C) 33/33/33 split
Presenter: Cindy Lu
Title: Sr. Marketing Manager, Digital Marketing Strategy
Company: VMware

SMB: Identify and capitalize on quick wins
Control Variant B: vSphere Essentials Hero
Variant C: vCenter Log Insight Hero

Experiment 3: Results
What You Need to Understand: For a quick win, start with thinking about
where low-hanging fruit opportunities can be found.
Buy Online CTR
Conversion
Rate
Relative
Difference
Level of
Confidence
Control Page 1.56% - -
Treatment A – vSphere Essentials “Get Pricing” CTA 16.50% 956.16% 98%
Treatment B – vCenter Log Insight “Buy Online” CTA 6.73 330.49%
956% increase in clickthrough
Moving the CTA above the fold increased CTR 956%.

From one-size-fits-all to customized experience
Offer Page Transformation
Emily Titcomb
Sr. Manager, Product Marketing
Ancestry.com
Julia Babiarz
Sr. Interactive Art Director
Ancestry.com

13
Self-selectors Interrupted
Browsers

21
Self-selectors Interrupted Browsers

Experiment 2
New Template Design: Background
Objective: Replace the current high-performing Free Trial Deny page design with a
template-based design, so that we can then target, customize and optimize.
Primary Research Question: Can we succeed with a total redesign of the Free Trial
Deny page?
Test Design: A/B test – old design vs. new design
Research Notes: Since our “Interrupted Browsers” group is the
largest, we decided to test the Free Trial Deny page first.

24
Control Treatment: Template Format

!
What You Need to Understand: We failed on our first attempt at a template
design, but couldn’t isolate why.
Recipes Conversion % Lift
Control 2.57% -
Treatment 1 – New template design 2.38% -7.5%
7.5% decrease in conversion

Experiment 3
New Template Design Take 2: Background
Objective: Examine three different factors of the new template design in an
attempt to isolate exactly why the new template design failed.
Primary Research Question: Can we figure out what factors matter most for
conversion?
Test Design: 9-cell full-factorial test. Default vs. new template with three factors:
button size, price placement and imagery placement
Research Notes:

27
Button size:
Standard vs. large

27
Button size:
Standard vs. large
Price placement:
Near CTA vs.
far from CTA

27
Button size:
Standard vs. large
Price placement:
Near CTA vs.
far from CTA
Imagery placement:
Side vs. top

!
What You Need
to Understand:
Price placement
was the most
influential
element on
conversion.
Recipes Conversion
Control 1.93%
Standard Button, Price Top, Images Left 1.73%
Standard Button, Price Below, Images Left 1.94%
Standard Button, Price Below, Images Top 1.83%
Standard Button, Price Top, Images Top 1.66%
Large Button, Price Top, Images Left 1.69%
Large Button, Price Top, Images Top 1.66%
Large Button, Price Below, Images Top 1.77%
Large Button, Price Below, Images Left 1.80%
Conversion matched

29
Deny Page Consistency and Template Format: The winning treatment

29
Interrupted
Browsers:
Free Trial Deny

29
Interrupted Past
Free Trialers:
Hard Offer Deny

29
Upgraders:
Upgrade Deny

29
Upgraders:
Upgrade Deny
10% lift

Takeaway: Test multiple elements on your page to appeal to the multiple persona
types visiting your site – never stop discovering things about your visitors.

Test Snapshot: Publishers Clearing House
Objective: To convert unengaged visitors into engaged customers.
Primary Research Question: Will a simple, but attention-grabbing, header
convince unengaged visitors to play a game?
Test Design: A/B split test
Presenter: Michael Zane
Title: Sr. Director of Online Marketing
Company: Publishers Clearing House
Research Notes:

Test Design: Runs for 7 days
Unengaged repeat visitor
Message drops from top
after 2-second delay

Hey there, Speedy! Don’t leave yet! We have winners every day, MichaelB – Play
now and you could be one of them!

Unengaged message variations
Hey there, Speedy! Don’t leave yet! We have winners every day, [username] – play
now and you could be one of them!
FREE Money! Daily WINNERS Guaranteed! [username], how can you say no?
Is it a Bird? Is it a Plane? No, it’s [username] coming and going SO FAST! Slow down &
play awhile – it could win you BIG CASH!
Money! Money! Money! We’re giving it away, [username] – Play Now!
We saw you left without playing last time, [username]. Don’t you want to win?
Don’t leave without playing our $2,500.00 Instant Win Scratch Cards, [username]! A
BIG WIN could be just a few clicks away!

36% lift in engaged users
Compared to the control
Engaging the Unengaged: Results
Initial test shows strong results, but they are only valuable if it can be repeated.
Size % Engaged
Control – No real-time messaging 152,634 14%
Treatment – Real-time messaging with in-line banner 273,198 19%
Lift 36%
!

Experiment 1.2: Engaging with pop-up
Objective: To convert unengaged visitors into engaged customers using a
different presentation of the messaging.
Primary Research Question: Can we improve on results of round 1 by using
a pop-up instead of a banner?
Test Design: A/B split test
Experiment Research Notes:

Test Design: Runs for 7 days
Unengaged repeat visitor
Message pops up in the
center after 2-second delay

-1.3% decrease in engaged users
Compared to the control
Experiment 1.2 Results
Experiment 1.2 wasn’t working, so it was stopped after four days.
Size % Engaged
Control – No real-time messaging 94,270 20.8%
Treatment – Real-time messaging with lightbox pop-up 118,977 20.5%
Lift -1.3%
!

Test Snapshot: One Call Now
Presenter: Jacob Baldwin
Title: Digital Marketing Manager
Company: OneCallNow.com
Background: My curiosity led me to test the Request a Quote button copy on our
homepage.
Objective: To increase the number of people clicking the Request a Quote button.
Primary Research Question: Which button copy will produce the desired outcome?
Test Design: Sequential
Research Notes:

Request For Quote: Treatment
85

Side by side
86
Original Treatment
vs.

!
Experiment: Results
Having the right imagery matters!
Test Page Button Click Rate
Control – Stock image of older phone 1.98%
Treatment – Stock image of modern phone 3.88%
Relative Difference 95.96%
95% increase in conversion initiation
Led to an 82% increase in number of quote requests.

Peter Doucette
VP Consumer Sales & Marketing
The Boston Globe
How The Boston Globe moved beyond the landing page
Optimizing the Entire Business

Optimization beyond
the landing page –
Beyond marketing Reporting
Printing
Delivery

2 versions were tested
Did School Run by State Contractor Fail to Report Rape?

Live Headline Test
OR go to MarketingSherpa.com/LivePoll
Police Allege Sex Assault in Walpole Not Reported
Did School Run by State Contractor Fail to Report Rape?
Text GlobeB to 22-333
Text GlobeA to 22-333

My every intention is to push the kind
of boldness and investment that will
make The Globe a laboratory
for major newspapers across the
country.
http://www.bostonglobe.com/opinion/2013/10/27/why-bought-globe/QmFHhvRGFajQh1oMcAJ64M/story.html

Digital access price testing
Prices tested:
• Control price point = $3.99
• T1 price point = $0.99
Each treatment featured a 99¢ trial offer
for four weeks stepping to the full rate
noted in the treatments.

Digital Access Price Testing: Early results
36% increase in digital access subscriptions
The $0.99 price point drove 36% more conversions than the control.
Treatment Conversion Samples Successes Failures
Control $3.99 1.65% 17,788 327 17,461
Treatment 1 $0.99 2.18% 18,061 453 17,608
Treatment 2 $1.99 1.95% 17,946 400 17,546
Treatment 3 $2.99 1.80% 17,988 367 17,621
Treatment 4 $4.99 1.47% 18,017 301 17,716

Digital Access Price Testing: 52 week results
62% decrease in digital access revenue
The $0.99 price point drove 62% less conversions than the control.
Treatment Conversion Cancel Rate 52 Week Revenue Rev/Subscriber Rev/Visit
Control $3.99 1.84% 57% $45,189.15 $112.13 $2.09
Treatment 1 $0.99 2.51% 41% $17,172.54 $36.46 $0.77
Treatment 2 $1.99 2.23% 51% $26,048.01 $61.73 $1.18
Treatment 3 $2.99 2.04% 56% $31,611.53 $82.54 $1.44
Treatment 4 $4.99 1.67% 60% $39,386.81 $130.42 $1.80

Digital access accordion checkout testing

An accordion checkout
was successful for
ASSUMPTION

35% decrease in digital access revenue
The accordion checkout drove 35% less conversions than the control.

That everything we’ve done has been
successful is not reality. Some things
did not meet our expectations.
and we apply to the next test.
But we learn every time

What our audience thought:
At the end of the 30 minutes, B was
leading!

Takeaway: Optimization transcends marketing webpages – company-wide
optimization is a paradigm shift toward a culture of learning and constant
improvement.

Test Snapshot: Panel
Presenter: Lauren Pitchford
Title: Sr. Optimization Manager
Company: MECLABS
Background: Home delivery subscription has multiple delivery options such as daily,
weekend, weekday, Sunday only.
Goal: To determine if reordering the delivery options would have an effect on the
subscription mix and revenue.
Research Question: Which order of subscription options will yield the most revenue?
Test Design: A/B/C split test

Which price order generated the most revenue?

Results
! What You Need to Understand: By simply reordering the options displayed,
we were able to significantly affect the product mix.
Design Revenue per Visit
Price high to low .4
Price low to high .5
Popularity .4
Relative Difference 20%
20% increase in revenue per visit (93% LOC)
Treatment 1 increased revenue per visit by 20%.

Micahel Lykke Aagaard
Senior Conversion Optimization Consultant,
Founder of ContentVerve.com
ContentVerve ApS/Atcore ApS
How, When and Why Minor Changes
Have a Major Impact on Conversion

“It’s not about making extensive changes – it’s
about making the right changes.”

- When they address critical friction in the conversion process
- When they are made strategically to prominent and/or
mission-critical elements that directly impact the decision-
making process of the prospect
114
Small changes have a major impact

When do small changes have
little or no impact?

Hell no! Low motivation Hell yes! High motivationHmmm, maybe …

Landing
Page
Email
PPC SEO
Banner

38.46% more sign-ups
During the campaign, the treatment copy
sold 38.46% more memberships.

23.88% higher open rate
Statistical confidence: 95%

31.95% more sales via product pages
Treatment
Control

213.16% higher CTR

“It is not the magnitude of change on the ‘page’ that
impacts conversion; it is the magnitude of change in
the ‘mind’ of the prospect.”
– Flint McGlaughlin, MECLABS

Takeaway: Big impacts can come in the form of small changes, but these changes
must be made to the right elements.

One more test from Michael Aagaard …

100% privacy - we will never spam you!

Variant Conv. Rate Rel. Diff. Confidence
Control 2.03%
Treatment 1.65% -18.70% 96%
18.70% less sign-ups
The treatment with privacy policy decreased sign-ups.
Sample: 16,152 visitors
Conversions: 297

We guarantee 100% privacy. Your
information will not be shared.

19.47% sign-ups

Selling the Intangible
Tim Kachuriak
Chief Innovation & Optimization Officer
Next After
How e-commerce and subscription-based companies can
learn to sell intangible value like nonprofit organizations

The conference that
changed my life.

Experiment: My very first experiment
Experiment ID: NA00001
Location: Next After Research Library
Test Protocol: GWBPC01
Background: Email donor acquisition campaign for the George W. Bush Presidential
Center.
Objective: To increase clickthrough, increase revenue.
Primary Research Question: Does this MECLABS stuff really work?
Test Design: Single-factorial A/B split test
Research Notes:

Version A - Control Version B - Treatment

Version A - Control
141% increase in clickthrough rate
38% increase in revenue
Version B - Treatment

Optimized
Original
141% increase in clickthrough

!
Experiment: Results
What You Need to Understand: Small changes to the right elements can
produce dramatic results. By increasing the force of the value proposition,
we inspired more clicks and received more donations.
Email Version CTR
Control – “Make a tax-deductible contribution” 0.8%
Treatment – “Become a charter member” 1.9%
141% increase in clickthrough rate
38% increase in revenue.

Strategic framework
When it comes to increasing online revenue, there
are three primary strategies:
– Increase traffic
• Get more of the (right) people to show up
– Increase conversion rate
• Get more people to say, “Yes!”
– Increase average order value
• Get more people to say, “Heck yes!”

You already sell intangible value every day.

Experiment: The effect of value proposition on
average donation
Experiment ID: NA01011
Location: Next After Research Library
Test Protocol: THF024
Background: The Heritage Foundation is a think tank in Washington, D.C.
Objective: To increase donation conversion rate.
Primary Research Question: How does value proposition effect conversion rate?
Test Design: Radical redesign A/B split test
Research Notes:

74% increase in donor conversion
189% increase in average gift
TreatmentControl

74% increase in donor conversion rate
189% increase in average gift

• Headline: Conveys value and
vision
• Bulleted Copy: Provides reasons
why you should give
• Embedded Quotes: Bolsters
credibility
• Call-to-action and Donation
Form: Introduced at the end of
thought sequence when value has
been clearly communicated

!
Experiment: Results
What You Need to Understand: By increasing the force of the value
proposition, not only did we get more people to say, “Yes,” and give a gift,
but they said, “HECK YES!” and gave at a much higher level.
Email Version Avg. Gift
Control – Two column, donation form above the fold $37.50
Treatment – Single column, long form, donation below fold $108.30
189% increase in average donation
74% increase in donation conversion rate.

Takeaway: Optimization doesn’t always result in the most beautiful pages, but it
does result in the most user-friendly experiences.

Allison Banko
Reporter
MECLABS
@AllisonBanko
Daniel Burstein
Director of Editorial Content
MECLABS
@DanielBurstein
Thank you!

Marketing Mashup: Top takeaways from Web Opt Summit 2014

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Marketing Mashup: Top takeaways from Web Opt Summit 2014

Ähnlich wie Marketing Mashup: Top takeaways from Web Opt Summit 2014 (20)

Mehr von MarketingSherpa

Mehr von MarketingSherpa (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Marketing Mashup: Top takeaways from Web Opt Summit 2014