At MarketingExperiments, 2015 has been another great year full of new research and discoveries about what makes customers say 'yes.' It’s not easy to attend or watch every Web clinic, so we want to provide you with the opportunity to catch up on what you may have missed. In this 60-minute webinar, we looked back at our thousands of hours of research from 2015 to bring you five of the most surprising, actionable discoveries from the year. These simple principles will help you to quickly improve multiple aspects of your digital campaigns — from subject lines and calls-to-action to visitor engagement and mobile design. With 2016 fast approaching, now is the time to make sure that all of the elements of your digital marketing campaigns are as effective as they can be. Join us on December 9 to learn quick, actionable tactics for improving your marketing collateral in time for the new year.

1. The Top 5 Marketing Discoveries of 2015:
12 months of research in just 60 minutes

2. #WebClinic
Today’s Session
• 5 illustrative experiments of each discovery
• Cross-referenced recent examples
• Summary key points

3. We’re sharing on Twitter!
#WebClinic

4. #WebClinic
Today’s speakers
Jon Powell
Senior Manager of
Executive Research and
Development,
MECLABS Institute
Ben Huppertz
Senior Manager,
Research and Strategy
MECLABS Institute

5. Discovery #1

6. Experiment: Background
Background: A company offering a variety of dieting programs and memberships with the
goal of helping their audience lead a healthier lifestyle.
Goal: To increase landing page membership conversions on mobile and tablet devices.
Research Question: Which use of video will generate the highest conversion rate?
Test Design: Multi-treatment variable cluster split
Experiment ID: TP 11157
Record Location: MECLABS Research Library
Research Partner: (Protected)

7. Mobile side-by-side
*Anonymized
Control Treatment A Treatment B Treatment C
*Video is the same for the Control and Treatments

8. Design Conversion Rate Level of Confidence % Relative Change
Control 0.14% ---
Treatment A 0.19% 99% 34.4%
Treatment B 0.17% 97% 22%
Treatment C 0.16% 89% 14.5%
Experiment: Mobile results
Relative increase in conversion rate34%Treatment A resulted in a 34.4% relative increase in conversion rate.

9. Two questions:
1. What common thing did the three treatments do (in different
ways) to change the customer experience?

10. Two questions:
2. What did treatment A do that was different than B and C?

11. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
Top 5 Marketing Discoveries in 2015

12. Example Source: https://blog.bufferapp.com/power-of-story
From This To This
296%More Full-page reads
520%Average time on page

13. Discovery #2

14. #WebClinic
What would you test/change
to increase engagement?

15. #WebClinic
Experiment: Background
Background: A tourism commission seeking to enhance visitor interaction with their website
content in order to boost appeal for choosing their city over other destinations.
Goal: Increase visitor engagement with site content.
Primary Research Question: Which treatment will increase site engagement?
Test Design: A/B/C split test
Experiment ID: TP 11211
Record Location: MECLABS Research Library
Research Partner: (Protected)

16. #WebClinic

17. #WebClinic
Experiment: Treatments
*Anonymized
Treatment A
Treatment B
• Primary navigation tabs were
streamlined based upon visitor
interaction data with site content
over a calendar year.
• The most engaged-with category
content and secondary content were
prioritized and emphasized, while
less popular content was removed in
order to reduce friction.

18. #WebClinic
Design Clickthrough Rate Level of Confidence % Relative Change
Control 19.39% ---
Treatment A 26.26% 99% 35.39%
Treatment B 19.58% 55% 0.99%
Experiment: Results
Relative increase in navigation CTR35%Treatment A resulted in a 35.39% relative increase in navigation clickthrough rate.

19. #WebClinic
F
Design Lead Gen Rate Level of Confidence % Relative Change
Control 1.01% ---
Treatment A 1.36% 99% 34.93%
Treatment B 1.19% 91% 18.67%
Experiment: Results
Relative increase in total leads34%Treatment A resulted in a 34.93% relative increase in total leads.
What you need to understand:
Navigation content was curated based upon visitor behavior with the control nav.

20. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
Top 5 Marketing Discoveries in 2015

21. Example
From This To This
548%Conversions

22. Discovery #3

23. Experiment #1: Background
Background: A regional marketing commission.
Goal: To raise awareness of local activities and events, increasing number of travelers and
tourist spending in the area.
Research Question: Which subject line will generate the most opens and clickthrough?
Test Design: A/B split test
*Note: Test has been anonymized to protect partner confidentiality.
Experiment ID: TP 20211
Record Location: MECLABS Research Library
Research Partner: (Protected)*

24. Experiment #1: Version A
SUBJECT: Are you a fall foodie? See what’s
going on in Boston this November.

25. Experiment #1: Version B
SUBJECT: Clam Chowder, Lobster Rolls and More:
Calling all foodies for Boston’s best fall fun

26. Experiment #1: Which subject line will win?
Version A
SUBJECT: Are you a fall foodie? See what’s going on in Boston this
November
SUBJECT: Clam Chowder, Lobster Rolls & More: Calling All Foodies
for Boston’s Best Fall Fun
Version B

27. Experiment #1: Results
Relative increase in opens10%
Version
Open
Rate
Relative
Difference
Level of
Confidence
Version A 19.8% -- 99%
Version B 21.7% 10% 99%

28. Experiment #1: Results
Relative increase in clickthrough rate15%
Open
Rate
Relative
Difference
Level of
Confidence
Version A 2.7% -- 99%
Version B 3.1% 15% 99%

29. Experiment #2: Background
Background: A regional marketing commission.
Goal: To raise awareness of local activities and events, increasing number of travelers and
tourist spending in the area.
Research Question: Which subject line will generate the most opens and clickthrough?
Test Design: A/B split test
*Note: Test has been anonymized to protect partner confidentiality.
Experiment ID: TP 20219
Record Location: MECLABS Research Library
Research Partner: (Protected)*

30. Experiment #2: Version A
SUBJECT: Holiday shopping, historic tours, Pops
Orchestra concerts and more Boston traditions

31. Experiment #2: Version B
SUBJECT: Book now to save up to 30% on hotel
stays this December in Boston

32. Experiment #2: Which subject line will win?
Version A
SUBJECT: Book now to save up to 30% on hotel stays this December in
Boston
SUBJECT: Holiday Shopping, Historic Tours, Pops Orchestra
Concerts and more Boston Traditions
Version B

33. Experiment #2: Results
Relative increase in opens9%
Version
Open
Rate
Relative
Difference
Level of
Confidence
Version A 19.3% 9% 99%
Version B 17.6% -- 99%

34. Experiment #2: Results
Relative increase in clickthrough rate26%
Version
Click
Rate
Relative
Difference
Level of
Confidence
Treatment 3.1% 26% 99%
Control 2.5% -- 99%

35. What do winning subject lines have in common?
SUBJECT: Are you a fall foodie? See what’s
going on in Boston this November
SUBJECT: Holiday shopping, historic tours,
Pops Orchestra concerts and more Boston
traditions
SUBJECT: Book now to save up to 30% on
hotel stays this December in Boston
SUBJECT: Clam Chowder, Lobster Rolls and
More: Calling all foodies for Boston’s best
fall fun
Losing Versions Winning Versions
Why?

36. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
3. The impact of proper specificity in the email subject line
Many marketers worry about their ability to write persuasive copy, but the marketer’s art is
not persuasion; it is clarity.
Top 5 Marketing Discoveries in 2015

37. Experiment #1:
Loser
SUBJECT: Are you a fall foodie? See what’s going on in
Boston this November
SUBJECT: Clam Chowder, Lobster Rolls and More:
Calling all foodies for Boston’s best fall fun
Winner
• Confusing
• Vague

38. Experiment #1:
Loser
SUBJECT: Are you a fall foodie? See what’s going on in
Boston this November
SUBJECT: Clam Chowder, Lobster Rolls and More:
Calling all foodies for Boston’s best fall fun
Winner 10% opens
Specific
Subject Line
• Tangible
• Clear
• Confusing
• Vague

39. Experiment #2:
Loser
SUBJECT: Book now to save up to 30% on hotel
stays this December in Boston
• Salesy
• Ambiguous

40. Experiment #2:
Loser
SUBJECT: Book now to save up to 30% on hotel
stays this December in Boston
SUBJECT: Holiday shopping, historic tours, Pops
Orchestra concerts and more Boston traditions
Winner 9% opens
Specific
Subject Line
• Specific
• Valuable
• Salesy
• Ambiguous

41. Discovery #4

42. Experiment: Background
Background: Think tank soliciting year-end donations to meet annual fundraising goal.
Goal: To increase donations.
Research Question: Which email will generate the most revenue?
Test Design: A/B split test
Experiment ID: #616
Record Location: NextAfter Digital Research Library
Research Partner: The Heritage Foundation

43. Experiment: Version A
• This email was sent by Jim DeMint,
the president and most well-known
leader of The Heritage Foundation,
asking recipients for their support
with a year-end gift
• The tone of the email is formal and
professional

44. Experiment: Version B
• The email was sent by Christie
Fogarty, the foundation’s lesser
known Director of Membership
Programs, requesting donations
• The email uses a much friendlier,
informal tone

45. Version A
Experiment: Side-by-side comparison
Version B
?
Audience Question:
Which email will result in more donations?

46. Design
Revenue per
Visitor
Relative
Difference
Statistical
Significance
Version A – Jim DeMint $0.03 - -
Version B – New Sender $0.14 380.7%
% Relative Change: 380.7%
99%
Experiment: Results
Increase in Revenue381%The optimized version’s revenue increased by 380.7%.

47. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
3. The impact of proper specificity in the email subject line
Many marketers worry about their ability to write persuasive copy, but the marketer’s art is
not persuasion; it is clarity.
4. The growing effectiveness of an “everyday tone” in email
People still don’t buy from (or give to) companies, stores or websites; people buy from people.
Top 5 Marketing Discoveries in 2015

48. An Everyday Tone
• The winning email hypothesized
(and confirmed) that a lesser-
known sender with a shorter,
briefer message would imply a
more believable, personal email
send

49. An Everyday Tone
• The winning email uses language
that more closely aligns to the
personal nature of email
• In fact, it was designed to read like
a conversation and is true to the
style of the actual sender

50. An Everyday Tone
Version B
• The winning email is plain about
the purpose of the ask and how it
will be used
• In addition, the call-to-action is
very clear and direct

51. Example
60%In Click-Rate
159%In Completion Rate
From This To This

52. Discovery #5

53. Experiment: Background
Experiment ID: TP11081
Record Location: MECLABS Research Library
Research Partner: Protected*
Background: A regional marketing commission.
Goal: To raise awareness of local activities and events, increasing number of travelers and
tourist spending in [City].
Research Question: Which sign-up page will generate the most response?
Test Design: A/B split test
*Note: Test has been anonymized to protect partner confidentiality.

54. The control

55. Control
The control

56. The treatment

57. The treatment

58. Control Treatment
Side-by-side

59. Experiment: Results
Treatments Conversion Rate
Percent Relative
Change
Level of
Confidence
Control 56.4% -- --
Treatment 37.3% -33.94 99%
34% Decrease in conversions

60. Control Treatment
How did brand standards affect the user experience?

61. 5. The impact of brand standards on end-behavior
We must ensure that our defined elements of our brand help, rather than hinder, people when
it comes to making the right decision.
Top 5 Marketing Discoveries in 2015

62. Example
From This To This
40%Subscriptions

63. Summary

64. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
3. The impact of proper specificity in the email subject line
Many marketers worry about their ability to write persuasive copy, but the marketer’s art is
not persuasion; it is clarity.
4. The growing effectiveness of an “everyday tone” in email
People still don’t buy from (or give to) companies, stores or websites; people buy from people.
Review: Top 5 Marketing Discoveries in 2015

65. 5. The impact of brand standards on end-behavior
We must ensure that our defined elements of our brand help, rather than hinder, people when
it comes to making the right decision.
Review: Top 5 Marketing Discoveries in 2015

66. Live Optimization

67. URL: https://www.hesonline.com
Primary Audience: B2B
Page:
Health Enhancement Systems

68. URL: http://www.north-american-waterway.com/
Primary Audience: B2C
Page:
North American Waterway

69. URL: http://www.alpineclubofcanada.ca/
Primary Audience: B2C
Page:
Alpine Club of Canada