Do you feel like you’ve tried just about every type of design and copy and still aren’t seeing results? What can you do to completely challenge your testing paradigm? Is it possible to leverage qualitative feedback to produce quantitative results? If so, how? At Email Summit 2015, we brought these challenges from one of your peers – the marketers at LifeWay – to the entire audience of Marketers at Summit. This experienced peer group provided suggestions for improving LifeWay’s email marketing. The team from MECLABS, MarketingSherpa’s parent research institute, then worked with LifeWay to test this qualitiatve feedback and will present key lessons learned from this experimentation during this unique live event we’ve dubbed a MarketingSherpa Web clinic.

1. Special Live Test Clinic:
The MECLABS team reveals the results of LifeWay’s email
copy test

3. Today’s speaker
Flint McGlaughlin
Managing Director and CEO, MECLABS Institute
@Flintsnotes

4. Case Study: Background
Background: A Web tool that lets you quickly create and customize Bible studies.
Goal: To increase clickthrough rate.
Research Question: Which email will produce the highest clickthrough rate?
Approach: A/B split test
Brand: SmallGroup.com
Location: MarketingSherpa Research Library
Protocol ID: Pending

5. Case Study: Background
Julie Langmade
Manager, Lifeway.com Merchandising
Lifeway Christian Resources

6. Case Study: Background
• Julie brought an email
to Email Summit 2015
to gain live feedback
from her peers.

7. Case Study: Original email
Subject Line:
The fast, free way to create custom
discipleship content.
Campaign Objective:
Increase free trial starts
About the List:
Pastors, small-group leaders, paid
church staff in general

8. Case Study: Original email
Subject Line:
The fast, free way to create custom
discipleship content.
Campaign Objective:
Increase free trial starts
About the List:
Pastors, small-group leaders, paid
church staff in general
How would you
improve this email?

9. Audience suggestions
1. “The headline is meaningless … it needs to be in
sentence form.”
2. “That image doesn’t give me any context for the
content.”
3. “The sub-header is not related to the content.”
4. “Strip out that whole last paragraph … it should
just be boiled down to ‘Try the Free Preview.’”
5. “Make [the button look] more like a button. It
looks more like a banner.”
6. Change button text to “Learn more”
7. Change button text to “Learn about our free
two-week preview”

10. Case Study: Treatment
• A treatment was created using
most of the feedback (but not
all of it).
• To see the full recording go
to: http://goo.gl/vJSTlt

11. Case Study: Side-by-side
Control Treatment

12. Case Study: Results
Email Design Total Click Rate Button Click Rate
Original 0.8% 0.2%
Treatment 1.1% 1.0%
% Relative Change: 42% 358%
Increase in Button Clicks358%
The email design that utilized suggestions from the live audience achieved significantly
more total clicks and button clicks.

13. Case Study: Results
Email Design Total Click Rate Button Click Rate
Original 0.8% 0.2%
Treatment 1.1% 1.0%
% Relative Change: 42% 358%
Increase in Button Clicks358%
The email design utilizing suggestions from the live audience achieved significantly
more total clicks as well as button clicks.
How did we develop a treatment from the
live feedback that achieved this result?

14. Case Study: Results
• Often, we generate unnecessary costs by conflating the objective of an
email with the objective of a landing page. The goal of most emails is simply
to get a “click.”
• We must challenge our emails with this question: “Is there a single word or
piece of content on the page that does not help to achieve a click?” Every
unnecessary piece of content is waste and reduces your chance of
achieving a click.
Key Principles

15. Which of these suggestions can help us get the
click and which won’t?
1. “The headline is meaningless … it needs to be in sentence form.”
2. “That image doesn’t give me any context for the content.”
3. “The sub-header is not related to the content.”
4. “Strip out that whole last paragraph … it should just be boiled
down to ‘Try the Free Preview.’”
5. “Make [the button look] more like a button. It looks more like a
banner.”
6. Change button text to “Learn more”
7. Change button text to “Learn about our free two-week preview”

16. What does this look like when translated into the
treatment?
1. The headline is made into a complete,
understandable sentence.
2. The image size is reduced and adjusted.
3. The sub-header is now more specific to the
content .
4. The whole last paragraph is replaced with
relevant bullet points.
5. The button is trimmed down and given a drop
shadow so it resembles a button.
6. The button copy is changed to a low-
commitment action.

17. Key lessons learned
• Collaboration is good, but not all feedback is necessarily helpful.
• To ensure that peer review helps achieve your marketing collateral
objectives, you must discipline the feedback with a framework/heuristic.
• For example: eme = rv (of + i) – (f + a)
• After you’ve applied the framework, you must discipline the analysis with a
hypothesis/experimentation cycle, guided by a robust design of
experiments.
• To learn about “Humanizing Email,” go to: http://goo.gl/RLh8e8

18. Live optimization

19. Submitted by:
Andrea H. (Leadables)
Campaign Objective:
Led gen
Live optimization

20. Submitted by:
Andrea H. (Leadables)
Campaign Objective:
Led gen
Live optimization

21. Submitted by:
Caroline L. (Intersil)
Campaign Objective:
Led gen
Live optimization

22. Live optimization
Submitted by:
Ana P. (Far & Wide Collective)
Campaign Objective:
Product sales