A/B Testing in Email Marketing -Ideas & Examples-

A/B TESTING FOR EMAIL MARKETING:
IDEAS AND EXAMPLES
By Elena Chailazopoulou
LinkedIn

Remember what we’ve seen so far..
1. Subject line categories like Funny, Curious or Direct
have completely different impact on different
audiences.
2. ESPs like Mailchimp share their knowledge over best
practices BUT still all agree that no size fits all.
Sometimes the most boring, direct subject lines get the
highest Open Rate.
3. Even from a single google search one can realize
that “Best Practices” vary from as little as 5 to as many
as 150!
Text to image ratio
WHY TESTING?
Elena Chailazopoulou | eBusiness Consultant

In 2012 Obama’s team led a giant
data mining and micro-targeting
operation of A/B testing.
Result $690 million

Tests included subjects,
body, footer!, sentiment,
profile background analysis,
personalization and many
other non-email tests.
Testing was calculated to
have brought $200 million
in additional revenue !!

WHY TESTING?
Plus remember.. the prettiest is
not always the best

WHY TESTING?
The Lean Startup by Eric Ries
It’s the only way to improve

WHAT IS TESTING?
Email Testing is the process of sending multiple versions of a newsletter (if only two it’s
called A/B testing, if more than two it’s called Multivariate testing) to segments of your
total database in order to determine which one outperformed the other based on your
preferred KPI.
So, you must always have one CONTROL and one or more VARIANTS. It’s best if you limit
your testing variables at one at a time so as to best assess the result.
for example if you are testing a symbol on your subject line, try keeping the tone similar to the average tone of your previous
newsletters (e.g. funny or direct)

SO HOW DOES IT ACTUALLY WORK?
After sending your control and test variant to a percentage if your database, you determine
the winner based on a KPI. That can be either the Open Rate or the Click Through Rate.
The winning email is then sent to the remaining percentage of your email database.

WHAT CAN WE TEST?
From Field Content Delivery timeSubject line

WHAT CAN WE TEST?
From Field Content Delivery timeSubject line Mailchimp blog: Feb-May 15’
 30% of these campaigns use more
than 1 emojis
 We don’t see a consistent positive or
negative impact
 Used best to convey feelings
o Symbol

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
Μην το χάσεις!, Do not miss your chance
of winning!
Τελευταία κομμάτια.., Last items
Λήγει σε 4 ώρες!, Expires in..
Weekend offer-μόνο γι’ αυτό το ΣΚ
Limited time only, bonus gift card offer!

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
Nothing Lasts Forever! Βρες Μοντέρνο Top σε Μαύρο Χρώμα μόνο 5,5€!
Ισχύει μόνο ως 19/6 - 09:59
Vs
Up to 20% off Bikes & Scooters!

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o CTA verbs & position
Specific
Action oriented
------------------------------------------------
----

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o Loss aversion
Εσύ θα μείνεις εκτός?
Do your landing pages pass this test?

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o Loss aversion
o Tone
o Direct
o Fun
o Curious
o relationship

WHAT CAN WE TEST?
Subject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o Loss aversion
o Tone
o Direct
o Fun
o Curious
o Relationship
o Pre-header text

WHAT CAN WE TEST?
From FieldSubject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o Loss aversion
o Tone
o Direct
o Fun
o Curious
o Relationship
o Pre-header text
o Name
o Email

WHAT CAN WE TEST?
From Field ContentSubject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o Loss aversion
o Tone
o Direct
o Fun
o Curious
o Relationship
o Pre-header text
o Name
o Email
o Text-image ratio
o Banners
o Product images
o CTAs
o Number
o Color
o Placement
o Links vs buttons
o Length
o Using (price)ribbons
o Using testimonials
o Changing image Alt-
texts
o Navigation menu

WHAT CAN WE TEST?
From Field Content Delivery timeSubject line
o Symbol
o Urgency
o Long vs short
o %, Incentives
o Brands
o Personalization
(name, email)
o Loss aversion
o Tone
o Direct
o Fun
o Curious
o Relationship
o Pre-header text
o Name
o Email
o Day of the week
o Time of day
o frequency
o Text-image ratio
o Banners
o Product images
o CTAs
o Number
o Color
o Placement
o Links vs buttons
o Length
o Using (price)ribbons
o Using testimonials
o Changing image Alt-
texts
o Navigation menu

Define sample size
 No of total subscribers ? (if less than 10K maybe 50%
split is better)
 TOTALLY RANDOM –unbiased?
 Statistical significance
 Get the Percentage out of total db
To how many should I send as a test?
http://www.surveysystem.com/sscalc.htm#one

Define Testing Frame
For how long shall I run the test?
 First think about the overall timing that you want your subscribers to get the newsletter
for example a limited set-time offer is probably not the best case for testing.
 Set a maximum
 Judge based on your current open rate curve..

Measure
 The test data –this is right after the end of the testing frame
 Define the final results’ measurement period (e.g 3 days but be consistent)
 Not all results are statistically significant (about 30% of total is fine)
 Consider avg. revenue per user as a KPI
Elena Chailazopoulou | eBusiness Consultanthttp://drpete.co/split-test-calculator

Sample Excel format for your measurements
Date
segment Opens OR Clicks CTR Recipients Opens OR (unique) CTR (unique) Revenue trans. avg. rev./user
12/2/2015 Control: Subject One 4.000 129 3,23% 16 12,40%
Variant: ▶ Subject Two 4.000 112 2,80% 19 16,96%
Variant: ▶ Subject Two 4.000 179 4,48% 19 10,61%
segment Opens OR Clicks CTR Recipients Opens OR (unique) CTR (unique) Revenue trans. Avg. rev./user
Variant: Don’t miss! Subject Two 4.000 132 3,30% 8 6,06%
Variant: Don’t miss! Subject Two 4.000 77 1,93% 5 6,49%
Subject
YES
NO
50.000 7.743 15,5% 16,85% 3.500 € 73 0,070 €
NO
YES
Testing Type: Urgency
50.000 8.953 17,9% 14,80% 4.000 € 66 0,080 €
106 0,110 €
50.000 8.683 17,4% 17,08% 6.000 € 66 0,120 €
Test data -measure right afte the testing frame period Winner Data after 3 days Stat.
significant?Testing Type: Symbol
50.000 7.485 15,0% 16,01% 5.500 €

By Elena Chailazopoulou
LinkedInstay connected

A/B Testing in Email Marketing -Ideas & Examples-

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Empfohlen

Empfohlen (20)

A/B Testing in Email Marketing -Ideas & Examples-