Introduction to A/B Testing in E-mail Marketing
•Als PPT, PDF herunterladen•
0 gefällt mir•548 views
Presentation introducing why e-mail is important for digital marketing in 2014. It explains what A/B testing is and walks through, step by step how to run your own A/B test using Mailchimp. Originally recorded as a webinar. See http://youtu.be/S0X35aob68Y for full webinar
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Introduction to A/B Testing in E-mail Marketing
Ähnlich wie Introduction to A/B Testing in E-mail Marketing (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Introduction to A/B Testing in E-mail Marketing
- 1. By Michaela Simpson Student MSc (Management) Digital Marketing DCU, 2013/14
- 4. E-mail budgets are increasing as Return on Investment explodes to 2,500% (DMA, 2014)
- 6. DMA, 2014
- 12. 1.1.HypothesisHypothesis 2.2.One VariableOne Variable 3.3.Clear Success MetricClear Success Metric 4.4.Volume and Statistical SignificanceVolume and Statistical Significance 5.5.RandomisationRandomisation 6.6.Always be Testing but Apply Common SenseAlways be Testing but Apply Common Sense 7.7.DocumentationDocumentation (Search Engine Watch, 2012)(Search Engine Watch, 2012)
- 23. Sent a test e-mail to yourself and to other colleagues Checked these on different browsers and devices, including mobile devices Checked all links within the e-mail are working and relevant Checked spelling and grammar
- 24. And when you have carefully checked all of that….
- 27. Compare your open rate, click rate and bounce rate with the industry. http://mailchimp.com/resources/research/email-marketing-benchmarks/
- 28. Compare your open rate and click rate with your own average.
- 29. Be aware of how many of your customers are reading their e-mails on their mobile device.
- 30. E-mail is not dead! When exploring A/B testing, explore Dan Zarrella’s work. Before setting up your A/B test follow the rules Only send your e-mail when you’ve checked and re-checked your campaign. Be anal! Measure against industry averages and your own averages.
Hinweis der Redaktion
- Hi everyone! I’m Michaela and I’m a current Masters Student in Digital Marketing in Dublin City University. Over the year I’ve learned quite a bit about e-mail marketing, but of course, when you come to actually doing it, it still feels a bit daunting. So I’ve put together this webinar to introduce the basics A/B testing in E-mail Marketing. I’ll walk through, step by step, exactly how you run an A/B test using Mailchimp.
- But, first things first…
- Well, contrary to popular belief, e-mail is far from dead. There is plenty of evidence that e-mail is one of the most successful and cost effective channels a Digital Marketer can use, however, in many companies, e-mail is not seen as old fashioned and just not trendy in digital marketing today.
- But the one thing bosses can’t argue with is being presented with ROI. This is a from research carried out by the Direct Marketing Association in the UK in 2013. They found that E-mail ROI has hit an unbelievable 2,500%. So it’s not surprising that respondents indicated that they plan to invest further in E-mail marketing.
- The ROI from e-mail marketing is also increasing. In 2012, it was found to return £21 pounds for every £1 pound spent and that was up to £25 pounds for every £1 pound spend in 2013. It’s very important to note though, that e-mail is not magic. It needs care and attention in order to get this kind of success out of it. As the DMA say here, 77% of ROI comes from segmented, targeted and triggered campaigns. Customers who have signed up to your newsletter, don’t really care about what you have to say unless it is of some sort of value to them. Segmenting your e-mail lists is key to ensuring that your customers are receiving only email content which is of relevance to them.
- One final interesting point from the DMA research. They found the best performing emails in 2013 to be the regular old newsletter, followed by welcome messages and customer surveys. I think that’s an interesting point to keep in mind. Particularly the customer surveys. If you have customers who are willing to answer a survey for you, not only do you gain really valuable feedback, but you also gain great material to possibly form the basis of a really insightful blog post or white paper that can feed into your content marketing.
- Getting started with A/B testing. So, if you’re not quite sure what an A/B test is, I would describe it simply as creating 2 random groups. Group 1 is a “Control” group and group 2 is a “Treatment” group. Your treatment group is going to receive some sort of special treatment and then you will see if it has had any effect, based on the difference in outcomes between them and your control group.
- Getting started with A/B testing. So, if you’re not quite sure what an A/B test is, I would describe it simply as creating 2 random groups. Group A is a “Control” group and group B is a “Treatment” group. Your treatment group is going to receive some sort of special treatment, like maybe you want to try out phrasing your subject line as a question and seeing if that has any impact on the open rate. If you keep everything else in the e-mail the same, the time you send it, the content of the e-mail and so on, you will be quite confident that if there is any outrageous variation between the two groups, that it is very likely down to the thing you were testing, in this case the subject line. So once you have a handle on A/B testing, in my opinion, the next thing you should do is
- Google Dan Zarrella. He is what they call, a social media scientist. He takes different things like Twitter, Facebook and E-mail marketing and methodically tests the hell out of them. He asks simple questions like, does saying “please retweet” increase the likelihood of being retweeted? By the way , it turns out it does!
- For instance, this is from Zarrella’s Science of E-mail marketing where he tests the open rates and click rates by day. He had access to a large collection of e-mail data from Hubspot and found that open rates and click rates on weekends show an increase compared to midweek. This doesn’t mean that this would be the same in every case, but it might be something which you’d want to carry out an A/B test on.
- He also tested to see if the time of day made any difference to e-mail open rates and click rates. He got this this indication that sending e-mails early in the morning seemed to have the most favourable open rates and click rates.
- I’ll quickly outline the rules of A/B testing before walking through, step by step, how to carry out your own A/B test using MailChimp. Firstly, you need a hypothesis where you should write down a short sentence that summarizes what you’re trying to prove. Secondly, and as I mentioned a few slides back, an A/B test can only be successful if you are testing only one variable while everything else stays constant. Third, you need to choose which metric is going to show which group wins. I’m going to mention all of these again when we’re doing the walkthrough, so don’t worry if you’re not really sure what this right now. Fourth, make sure that the sample that you are using for the test is large enough to be statistically significant. Fifth, make sure the groups are selected randomly. When you are using MailChimp – it looks after this random splitting for you. Like Zarrella, always be testing, but try to keep your head screwed on and think carefully about what is worth testing. Finally, once you’ve put the work in to carrying out these tests, make sure to document it so that the findings aren’t wasted. Again, Search Engine Watch, makes a good suggestion saying why not make a blog post out of your findings – this way you document your work and you feed your content marketing channel!
- Ok, so now I’m going to walk through, step by step, how to run an A/B test on MailChimp. They actually make it really simple so you shouldn’t have any trouble with it, but if you haven’t tried it before, this should help you if you get stuck anywhere.
- So when you log into your Mailchimp, you’ll arrive at your dashboard as usual. You just go up to Create Campaign in the top right.
- That’ll bring you to a screen like this, where you have a choice to select a regular campaign, or down here you have you’re A/B Split Campaign. Click select.
- If you’re Mailchimp account is very new, you may not see this screen. They don’t seem to allow A/B tests until there is some data in the account from previous campaigns, probably showing them that you have made yourself familiar with the basic features of mailchimp before delving into the more complicated stuff. So on this page, you select what you would like to test. So in our case we’re going to test two different subject lines. What this means down here is that the test is going to be sent to 20% of your total list initially before picking the winner and sending on to the remaining 80% of your list. So in my case, my list had about 2,500 emails in it so 250 would have received email A and 250 would have received email B. Like I mentioned previously, Mailchimp picks these test segments randomly for you. Finally on this page, you choose how long we want Mailchimp to wait before making its decision on a winner and we’ll select one day here. Click next
- Next you are prompted to choose your recipients. All of your lists will be listed here and you pick the one you want to send this campaign to.
- Next you set up your campaign info. We said we were going to test two different subject lines and see if that has any impact on open rates. Here, I’ve taken inspiration from one of Zarrella’s findings where he found that contrary to what was often thought, posing a question in an e-mail subject line, actually correlated with lower open rates. But this may not be the case for everyone so we’ll test it out and see how our list responds to a question in the subject line.
- The next steps are where you will spend most of your time which is designing your e-mail content.
- And once that’s completed, you’ll be ready to send your e-mail.
- But wait, if I’ve learned anything the hard way it’s this…
- When the test is complete, Mailchimp presents the results like this. You can see all of the different metrics and how each performed. Because we were testing subject lines, the best metric to determine the winner is open rate. It’s the best indicator of how a subject line has performed as it is the most prominent feature a user sees before deciding whether or not to open the mail. In this case, for another test I ran, Group B was the winner with 40% opening the mail versus 31%.
- Other ways to guage the performance of your campaigns is to compare your metrics to the industry averages. Mailchimp has a great resource of industry averages for email marketing campaigns where you can see how your campaigns are measureing up.
- You should also check how each campaign measures up to your own average metrics for that list. You’ll be able to see whether this campaign did better than usual and perhaps get some hints as to why that was. All of these insights can be used to inform your decisions for your future campaigns
- Finally, you should make sure not to neglect mobile users. If you check your stats for how many of your customers are reading their emails on a mobile device, you might be surprised. If this is a large proportion, you should make sure that all of your campaigns are optimised for mobile and read well on the smaller screen.
- So just a quick recap of the key takeaways of this webinar. E-mail is not dead! For digital marketers who are giving it the time, attention and investment it deserves, it is returning hugely impressive ROI. Careful segmentation of your customers is the key to seeing the kind of results we saw earlier. When exploring A/B testing explore Dan Zarrella’s work. In my opinion, it’s a great place to start as he presents this kind of thing in a nice simple way for beginners, and helps experienced e-mail marketers see the wood for the trees. Before setting up your a/b test, take note of the rules I went through and allow these to guide you in forming a hypothesis and deciding how to choose your winner. Only send your e-mail when you’ve checked and re-checked your campaign. If ever you are going to be anal, be anal before you send out an e-mail campaign! Measure your results against industry averages and your own averages to get more insight into how your campaign has performed.
- So thanks a million for watching. If you have any questions or want to connect, you can get my on my Twitter @MichaelaTweets, my LinkedIn michaelasimpson1 or you can follow my digital marketing blog at michaela85.wordpress.com. Thanks everybody!