6. What should you test?
Almost anything...
● Subject lines
● Email content
● Site architecture
● Site usability
● Donation forms
● Landing pages
● Facebook ads
● Google Adwords
● Images
7. Know your goal
Decide what you’re trying to improve
● Open rates
● Click rates
● Form completion
● Traffic
8. How to test
1. Form your hypothesis — what do you predict will
happen?
2. Create your variants
3. Decide sample size and test duration
4. Test!
5. Interpret results
11. Easy signifance testing tools are online, such as http://butisitstatisticallysignificant.com/
12. Mind your p-values
● Check for statistical significance, a measure of
confidence in your results
● The lower the p-value, the more confident you
can be that your results are due to your test
variables
● The standard measure is 0.05 or lower, so you
can be at least 95% confident your hypothesis is
correct
13. p-value is less than
0.05 so you can
accept your
hypothesis - hooray!
16. Testing does a lot… but not everything
● AB testing can’t tell you why — but you can
make educated guesses
● It’s not suitable for large redesigns as it won’t be
clear what influenced the results
● Testing won’t tell you if you’re addressing the
right problem, so clear goals are important
● Don’t let testing dictate creative design
decisions