This document discusses A/B testing and the A/Bingo plugin. It provides instructions on installing A/Bingo and using it to conduct A/B tests on signup forms. Key points covered include how A/B testing helps avoid relying solely on opinions and seek quantitative answers, how A/Bingo makes it easy to define tests and track conversions, and how the dashboard shows test significance and progress. Limitations mentioned are a lack of cohort selection and weighting of alternatives.

1. A/B and A/Bingo
Rob Di Marco

2. A/B Testing
• Test (Version A and Version B)
• Conversion Event

3. Who Cares?

4. A/B Helps If...
• Seeking quantitative answer over taste
• Looking to avoid HIPPO
• HIghest Paid Persons Opinion
• Have a clear test/conversion point

5. A/Bingo is..
• Methods for ActionController::Base and
ActionView::Base
• ActiveRecord objects
• Experiment
• Alternative
• Dashboard

6. Install A/Bingo
‣ script/plugin install git://
git.bingocardcreator.com/abingo.git
‣ script/generate abingo_migration
‣ rake db:migrate
‣ script/generate controller
abingo_dashboard

7. Before Using...
• Add a before_filter that sets the ABingo
identity
• Normally, some value that can be tested
across requests
• User ID if logged in

8. A/Bingo a Signup Form
• Define a test with ab_test
• views/users/new.html.erb
• Define a conversion with bingo!
• controllers/users_controllers.rb
• Go...

9. Forcing Options
• test_name=<value>
• signup_title=Registration
• signup_intro=true
• Useful in functional tests and real life

10. The Dashboard
• Shows tests
• Shows significance of tests
• 95% certainty is normal standard
• Statistical certainty != Impact

11. `

12. A/Bingo Coolness

13. Easy Install

14. Supports client or
server side tests

15. Easy to judge progress

16. No URL changes
required

17. Multi-Request/Session
Tracking

18. Minuses....

19. No cohort selection

20. No over-weighting of
alternatives

21. Unconfident
Multi-variate

22. Before A/B Testing,
ponder this...

23. Size == Confidence

24. Can You Convert?

25. Representative Samples

26. Multiple Tests /
Correlation Effects

27. Tactics vs. Strategy

28. Today’s code
• Railscast Episode 214
• http://github.com/ryanb/railscasts-episodes/
tree/master/episode-214/

29. Links
• http://www.bingocardcreator.com/abingo
• http://abtester.com/calculator/
• http://20bits.com/articles/statistical-analysis-
and-ab-testing/

30. Alternatives
• 7 Minute Abs - http://github.com/paulmars/
seven_minute_abs
• Vanity - http://vanity.labnotes.org/
• Google Website Analyzer - https://
www.google.com/analytics/siteopt/exptlist
• Omniture Test and Target - http://
www.omniture.com/en/products/
conversion/testandtarget