Growth hackers rely on A/B testing to validate their improvement, but A/B testing is slow and inefficient. How do we make it faster? By solving the bandit problem! Bandit algorithms provide an alternative to A/B testing that is faster and more efficient, allowing more tests in less time. Video of the talk: http://www.youtube.com/watch?v=h7SG_Wo8sPQ Myna: http://mynaweb.com/?utm_source=awareness&utm_medium=slideshare&utm_campaign=Growth%2BHacker

1. GROWTH HACKING AND
THE BANDIT PROBLEM
Starring Noel Welsh A Production
Friday, 8 February 2013

2. WHAT DO
GROWTH HACKERS
DO?
Friday, 8 February 2013

3. GROW!
Friday, 8 February 2013

4. Metric
(sign-ups, revenue, etc)
Friday, 8 February 2013

5. Make this happen
Friday, 8 February 2013

6. Friday, 8 February 2013
HOW?

7. Build
Learn Measure
Friday, 8 February 2013

8. Build
Friday, 8 February 2013

9. Build
Measure
Friday, 8 February 2013

10. Build
Learn Measure
Friday, 8 February 2013

11. FASTER IS BETTER
Friday, 8 February 2013

12. HOW
DO WE MEASURE &
LEARN?
Friday, 8 February 2013

13. A/B
Friday, 8 February 2013

14. Build Measure Learn
Friday, 8 February 2013

15. Build
Learn Measure
Friday, 8 February 2013

16. Build
Measure/
Learn
Friday, 8 February 2013

17. Build Measure / Learn
Friday, 8 February 2013

18. THE BANDIT PROBLEM
(EST 1952 ... OR 1933)
Friday, 8 February 2013

19. k arms (or variants)
Friday, 8 February 2013

20. Select a variant
Friday, 8 February 2013

21. 
Receive a reward
Friday, 8 February 2013

22. REPEAT
Friday, 8 February 2013

23. TOTAL
MAXIMISE
REWARD
Friday, 8 February 2013

24. Friday, 8 February 2013

25. HOW?
Friday, 8 February 2013

26. DISPLAY VARIANT WITH
HIGHEST AVERAGE
REWARD
Friday, 8 February 2013

27. Good Variant Bad Variant
Friday, 8 February 2013

28. Good Variant Bad Variant
Views: 1
Reward: 0
Average: 0
Friday, 8 February 2013

29. Good Variant Bad Variant
Views: 1 Views: 1
Reward: 0 Reward: 1
Average: 0 Average: 1
Friday, 8 February 2013

30. OOPS!
Friday, 8 February 2013

31. Good Variant Bad Variant
WILL NEVER TRY THE
GOOD VARIANT AGAIN!
Views: 1 Views: 1
Reward: 0 Reward: 1
Average: 0 Average: 1
Friday, 8 February 2013

32. BALANCE EXPLOIT
AND EXPLORE
Friday, 8 February 2013

33. EXPLOIT
TRY THE VARIANT THAT
HAS WORKED BEST IN
THE PAST
Friday, 8 February 2013

34. EXPLORE
TRY OTHER VARIANTS
TO SEE IF THEY’RE
BETTER
Friday, 8 February 2013

35. E-GREEDY
Friday, 8 February 2013

36. 10% EXPLORE
90% EXPLOIT
Don’t like 10% and 90%? Change it.
Friday, 8 February 2013

37. MORE INTRICATE
BALANCING IS POSSIBLE.
SEE UCB-1, THOMPSON
SAMPLING, etc.
Friday, 8 February 2013

38. Friday, 8 February 2013

39. DOES IT
WORK?
Friday, 8 February 2013

40. Friday, 8 February 2013

41. REAL WORLD
CONSIDERATIONS
Friday, 8 February 2013

42. WORKFLOW
Friday, 8 February 2013

43. DEFINING
REWARDS
Friday, 8 February 2013

44. STABLE PREFERENCES
Friday, 8 February 2013

45. THANK YOU!
NOW GO FORTH AND
ENGAGE IN
BANDITRY
Friday, 8 February 2013

46. CREDITS
Photos and artwork by the following people were used in
this presentation:
Moyan Brenn http://www.flickr.com/photos/aigle_dore/
Suicine http://www.flickr.com/photos/bigmikeyeah
ankakay http://www.flickr.com/photos/ankakay/
Cdr Aitch http://www.flickr.com/people/hjsouthgate/
StJost http://stjost.deviantart.com/
Friday, 8 February 2013