When to run how many A/B-tests (based on data) - MeasureCamp Amsterdam - Ton Wesseling
•
0 gefällt mir•969 views
• When to start A/B-testing • When should A/B-testing become a core competence • When to stop A-B/testing
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (8)
Ähnlich wie When to run how many A/B-tests (based on data) - MeasureCamp Amsterdam - Ton Wesseling
Ähnlich wie When to run how many A/B-tests (based on data) - MeasureCamp Amsterdam - Ton Wesseling (20)
Mehr von Online Dialogue
Mehr von Online Dialogue (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
When to run how many A/B-tests (based on data) - MeasureCamp Amsterdam - Ton Wesseling
- 2. M e a s u r e C a m p , A m s t e r d a m , A p r i l 9 t h 2 0 1 6 To n W e s s e l i n g - F o u n d e r - Te s t i n g . A g e n c y Tw i t t e r : @ To n W - To n @ Te s t i n g . A g e n c y Utilize your test capacity
- 3. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp
- 4. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp
- 5. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Johan Cruijff
- 6. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Johan Cruijff 1947 - 2016
- 7. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp
- 8. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Lead the way
- 9. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp ”You will only be able to see it once you get it.”
- 10. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Online Dialogue
- 11. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Testing.Agency
- 12. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Test capacity
- 13. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Test capacity maximum number of a/b-tests per year that you could run on your users
- 14. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp What do companies do within CRO?
- 15. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Are companies improving conversion?
- 16. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Is it because of the easy of testing?
- 17. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Most effective to improve conversions
- 18. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Regret
- 19. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Regret amount of money you are losing per year by not using your full test capacity
- 20. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp the ROAR model
- 21. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth
- 22. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth
- 23. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth
- 24. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk
- 25. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk + Optimization
- 26. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk + Optimization + Automation
- 27. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk + Optimization + Automation Re-think
- 28. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk + Optimization + Automation Re-think 1.000 conversions per month
- 29. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk + Optimization + Automation Re-think 10.000 conversions per month 1.000 conversions per month
- 30. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp How much impact do you need? http://ondi.me/size
- 31. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Optimization phases - ROAR Time span Conversionspermonth Risk + Optimization + Automation Re-think 10.000 conversions per month 1.000 conversions per month 15% impact needed 5% impact needed Impact needed: all conversions with an average conversion rate of 2%, a test length of 3 weeks and a power of 80%
- 32. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Yes, I’ve got those 1000+ conversions Time span Conversionspermonth Risk + Optimization + Automation Re-think 1.000 conversions per month 10.000 conversions per month
- 33. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp With 1000+ conversions a month Your test capacity is max. 20+ tests a year with a statistical power of 80% on predicted uplifts of 15%
- 34. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests with measurable impact On average 1 out of 3 Simple truth: you’re just not always able to create a winner with enough impact
- 35. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp So in this case, you’ve got impact Every 7 or 8 weeks
- 36. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Slower? Energy will run out! And your continous optimization program will die
- 37. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp What to test? Business cases! Delivery –test with in stock. 4 weeks or 2 days?
- 38. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp What to test? Business cases! Service charge, do it or not? Include in price or not?
- 39. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp What to test? Big design changes
- 40. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp ROAR – Optimization – moving up Time span Conversionspermonth Risk + Optimization + Automation Re-think 1.000 conversions per month 10.000 conversions per month
- 41. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Source: Psychology
- 42. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Use that persuasion knowledge Wheel-of-Persuasion.com Resource & courses
- 43. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 44. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 45. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 46. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 47. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 48. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 49. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Tests that can be done
- 50. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp + Optimization + Automation Back to the ROAR model Time span Conversionspermonth Risk Re-think 1.000 conversions per month 10.000 conversions per month
- 51. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp ROAR – Optimization – moving up Time span Conversionspermonth Risk + Optimization + Automation Re-think 1.000 conversions per month 10.000 conversions per month
- 52. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Test capacity users IT releases Campaigns Behavior
- 53. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp IT release testing
- 54. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp IT release testing Testing to see if the release is not lowering the conversion rate
- 55. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Main result example
- 56. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Campaign testing
- 57. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Multi-arm bandit problem
- 58. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Maximize your exploitation
- 59. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Behavior learning
- 60. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Behavior learning
- 61. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Store your learnings
- 62. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp That’s why you need the traffic Time span Conversionspermonth Risk + Optimization + Automation Re-think 1.000 conversions per month 10.000 conversions per month
- 63. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Test capacity users IT releases Campaigns Behavior 25% or less 25% or less 50% or more
- 64. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Calculating behavioral capacity
- 65. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Behavioral knowledge test capacity • Sum your # tests per element • Take the behavior share • divide by 2 (no major overlap!)
- 66. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Behavioral knowledge test capacity • Sum your # tests per element • Take the behavior share • divide by 2 (no major overlap!) • Consider your resources • Get resources or test with higher Power
- 67. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Winner ratio
- 68. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Winner ratio • Know your % winners bottom limit • Better testing • Bolder testing • Less testing (more data in 1 test)
- 69. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Winner
- 70. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp No Winner?
- 71. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Or Winner?
- 72. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Yes, business case
- 73. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Session: Annemarie Klaassen Usabilla room: 13.40 – 14:10
- 74. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Bayesian tools & pdf R Calculator: ondi.me/bayes White Paper PDF Free download: http://ondi.me/change
- 75. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Be a pro, like Johan was
- 76. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp ü Use your full test capacity by great planning ü Separate test bandwidth for IT, campaigns and for learnings ü Think like a manager – work with business cases Be a pro, like Johan was
- 77. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp An the whole company will love you
- 78. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp One more thing
- 79. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp One more thing Celebrate failures
- 80. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Celebrate failures
- 81. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Not enough winners? Your continous optimization program will die
- 82. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Session: Jorrin & Tom Usabilla room: 14.15 – 14:45
- 83. M e a s u r e C a m p , A m s t e r d a m , A p r i l 9 t h 2 0 1 6 To n W e s s e l i n g - F o u n d e r - Te s t i n g . A g e n c y Tw i t t e r : @ To n W - To n @ Te s t i n g . A g e n c y Utilize your test capacity
- 84. Ton Wesseling | @ TonW Utilize your test capacity #MeasureCamp Conversion.Camp Get a ticket: http://anali.st/ccfriends