Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

[Webinar] experimentation driven product development

524 Aufrufe

Veröffentlicht am

Experimentation is at the core of today’s most successful software products, from Amazon to Google to Facebook to Netflix. These companies use A/B testing and controlled rollouts to de-risk development and measure the impact they’re making with new ideas.

Join Product Manager Jamie Connolly to learn how Optimizely Full Stack enables any team to develop products like top companies--with experimentation-driven product development.
-Reduce the risk of launching new features
-Apply experimentation to your development process with Optimizely X Full Stack
-Prove the impact of every new feature or user experience change

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

[Webinar] experimentation driven product development

  1. 1. Experimentation-Driven Product Development Robin Pam Sr. Product Marketing Manager Optimizely Product Manager Optimizely Jamie Connolly With Optimizely X Full Stack
  2. 2. Robin Pam Sr. Product Marketing Manager Optimizely Product Manager Optimizely Jamie Connolly
  3. 3. Agenda ● Why Experimentation Matters ● What Is Experimentation-Driven Product Development? ● How to Build an Experimentation Platform ● Full Stack Demo
  4. 4. • We are recording today’s presentation • You will receive a copy of the slides after the webinar • Please submit questions via the text box Housekeeping
  5. 5. Innovation in the physical world can take decades
  6. 6. 3 YEARS LATER
  7. 7. Innovation by the hour
  8. 8. At any given point in time, there isn't just one version of Facebook running, there are probably 10,000. Mark Zuckerberg “ “
  9. 9. A B
  10. 10. A B
  11. 11. 9,800Google search experiments in 2016 www.google.com/search/howsearchworks/
  12. 12. 1,653Google search launches in 2016 www.google.com/search/howsearchworks/
  13. 13. I haven’t failed. I’ve just found 10,000 ways that won’t work.” Thomas Edison “
  14. 14. Experimentation-Driven Product Development
  15. 15. medium.com/netflix-techblog
  16. 16. uber.com/blog/colorado
  17. 17. LAUNCHBUILD DESIGN PRAY EXPERIMENT ITERATE
  18. 18. LAUNCH BUILD DESIGN PRAY EXPERIMENT ITERATE
  19. 19. LAUNCH BUILDDESIGN PRAY EXPERIMENT ITERATE
  20. 20. LAUNCHBUILDDESIGN PRAY EXPERIMENT ITERATE
  21. 21. LAUNCHBUILDDESIGN PRAY EXPERIMENT ITERATE
  22. 22. DESIGN BUILD EXPERIMENT ITERATE LAUNCH PRAY
  23. 23. How to build an experimentation platform that is fast and scales
  24. 24. // ab testing is easy if random() < 0.5: var = ‘a’ else: var = ‘b’
  25. 25. PlanOut SixPack Proctor phpA/B Convert Airlock Genetify FeatureBee Abba AlephBet abclub Confidence FluidFeatures Gertrude Rollout Split Trebuchet Vanity Wasabi
  26. 26. from planout.experiment import SimpleExperiment
 from planout.ops.random import *
 
 class FirstExperiment(SimpleExperiment):
 def assign(self, params, userid):
 params.button_color = UniformChoice(choices=['#ff0000', '#00ff00'], unit=userid)
 params.button_text = WeightedChoice(
 choices=['Join now!', 'Sign up.'],
 weights=[0.3, 0.7], unit=userid)
 
 my_exp = FirstExperiment(userid=12)
 # parameters may be accessed via the . operator
 print my_exp.get('button_text'), my_exp.get('button_color')
  27. 27. medium.com/netflix-techblog
  28. 28. medium.com/netflix-techblog
  29. 29. Analytics
  30. 30. medium.com/netflix-techblog
  31. 31. medium.com/netflix-techblog
  32. 32. eng.uber.com/data-viz-intel/
  33. 33. Traffic Splitting Remote Configuration Analytics Traffic Sampling Audience Targeting Mutual Exclusion State Persistence Multivariate Treatment Global Holdout QA Tool Automation Whitelisting Logging Permissions Feature Toggles Feature Rollouts Kill Switch Scheduling Staging Environments Audience Definitions Webhooks REST API Audit Trail Event Dispatching Event Storage Sessionization Statistical Methods Metric Definitions User Aliasing Segmentation DW Integration Notifications Visualizations 3 key ingredients for robust experimentation
  34. 34. 1. Trust
 “How can I trust the experiment was set up and tracked correctly?” 2. Process
 “What are the best instrumentation practices to minimize technical debt?” 3. Maintenance
 “The engineer who built this left the company… now what?” Three key challenges with experimentation
  35. 35. Demo
  36. 36. Full Stack Use Cases Feature Flags Performance Optimization Algorithms User Experience Roll outs Pricing $$ Marketing Channels
  37. 37. How many experiments do you run per year? How many changes do you launch per year?
  38. 38. Your Product Roadmap is your Experiment Roadmap
  39. 39. Join our next webinar: Getting Started with Server-Side Testing December 13, 2017 https://optimize.ly/serversidewebinar
  40. 40. Q&A Robin Pam Jamie Connolly

×