Managing Experiment at Spotify
•
130 gefällt mir•16,849 views
Spotify uses metric alignment and A/B testing to optimize user experience across their many autonomous teams and platforms. Education and automation are key to ensuring widespread adoption of the aligned metrics. Presenting results from A/B tests can demonstrate how certain features and functions impact vital metrics like daily and monthly active users, and teams are organized as a people as a service model to best implement optimizations.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Managing Experiment at Spotify
Ähnlich wie Managing Experiment at Spotify (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Managing Experiment at Spotify
- 1. May 20, 2014 Managing experiments at Spotify Ali Sarrafi Product Owner A/B testing and Experimentation
- 6. May 20, 2014 The right music for every moment 6
- 7. May 20, 2014 7 24M active users. 6M paying subscribers. 20Msongs.
- 8. Tribes Provide fast and reliable access to all the world's music Enable high product development speed while maintaining a highly available service P P P P Squad Squad Squad Squad Chapter Chapter P P P P Squad Squad Squad Squad Chapter Chapter Our team is growing rapidly 250 30 100 10
- 9. 9
- 10. 10 Autonomous fast moving teams Feature 1 Feature 3 Feature 2 Feature 4 Feature 5
- 11. May 20, 2014 Some Statistics Over 40 autonomous teams working on features Over 7 platforms with multiple features Over 3000 source repositories
- 12. 12 We don’t want a Frankenstein software
- 13. 13 Well at least a good looking 2014 edition
- 14. Section name 14 How can we align teams’ optimisation to avoid the frankenstein effect?
- 15. 15 Align on metrics rather than actual tests
- 16. 16 DAU (Daily Active Users) MAU (Monthly Active Users)
- 17. Section name 17 Evolving Metrics
- 18. May 20, 2014 Focus on functionality as a metric
- 19. 19 DAU/MAU Function 1 Function 2 Function 3 Function 4 CTR/Team Metrics Importance (High to low) Order of optimisation
- 20. Section name 20 Now we know what would work how do we make sure everyone uses it?
- 21. 21 It’s all about automation
- 22. 22 and of course education
- 23. Section name 23
- 24. 24 Make magic happen with desired metrics Vital Sign Group A Group B Functionality 1 +1.5% -0.5% Functionality 2 +2.5% -3% Functionality 3 +0.5% 0% Functionality 4 +1% -7%
- 25. 25 People/Infrastructure as a service Feature 1 Feature 5 Feature 4 Feature 2 Feature 3
- 26. Summary • Metric alignment is the most effective way of making sure the overall experience does not suffer • Education and Automation are key for adoption • People as a service is a most effective way of reaching out to teams.
- 27. May 20, 2014 Copyright stuff Napster pic by bixentro, cc 2.0 http://www.flickr.com/photos/bixentro/ Busy record store pic by gomattolson http://www.flickr.com/photos/gomattolson Empty record store pic by passetti http://www.flickr.com/photos/passetti/ First Frankenstein screenshot form : “Frankenstein” movie by Kenneth Branagh, 1994 Second Frankenstein screenshot form : “I, Frankenstein” movie by Stuart Beattie, 2014