Success and failure with conversion rate optimization largely depends on how well a company is able to embrace a culture of experimentation. Companies that fail with ongoing optimization are generally those that treat it as a side project, so that CRO efforts gain little organizational support. In this presentation, Sean will discuss how leading companies collect unique insights from across the organization, and provide full transparency into the testing process and results. Creating a company wide culture of experimentation requires patience, but the payoff is well worth the effort. Companies with experiment-driven cultures outperform their peers in everything from scaling customer acquisition channels to changing the way all team members focus their efforts towards high value activities.

1. Building a Company-Wide Culture
of Experimentation
May 18, 2016
Founder & CEO GrowthHackers

2. Sept 2008: Dropbox Blank Slate
• Excellent product coming
out of Beta
• Team of 7 engineers
• Skeptical of marketing
• Goal: Create culture of
growth experimentation
#ConvCon

3. Seeds of Testing Culture
• Built tracking and testing
infrastructure
• Shared insights (qual and
quant) with team
• Shared testing wins with
team
• Encouraged broader
participation
#ConvCon

4. Today Dropbox has ½ Billion Users
• No marketer from April 2009 – January 2010
• Product/engineers continued to experiment
#ConvCon

5. Growth is Company Wide Effort
Product
External
Channels
Promise
1st Experience
Ongoing Experience
Acquisition
Activation
Revenue
Retention
Referral
Marketing
PR
Support
Success
biz dev
CRO or
Growth
Team
#ConvCon

6. AKA Growth Hacking
Rapid experimentation across the
full funnel to learn the most effective ways
to scale sustainable customer adoption.
#ConvCon

7. Testing Drives Growth
• No certainty about what works
• Rapid testing across all levers
• More testing = More learning
Acquisition Referral
Activation Revenue
Retention Resurrection
#ConvCon

8. Twitter Example
#ConvCon

9. Change Harder Established Company
• Day onsite at Microsoft (May 2016)
• Goal: Transform to culture of growth
experimentation
• Founded 1975, 118K employees
• Results: TBD
#ConvCon

10. Steps to Experimentation Culture
Broad
Participation
Transparent
Learning
Effective Process
#ConvCon

11. Process First
Broad
Participation
Transparent
Learning
Effective Process
#ConvCon

12. Effective Process
Idea
Backlog
Prioritize
backlog
Launch
Tests
Analyze
Results
#ConvCon

13. Core Team to Manage Process
• Start with product manager
• Add dedicated people to core as needed
to hit testing tempo goals
• Core often includes designers, growth
engineers, analysts…
#ConvCon

14. Effective Process
Idea
Backlog
Prioritize
backlog
Launch
Tests
Analyze
Results
#ConvCon

15. Goal: Build Large Backlog of Ideas
• Build large backlog of ideas
#ConvCon

16. Prioritize Backlog
Unbridled
Ideation
Prioritize
backlog
Launch
Tests
Analyze
Results
#ConvCon

17. Select Tests in Weekly Meeting
• Use system for prioritization (we use ICE)
• Choose tests to launch (no brainstorming)
IMPACT CONFIDENCE EASE
(1 – 10) (1 – 10) (1 –
10)
#ConvCon

18. Launch Tests
Unbridled
Ideation
Prioritize
backlog
Launch
Tests
Capture
learning
#ConvCon

19. Launch Tests
• Assign a project manager to each test
• Use tools and allocate skills needed to implement
#ConvCon

20. Analyze Results
Unbridled
Ideation
Prioritize
backlog
Launch
tests
Analyze
Results
#ConvCon

21. Analyze Results
• Ensure tracking up front for
hypothesis
• Run sufficient time
• Triage analysis by
hypothesis first
#ConvCon

22. Example Weekly Meeting Agenda
15 min: KPI review & update focus area
10 min: Review last week’s testing sprint
15 min: Key lessons learned from analyzed tests
15 min: Select tests for this week’s sprint
5 min: Check growth of idea backlog
#ConvCon

23. Requires Transparent Learning
Broad
Participation
Transparent
Learning
Effective Process
#ConvCon

24. Transparent Learning
• Every test leads to learning
• Proactively share with idea originator
• Share wins with opt in email list
• Organize learning for everyone’s easy access
#ConvCon

25. Drive Broad Participation
Broad
Participation
Transparent
Learning
Effective Process
#ConvCon

26. Company Kickoff
• CEO: “everyone seek opportunities for
improvement and growth”
• Share recent wins and insights
• Idea input training
• Pizza and beer brainstorm
#ConvCon

27. Collect Ideas Widely
External Experts
Internal Experts
Ideas Tests Results
Core Growth Team Drives Process
#ConvCon

28. Standard Experiment Doc for Ideas
• Include research, hypothesis, target lever
#ConvCon

29. Provide Feedback on Ideas
• Acknowledge all ideas
• Praise favorite ideas
• Notify if selected
• Report back results
#ConvCon

30. Keep it Fun
#ConvCon

31. Idea from Engineer - oEmbed
• Data: Embeds = longer
sessions
• Hypothesis: Same
action = more embeds
• Result: 41% increase in
embedded media
Slides
Videos
#ConvCon

32. Idea from Analyst: To-do-list
• Data: low % of newly
registered users commenting
in 1st week
• Hypothesis: Increase % of first
week commenters
• Test: Use a To-do-list to guide
new users (start with easy
activities)
#ConvCon

33. Result: To-do-list
• Result: over 700% lift in 1st
week commenting
#ConvCon

34. Key Takeaways
• Share learning & celebrate wins
• Empower full team to generate ideas
• Ensure core team has process to manage inputs
#ConvCon

35. Join GH Projects Private Beta
at Projects.GrowthHackers.com
Where Teams Drive Results with High Tempo Testing:
1) Team Ideation 2) Prioritize Testing 3) Capture Learning
#ConvCon