Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Always Be Testing
Learn from every thing you do!
Hiten Shah
hnshah@gmail.com
@hnshah #FILIVE
Product, Design, Engineering, Marketing, Growth, Sales
What do you do?
 
How do you make
all the things
be er?!
Figuring this out is your job.
A/b testing
Only	
  1	
  out	
  of	
  5	
  tests	
  win.
4	
  out	
  of	
  5	
  tests	
  cost	
  
you	
  money.
So, why experiment if so many tests fail?
Go Where Customers Are!
Long, slow, SaaS ramp of death from 2006…
Go Where Customers Are!
Now…
Go Where Customers Are!
Growth can be sped up!
Increase Hello Bar’s installation rate
Improve the KISSmetrics features page
Increase signups from Crazy Egg’s homepage
Grow Crazy Egg’s revenue
Results from 3 months of focused growth effort
Results: KISSmetrics signups over time
Simple 4-step experimentation process
Data
Figure out what to improve
Hypothesis
Make an educated guess
Experiment
Test yo...
Data
Figure out what to improve
Data doesn’t make decisions for you.
Data informs your decision making.
Data helps you identify the most
painful problems people have.
(not the ones that bother you the most)
Find problems worth solving by
starting with quantitative data.
When in doubt, look at your top pages. Usually the homepage.
A simple framework for choosing what to improve
Find pages with high traffic and high bounce rate
HTTP://GOOGLE.COM/ANALYTICS
Find pages with high traffic and low conversion rate
HTTP://UNBOUNCE.COM
Find sources with high traffic and low conversion rate
HTTP://HUBSPOT.COM
Find opportunities based on user paths
HTTP://KISS.LY/KMPATH
Increase Conversions
By A/B Testing
Do a Happy Dance
But Watch it Closely
Get More Traffic
So You Can Start A/B Testing
Get ...
What percentage of your
customers or users have
adopted each feature?
Intercom on Product Management
When you’ve already g...
Where do you have the most
visitors, users, customers,
emails, or notifications?
More volume helps you test faster.
๏ Website Traffic / App Installs
๏ Welcome Emails
๏ Web / Mobile Onboarding
๏ Email Digests
๏ Triggered Notifications
๏ Cart ...
Find out why by ge ing
qualitative data.
A few qualitative tools
HTTP://USERTESTING.COM HTTP://SURVEYMONKEY.COM
A few more qualitative tools
HTTP://QUALAROO.COM HTTP://OLARK.COM
Use exit surveys to learn why people bounce
HTTP://QUALAROO.COM
Use post-conversion surveys to understand why
HTTP://QUALAROO.COM
Discover if your web pages are effective at their goal
HTTP://QUALAROO.COM
What frustrates people?
EMAIL
Find people who don’t do things and email them
EMAIL + SURVEYMONKEY
User Research gives you a headstart
Hypothesis
Make an educated guess
Ideas
Designs
Solutions
Assumptions
(A guess about the best way to solve a problem)}
Everything can start with a
testable hypothesis.
Before you create anything, make an educated guess.
Use simple equations to assess potential impact of each idea
DAN MCKINLEY HTTP://KISS.LY/DANETSY
visits	
  x	
  conversion...
Assess the potential impact,
difficulty and priority of
each test idea you have.
Prioritize your tests
Experiment
Test your guess
Test your hypothesis by starting
an experiment.
Not bu on colors…
Don’t copy “best practices” blindly
Test specific bu on copy
Sensitivity tests help you
learn what your visitors care
about and what they don’t.
Run sensitivity tests on your web pages
Remove graphical elements and see what happens
Does social proof and logos ma er for your visitors?
Remove social proof and learn what happens
Act
Decide what’s next
You’ve probably seen an A/B testing report before
1. Your starting hypothesis.
2. Dates for when you ran the test.
3. Screenshots of control and all variations used during ...
What learnings do we want to
carry over from the past?
Helps you avoid unknowingly
decreasing your conversions.
✓ Keep the footer links
✓ Autofocus on the first form field
✓ Do not include social proof
✓ Link to the features page
✓ Use ...
If you’re not testing,
you’re not learning.
☀
Always have tests running,
no ma er how small.
⌾
If you don't have a process,
you won't continuously improve.
∞
Work on things that ma er.
Make those things be er.
Be happier ☺
Thank Y0u =)
Got questions? Need help?
Contact me any time!
Hiten Shah
hnshah@gmail.com
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Nächste SlideShare
Wird geladen in …5
×

Always Be Testing - Learn from Every A/B Test (Hiten Shah)

36.478 Aufrufe

Veröffentlicht am

Session slides from Future Insights Live, Vegas 2015:
https://futureinsightslive.com/las-vegas-2015/

Hiten Shah, co-founder of KISSmetrics and Crazy Egg, talks you through his 4-step data-driven process for successful A/B testing.

Veröffentlicht in: Daten & Analysen, Software, Internet
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/qURD } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/qURD } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/qURD } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/qURD } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/qURD } ......................................................................................................................... Download doc Ebook here { https://soo.gd/qURD } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Manifestation Magic� - $27 - By Alexander Wilson - 96% Off ●●● http://t.cn/AiuvUMl2
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Sex in your area is here: www.bit.ly/sexinarea
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Unlock Her Legs(Official) $69 | Get 90% Off + 8 Special Bonus? ●●● http://t.cn/AiurDrZp
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Always Be Testing - Learn from Every A/B Test (Hiten Shah)

  1. Always Be Testing Learn from every thing you do! Hiten Shah hnshah@gmail.com
  2. @hnshah #FILIVE
  3. Product, Design, Engineering, Marketing, Growth, Sales What do you do?  
  4. How do you make all the things be er?! Figuring this out is your job.
  5. A/b testing
  6. Only  1  out  of  5  tests  win. 4  out  of  5  tests  cost   you  money.
  7. So, why experiment if so many tests fail?
  8. Go Where Customers Are! Long, slow, SaaS ramp of death from 2006…
  9. Go Where Customers Are! Now…
  10. Go Where Customers Are! Growth can be sped up!
  11. Increase Hello Bar’s installation rate
  12. Improve the KISSmetrics features page
  13. Increase signups from Crazy Egg’s homepage
  14. Grow Crazy Egg’s revenue
  15. Results from 3 months of focused growth effort
  16. Results: KISSmetrics signups over time
  17. Simple 4-step experimentation process Data Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide what’s next
  18. Data Figure out what to improve
  19. Data doesn’t make decisions for you.
  20. Data informs your decision making.
  21. Data helps you identify the most painful problems people have. (not the ones that bother you the most)
  22. Find problems worth solving by starting with quantitative data.
  23. When in doubt, look at your top pages. Usually the homepage.
  24. A simple framework for choosing what to improve
  25. Find pages with high traffic and high bounce rate HTTP://GOOGLE.COM/ANALYTICS
  26. Find pages with high traffic and low conversion rate HTTP://UNBOUNCE.COM
  27. Find sources with high traffic and low conversion rate HTTP://HUBSPOT.COM
  28. Find opportunities based on user paths HTTP://KISS.LY/KMPATH
  29. Increase Conversions By A/B Testing Do a Happy Dance But Watch it Closely Get More Traffic So You Can Start A/B Testing Get More Traffic So You Can Do a Happy Dance High Traffic Low Traffic High ConversionsLow Conversions How To Figure Out What To Improve
  30. What percentage of your customers or users have adopted each feature? Intercom on Product Management When you’ve already got usage, analyze it.
  31. Where do you have the most visitors, users, customers, emails, or notifications? More volume helps you test faster.
  32. ๏ Website Traffic / App Installs ๏ Welcome Emails ๏ Web / Mobile Onboarding ๏ Email Digests ๏ Triggered Notifications ๏ Cart / Funnel Abandonment ๏ Dormant / Churned users Common areas with large volume and high impact potential
  33. Find out why by ge ing qualitative data.
  34. A few qualitative tools HTTP://USERTESTING.COM HTTP://SURVEYMONKEY.COM
  35. A few more qualitative tools HTTP://QUALAROO.COM HTTP://OLARK.COM
  36. Use exit surveys to learn why people bounce HTTP://QUALAROO.COM
  37. Use post-conversion surveys to understand why HTTP://QUALAROO.COM
  38. Discover if your web pages are effective at their goal HTTP://QUALAROO.COM
  39. What frustrates people? EMAIL
  40. Find people who don’t do things and email them EMAIL + SURVEYMONKEY
  41. User Research gives you a headstart
  42. Hypothesis Make an educated guess
  43. Ideas Designs Solutions Assumptions (A guess about the best way to solve a problem)}
  44. Everything can start with a testable hypothesis.
  45. Before you create anything, make an educated guess.
  46. Use simple equations to assess potential impact of each idea DAN MCKINLEY HTTP://KISS.LY/DANETSY visits  x  conversion  rate  x  order  value  x  li4  =  ? users  x  order  value  x  resurrec7on  rate  =  ? ESTIMATED ESTIMATED MODELED MODELED
  47. Assess the potential impact, difficulty and priority of each test idea you have. Prioritize your tests
  48. Experiment Test your guess
  49. Test your hypothesis by starting an experiment.
  50. Not bu on colors…
  51. Don’t copy “best practices” blindly
  52. Test specific bu on copy
  53. Sensitivity tests help you learn what your visitors care about and what they don’t. Run sensitivity tests on your web pages
  54. Remove graphical elements and see what happens
  55. Does social proof and logos ma er for your visitors?
  56. Remove social proof and learn what happens
  57. Act Decide what’s next
  58. You’ve probably seen an A/B testing report before
  59. 1. Your starting hypothesis. 2. Dates for when you ran the test. 3. Screenshots of control and all variations used during the test. 4. Expected change in conversion. 5. Probability of change. 6. Screenshot of the raw data used in your analysis. 7. A screenshot to the A/B Test Report in KISSmetrics. 8. The decision you've made from the test. 9. What you learned from the test. Document every A/B test and create a playbook
  60. What learnings do we want to carry over from the past? Helps you avoid unknowingly decreasing your conversions.
  61. ✓ Keep the footer links ✓ Autofocus on the first form field ✓ Do not include social proof ✓ Link to the features page ✓ Use “Log in” copy on the bu on Create experiment checklists based on learnings
  62. If you’re not testing, you’re not learning. ☀
  63. Always have tests running, no ma er how small. ⌾
  64. If you don't have a process, you won't continuously improve. ∞
  65. Work on things that ma er. Make those things be er. Be happier ☺
  66. Thank Y0u =) Got questions? Need help? Contact me any time! Hiten Shah hnshah@gmail.com

×