The document discusses usability testing as a way to validate products for lean startups. It provides an overview of usability and users, explains how to conduct usability testing, and discusses how usability testing fits into the lean startup process. Examples of guerrilla usability testing techniques are also presented. The document encourages testing with a small number of users and running many small tests to improve products iteratively.
2. About Me
Amy Rubino
User Experience Consultant, Systems
Alliance
• 15 years of design experience as a
print and web designer, information
architect and user experience
advocate.
arubino@systemsalliance.com • Worked with universities,
linkedin.com/pub/amy-rubino/13/837/31b corporations and non-profits.
@amy_rubino
• Master’s degree in Interaction Design
and Information Architecture.
Usability Testing as Validation 2
3. What We’ll Cover
Usability
Users
Learn About Users
Learn From Users
Usability Testing
How to Do It
What You Can Test
Why You Should Do It
Usability Testing and Lean Startup
Demo
Resources
Photo credit: Amy Rubino
Usability Testing as Validation 3
5. User Experience Honeycomb
Photo credit: http://semanticstudios.com/
Usability Testing as Validation 5
6. Usability
Creating products that are easy to use
Require less mental effort to use
Decrease frequency of mistakes and mistakes are less disastrous
Can be used to do more or do it faster
Can be figured out quicker
Usability Testing as Validation 6
7. Usability
Photo credit: http://impossibleobjects.com
Usability Testing as Validation 7
8. Usability
Based on human behavior
Focus on people who will use
it or users
Photo credit: sanjaykumarux.blogspot.com
Usability Testing as Validation 8
9. Users = Real people
Use your website or app
Part of your target
audience
Not directly affiliated with
your company
Photo credit: http://www.leveltendesign.com/ 9
10. You Are NOT Like Your Users
You Your Users
Know a lot about Know little or nothing about
your product and company your product or company
Are passionate about Just want to get things
what you offer done
Have ideas on how your May use your website or
website or app will be used app differently than was
based on being an insider intended
Usability Testing as Validation 10
11. Learn About Users
Surveys
Focus groups
Web analytics
Photo credit: http://www.flickr.com/photos/scottandsarah 11
12. Learn From Users
Talk to them
Observe them in their own
environment
Usability testing
Photo credit: http://www.flickr.com/photos/blesseurope 12
14. “ … If you want to know whether your software…
is easy enough to use, watch some people while
they try to use it and note where they run into
trouble. Then fix it, and test it again.”
Steve Krug
Author, “Don’t Make Me Think” and “Rocket
Surgery Made Easy”
Usability Testing as Validation 14
15. Traditional Process
Develop a test plan
Choose a testing environment
Find and select participants
Prepare test materials
Conduct the sessions
Debrief with participants and
observers
Analyze data and observations
Create findings and
recommendations
Photo credit: http://www.flickr.com/photos/doos/3050195096 15
20. In-Person Usability Testing
Invite people to see a beta version and
watch them complete tasks
Pros
See and hear participants’ reactions
Can record conversation and click path
Your stakeholders and team can
participate
Cons
You or participants have to travel to meet
up
Photo credit: http://www.flickr.com/photos/curiouslee/
May not have access to participants who
live outside of this region
Usability Testing as Validation 20
21. Remote Usability Testing
Use screen sharing software to
watch participants complete tasks
in another location
Pros
Don’t have to travel to participants
Have access to participants who live
outside of this region
Can record conversation and click
path
Your stakeholders and team can
participate
Cons Photo credit: http://www.flickr.com/photos/rosenfeldmedia/
Don’t see participants’ reactions
Usability Testing as Validation 21
22. Unmoderated Usability Testing
Use a service that allows participants to complete the tasks
on their one time without you moderating
Pros
Participants can do it when it’s convenient for them
Have access to participants who live outside of this region
Cons
Can’t ask follow up questions
Don’t hear participants think out loud
Don’t see participants’ reactions
Usability Testing as Validation 22
23. What You Can Test
Real products
Digital prototypes
Paper prototypes
Photo credit: http://www.jamesjpierce.com/energy_challenge_08.html 23
24. DO
Do test the site NOT the participants
Do note participant’s success in “I would open
completing each task Goggle and search
Do ask open-ended follow up for the website.”
questions Quote from usability testing
participant
Do compensate participants
Usability Testing as Validation 24
25. DON’T
Don’t give participants tasks that
“Do you really can not be completed
think people will Don’t guide participants
use this?” Don’t answer questions until the
Quote from usability testing
end
participant
Don’t take it personally
Usability Testing as Validation 25
26. Why You Should Do It
Reduce costs Fix Errors
Once a piece of software $120
makes it into the field, $100
the cost of fixing an error
$80
can be 100 times as
high as it would have $60
been during the $40
development stage.
$20
Source:
$0
IEEE (Institute of Electrical and Electronics During After
Engineers)
http://spectrum.ieee.org/computing/software/
Development Completion
why-software-fails/
Usability Testing as Validation 26
27. Why You Should Do It
Increase revenue
Increase sales, applications, memberships or donations
Increase retention
Remain competitive
Usability Testing as Validation 27
28. Why You Should Do It
Improve effectiveness
Reduce user error
Increase efficiency
Increase user satisfaction
Photo credit: http://www.flickr.com/photos/terryfreedman
Usability Testing as Validation 28
31. Traditional Process
Develop a test plan
Choose a testing environment
Find and select participants
Prepare test materials
Conduct the sessions
Debrief with participants and
observers
Analyze data and observations
Create findings and
recommendations
Photo credit: http://www.flickr.com/photos/doos/3050195096 31
32. Lean Process
Develop a test plan
Choose a testing environment
Prepare test materials
Find and select participants
Conduct the sessions
Debrief with participants and
observers
Analyze data and observations
Create findings and
recommendations
Photo credit: http://www.flickr.com/photos/adactio/ 32
33. Guerilla Usability Testing
Go to a public place and ask customers
to complete tasks
Pros
See and hear participants’ reactions
See participants use product on different
devices
Cons
Might need to get manager’s approval
Customers may not be your target
audience
Your stakeholders and team can’t Photo credit: http://www.flickr.com/photos/_dchri
participate
Usability Testing as Validation 33
34. “ … The best results come from testing no more
than 5 users and running as many small tests as
you can afford.”
Jakob Nielsen
principal, Nielsen Norman Group
Usability Testing as Validation 34
36. Try it Yourself!
Join Baltimore Lean Startup on Facebook:
https://www.facebook.com/groups/BaltimoreLeanStartup
Do some guerrilla testing
Post your results
Usability Testing as Validation 36
38. Books
“Don’t Make Me Think: A Common
Sense Approach to Web Usability”
by Steve Krug
“Handbook of Usability Testing:
How to Plan, Design, and Conduct
Effective Tests”
by Jeffrey Rubin and Dana Chisnell
“Remote Research: Real Users,
Real Time, Real Research” Photo credit: amazon.com
by Nate Bolt and Tony
Tulathimutte
Usability Testing as Validation 38