This document discusses the benefits of guerrilla usability testing and provides tips for low-cost, DIY testing. It outlines that imperfect tests are still worthwhile as they can uncover unexpected issues, fix problems early, and introduce objectivity. The document then shares three stories of guerrilla testing experiences, highlighting lessons learned around recruitment strategies, engaging stakeholders, and testing on the go. It emphasizes that the best usability test is one that is done repeatedly and stresses making the testing and reporting process painless.
2. Benefits of in-house, DIY testing
(but you know this already!)
• Uncover unexpected issues
• Fix problems early, while it is still relatively easy
• Test assumptions, early and often
• Introduce objectivity into opinion wars
• Continually cultivate empathy in the hearts of
stakeholders
• Set baselines and measure improvement
• Great ROI at such low cost, we’d be crazy not to!
3. “Listening to what people say is
misleading: you have to watch
what they actually do.”
-Jakob Nielsen, Usability 101, useit.com
4. How Usable is your Usability
Testing?
1. Get representative users.
2. Ask users to perform
representative tasks with the
design.
3. Observe where they succeed,
and where they run into
problems.
Easy, right? So how frequently are
you testing?
5. The best usability test is the one
you actually do. Repeatedly.
The process & reporting MUST be fairly painless, or we’ll
find some excuse to put it off:
“Too busy, can’t stop for testing right now.”
“I’d like to get one more feature built before showing it
to anybody.”
“The testing software is too expensive.”
(“I am scared or embarrassed to find out the truth: that
my website isn’t usable!”)
6. Story 1: “It is a quick read, and
there are lots of pictures.”
7. Story 1: Lessons Learned
• Shy? Get over it. Human-to-human interaction IS
happening.
• Tailor your recruitment strategy for each audience you
are seeking (the yell vs. whisper)
• Live-streaming tests can be technologically tricky, but
really engages stakeholders.
• Pre-screen your candidates a bit before you sit them
down at the computer.
• Make graduate students (like me) write the reports!
8. Story 2: “Pay no attention that row
of women in glasses.”
9. Story 2: Lessons Learned
• The best way to convince management of the value of
testing is by (duh) testing…even without software.
• Recorded tests are simpler to set up, especially if you
don’t have much IT help.
• Be creative with non-monetary incentives.
• Group screenings + coffee + donuts = maximum
stakeholder engagement!
• Understand “Human Subjects Research” policies
11. Story 3: Lessons Learned
• Still shy? Still too busy? Recruit a usability intern, treat
them well, and keep them around as long a you can.
• When recording on the go, good sound is
EVERYTHING (mic + facilitator earphone test).
• Test out tasks on staff who may not use website often.
If one task seems too hard, IT IS! Drop it!
• Really nice people hang out in art museums, and boy,
do they love tote bags.
12. Mistakes aren’t forever. When you’re testing
often, it lowers the stakes.
It will always grow back.
13. D.I.Y. Usability “Lab”
•Laptop & USB microphone
•Software options
•Morae ($1,500)
•Camtasia ($300)
•Silverback for Mac ($0-39)
•Paper and pencil (free)
14. Analyzing and Reporting On
Results
•Holding screenings for staff
helps develop their interest
level & empathy
•A one-page list of
improvements you can act on
is better than a 10-page report
that sits in a drawer!
15. Let the next round of imperfect-
but-still-good-enough testing start!
