This document discusses low-cost user research methods that non-profits can use to test their websites and improve outcomes. It promotes brief user tests at Starbucks to observe people completing tasks on a website. Card sorting is presented to understand how users organize information. Surveys are suggested to answer specific questions about current users, with tips on keeping them short and analyzing responses. The presentation aims to demonstrate user research can be done easily and cheaply.
16. • Why are people abandoning the donation form?
• Why don’t more people give recurring gifts?
• What are the main things people are looking for on
my site?
• Do people notice the promotion we’re doing?
• How should we organize our new website (or
newsletter, etc)
POTENTIAL CASES
20. PARTICIPANTS
• Good targets
- People somewhat involved with your organization
• Event participants
• Donors
- The general public
• Avoid
- Employees
- Board
• Generally avoid*
- Major volunteers
- Friends & family
35. 1. Make sure you pick something that regular people
use, you will get regular people at Starbucks.
2. Test internally for practice
3. Test internally to make sure it is truly a 5 minute
test
4. Take 1 person with you. 1 person stays with laptop,
1 person solicits.
BEFORE YOU GO
36. 1. Make sure Starbucks is ok with it. Talk to the
manager on duty first, have a pitch ready for that
conversation so you can easily explain what you
want to do.
2. Bring your charger, mouse, test internet connection
(if needed), etc.
3. Grab a table - go early or avoid rush times.
BEFORE YOU START
37. 1. Talk to the people that are sitting around, you are buying
their NEXT cup of coffee, avoid the people in line.
2. Put your best foot forward, pick someone that will get a
good response.
3. Work on your pitch, it has to be faster than an elevator
pitch. Think of your pitch like a tweet: short, sweet and
to the point.
4. Make it obvious that you are a non-profit, lead with that,
wear the gear, beat people over the head with it; people
are more likely to help.
5. It helps to stress that this will take 5 minutes or less.
WHEN YOU ARE READY
38. 1. Don’t record names
2. Do get a wide cross section of users (age,
gender, race, etc)
3. Test the savvy level of the user, make sure they
aren’t extremely savvy or extremely novice.
4. Give them their task
5. WATCH
6. Try not to help
7. Ask follow up
WHEN YOU HAVE A USER
39. • If you find a massive gotcha, STOP. No need to keep
testing. Go back and fix it and try again another day.
• Once you have 3-5 users you probably have some really
good feedback, no reason to keep going. Maybe switch
to a new thing to test.
• This whole process might take 1.5-4 hrs
• You will most likely learn things that will blow your mind.
Things you never would have thought of.
WHEN THE TEST IS OVER
49. • Write down each 2nd level menu item on
its own note card
- In this example
• Write down
– News Archive
– Wayne’s blog
– Press releases
– Media contacts
• Don’t write
– NEWS
CARD SORTS
50. 1. Get a participant and some table space
2. Give them
- your note cards
- some blank cards or post-it notes
• (No more than you have space in your top level
navigation)
3. Let them sort the pre-written note cards into
groups
4. Ask them to label each group (using the
blank note cards)
5. Record the results
6. Repeat at least 10 times
CARD SORTS
52. • Tips
- Be sure to stay quiet while the participant is working
- Ask them to think aloud and listen to their commentary
- Take a photo when each person is done to capture their
work
• Variations
- Pre-write the headers (closed card sort)
- Conduct the card sort online
• http://www.optimalworkshop.com/optimalsort.htm
CARD SORTS
58. • 11 out of 15 people surveyed had never done any
user testing
- Surveys were the way most people do user
research
• 15 of 15 people surveyed had abandoned a survey
half way through because it was too long or too
confusing
OUR USER RESEARCH
59.
60.
61.
62.
63.
64. • Keep it short
- Usually have 5 – 15 questions
- Should take less than 5 minutes to complete
• Always have a general comment field
- Let them speak to you, not just answer questions
• Have someone proofread your survey
- Make sure none of the questions are confusing
• Be neutral
- You want information, not vanity metrics
• You don’t need to survey everyone
- How many responses do you need to answer your questions?
• You can use statistics, or
• General questions, homogeneous audience ~100 responses
- Assume a response rate of 10 – 20% for people somewhat involved with
your organization
SURVEY ETIQUETTE
65.
66. • Think of each question as testing a hypothesis. It
can help to word them that way
- Less useful: We should ask how many times per year they visit our
museum
- Better: I suspect that most people visit either only once per year or
more than 4 times. This will affect how we price and market our
membership programs. If we find a lot of people who visit 3-4 times
a year, we might decide to adjust our programs
SURVEY ANALYSIS TIPS
67. • Avoid “neutral” options
- Very likely
- Somewhat likely
- Somewhat unlikely
- Very unlikely
• (But you can use “not applicable”, if applicable)
• Experiment with forced rankings
• If you’re doing a large survey, consider how many open-ended
questions you want to analyze
SURVEY ANALYSIS TIPS
75. THIS PRESENTATION BROUGHT TO YOU BY
BLACKBAUD USER EXPERIENCE
• We want to learn from you so we can design
experiences that delight
• Participate in Blackbaud Discovery!
- Help us with our research
- Have your voice heard for the products you use
• Sign up to participate at
www.blackbaud.com/discovery
• Stop by the Discovery booth in the expo hall