A talk I gave at a conference sponsored by the Pew Making Voting Work project and the JEHT Foundation in Washington, DC on December 9, 2008.

1. 1
What is a usability test?

2. 2
Sit next to someone.
Watch. Listen.
Dana Chisnell

3. 3
When
something
changes
Take
constraints
into account
Complement to Reveal
best practice remedies

4. 4
Hesitating
Comments
Questions
Body
language
What to look for Behaviors

5. 5
Ballot testing.
Dana Chisnell

6. The question
6

7. 7
The question
How
successfully
do voters
vote?

8. The plan
8

9. 9
1 day
Dana Chisnell

10. 10
a ballot and instructions
Nancy Frishberg

11. 11
voter participants
Dana Chisnell

12. 12
observers
Dana Chisnell

13. 13
results due: today

14. 14

15. 15

16. 16
How we got value
 Used the ballots and instructions at hand
 Focused on one thing at a time
 Had participants generate and collect
data
 Drafted observers and debriefed

17. 17
1. Plan, minimally
2. Get the team on
board
3. Use the LEO Kit
4. Recruit
participants
5. Conduct
sessions
Follow these steps: 6. Debrief and
decide

18. 18
Conduct
sessions
 Rehearse (P1)
 Checklist of tasks
 Short, focused sessions
 Adjust what you’re doing, if you
need to
Dana Chisnell

19. 19
Greet the
participant
Explain the test,
your role, and
their role
Watch the voter
vote
Ask follow-up
questions
Debrief with
participant
Session outline Debrief with
observers

20. 20
Impartial,
unbiased
observing
No teaching!
Listen and watch
Ask open-ended
questions: Why?
Moderating, not training How? What?

21. 21
Debrief and
decide
 What could cause a recount?
 What could make it difficult to
tell intent?
 What could draw unwanted
attention?
 What could cost money if it is
not fixed?
Dana Chisnell

22. 22
Resources
• Usability Professionals’
Association:
www.usabilityprofessionals.org/
• LEO Usability Testing Kit:
www.usabilityprofessionals.org/
civiclife/voting/leo_testing.html
• Ballot Usability and Accessibility
Blog: ballotusability.blogspot.com/