Presentation given at Designing For Digital on February 26, 2015.

1. How do students really do
research? Revelations from the
Research Confession Booth
Odile Harter, Harvard University
Lisa Junghahn, University of California, Irvine
Emily Singley, Harvard University
How do students really do research?
findings from the
Research Confession Booth
Odile Harter, Harvard University
Lisa Junghahn, UC Irvine
Emily Singley, Harvard University

2. Harvard Library Search & Discovery Initiative Education Subgroup, 2013-present

3. 1. Background
1.1. Provide the scientific background, rationale for the study, and importance in
adding to existing knowledge.
The initial goal of the study is to obtain a better understanding of how patrons use library
resources and what they most value about the resources we do have. This information will
be used to inform the design and implementation of the Library’s new discovery platform.
As the Library keeps pace with a constantly changing research environment, the study will
continue to investigate how researchers interact with and feel about library products and
services.
Harvard Library has access to several measures of patron behavior, including user surveys,
focus groups, and usability studies, but none of these offer sufficiently granular or
contextualized information for us to understand the whole picture of what our users want.
This study will collect information that is granular, qualitative, and not tied to a particular
website or library vendor. It will also provide a recognizable site for library patrons to
quickly and easily give very specific feedback about their research experiences.
Study protocol

4. Ingredients:
 1-3 well-traveled campus
locations with ambient noise
 10-20 candy and granola bars
 1 poster
 20 info sheets
 1 laptop
 1 wireless mouse
 1 cardboard box
 1 table & chair
 1 Quicktime screen recording
 1 Qualtrics survey
 1 task
 USB drives
 1 IRB approval
Study protocol

5. TASK 1
“Please show us a feature you really like in a
resource you use to do research.”
Study protocol

6. TASK 2
“Please walk us through a situation in which you were trying to find
information to use in your coursework and encountered some kind of
obstacle or snag that interfered with your ability to find, obtain, or use
the information you were seeking.”
Study protocol

7. Status
Undergraduate 15
Graduate 3
Faculty 1
Experience (self-identified)
Novice 7
Intermediate 11
Expert 1
Study protocol
Participants:

8. Pros and cons:
 Flexible
 Nimble
 Qualitative
 Semi-controlled/slightly artificial
 Small dataset
Study protocol

9. What we’re learning:
 Global picture – which systems are
actually used
 Navigation behaviors
 Contextualized usage - how and why
systems are used
Findings

10. Category Resource Use
Harvard library
HOLLIS catalog 10
Harvard Library
website
1
Databases A-Z list 1
Fulfillment services
Scan & Deliver 1
Borrow Direct 1
Interlibrary Loan 1
Open web
Google Scholar 5
Google 5
Wikipedia 1
Databases
PubMed 4
Academic Search Premier 1
JSTOR 1
Findings

11. Navigation:
 Typing in memorized URL
 Googling site name
 Navigating via Harvard Library
homepage or intranet
Findings

12. Features identified as important:
 Confidence & comfort
 Filtering
 Getting relevant results:
o topical
o level of detail
o academic results
 Full-text fulfillment
 Citation management options
“This button right here is the best:”
Findings

13. Library as difficult, confusing, annoying:
Findings
“I sometimes feel like I don't know how to use the right ‘ands’
and ‘ors’ and quotation marks."
"I really don’t like "getting it" from interlibrary loan - I feel like
that's really annoying. So I actually usually just try to find it on
my own.”
“The resource I use most would probably be JSTOR, because
I’ve known that since high school (…) I’m not the best at
research.”
“See, this is the point where I usually give up and just go ask
the research librarian for help, you know, because I don’t
know how to do this stuff.”

14. Preference for open web:
 Quick fulfillment
 Familiar interface
 Trust in how it’s interpreting your query
 Greater tolerance for uncertainty
 Ability to match effort invested to information need
Findings

15. User interface problems:
 Invisible or incomprehensible button labels
 Complicated layouts
Findings

16. Example of user unable to find PubMed full-text link

17. Silo-to-silo confusion:
 What resource am I in?
Findings

18.  Other tasks we might try with this study
 Further analysis of the findings
 Companion studies
 Effects on our daily work
Future research

19. Thank you!
Questions?