The document summarizes a study evaluating search interface features for searching biomedical literature. It describes (1) conducting interviews and evaluations with bioscientists to understand common search tasks and compare interface features, (2) features like autocomplete, query expansions, faceted refinement, related searches and results preview were evaluated for different search tasks, and (3) results found that faceted refinement and results preview were generally useful, while other features depended on the specific search task. Checkboxes were preferred over links for faceted navigation.
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Search interface feature evaluation in biosciences
1. Search interface feature evaluation
in biosciences
Alyona Medelyan
joint work
with Anna Divoli
(University of Chicago)
HCIR-2011
20.10.2011
2. Talk Overview
Search interfaces and their features
Search tasks on the web and in bioscience
Experiment description
Side-by-side evaluation example
Data collected
Results
3. autocomplete
facetted
refinement
results preview related
searches
search
expansions
4. Talk Overview
Search interfaces and their features
Search tasks on the web and in bioscience
Experiment description
Side-by-side evaluation example
Data collected
Results
5. Search Tasks and their Types in Web Search
(Kellar et al. 2007)
weather
exchange rate Fact finding
… Information gathering
email
banking
grad schools Browsing
travel plans
shopping … Transactions
…
blogs Other
news
…
6. Search Tasks and their Types in Bioscience
I’d like to find out what kind of
animal models of huntington’s disease
are out there
Fact Finding
I need to collect publications
by others on connexins
& how they relate to our studies
Information Gathering
I want to find out whether there are
any new publications on
the mechanism that underlies Golgi
cisternal maturation in yeast
Browsing
7. Research Questions and Hypotheses
Which search interface features are useful
1 for searching the biomedical literature?
Hypothesis Users prefer different interface features
depending on the search task
Which approaches to facetted navigation
2 work best for this domain?
Hypothesis It’s better to display dynamically computed sets
of facets than a complete hierarchical list
8. Talk Overview
Search interfaces and their features
Search tasks on the web and in bioscience
Experiment description
Side-by-side evaluation example
Data collected
Results
9. The Study
Exploratory short study with 6 bioscientists
2 faculty, 2 postdocs, 2 PhD students
Q&A on 3 search types in their work, queries, resources, systems
10-15min in person sessions
Main study with 10 bioscientists
2 faculty, 7 postdocs, 1 PhD student
Email & 1-2hr in person sessions
Side-by-side comparison of anonymysed search interface features
Per participant: 1 baseline and 1 own query
10. Talk Overview
Search interfaces and their features
Search tasks on the web and in bioscience
Experiment description Baseline query
Side-by-side evaluation example connexin
Data collected
Results
17. Talk Overview
Search interfaces and their features
Search tasks on the web and in bioscience
Experiment description
Side-by-side evaluation example
Data collected
Results
18.
19.
20. Talk Overview
Search interfaces and their features
Search tasks on the web and in bioscience
Experiment description
Side-by-side evaluation example
Data collected
Results
22. Summary of Findings,
Participants’ Comments
Autocomplete is less important: “we feel pigeonholed by suggestions”
Facets are useful: “we focus and refine the search all the time”
Choose facets wisely: “a large number of facets is overwhelming”
Checkboxes are better than links: “we want to select multiple values”
Aesthetics are important but what really matters is the content
23. Conclusions
Which search interface features are useful
1 for searching the biomedical literature?
Facets and results preview are useful for any search task
Other features are more useful for browsing
Which approaches to facetted navigation
2 work best for this domain?
Few, query-oriented facets with specific values, in checkboxes!
Alyona.Medelyan@pingar.com
Anna.Divoli@pingar.com
Hinweis der Redaktion
Contacted Anna after reading Marti Hearst’s book and seeing a screenshot of Anna’s interface, very similar to one that Pingar came up withSuggested to do a research together around interface, usability and searchThrough Anna’s background we focused on bioscience
We looked specifically at 5 features that guide users in their search from input of the query to evaluating the results
Autocomplete, search expansion, facetted refinement, related searches, results preview on PubMed – commonly used system in bioscience
To add an additional dimension to our research we looked at search tasks.
A study by Kellar shows that nearly a half of search tasks related to transactions. The remaining tasks can be grouped equaly into ff, ig and br.
We found that in bioscience the differentiaon is somewhat difficult… It’s hard to know which query relates to each task.These are examples of real search needs descriptions, and how we classified them based on Kellar’s search tasks.
Based on existing literature we found that interface features are studied extensively independently from each other, but comparison of their usefulness for search has been missing.We chose to focus on the bioscience domain to have specific findings. Bioscience is one of the verticals, in which Pingar plans to specialize their software for.Our hypothesis was that the usefulness of features depends on the search task.For Pingar specifically we were also interested in testing various approaches to computing and representing facets.Our hypothesis was that dynamic set of facets is more effective and checkboxes are better than links.
The Experiment itself took place primarily at the University of Chicago
We run an exploratory short study with 6 bioscientists.We explained them the 3 types of search and asked them for examples of such searches they use for their work: queries and resources/systems. We used these findings to set up the main study with 10 bioscientists, all of them are researchers in various biological areas: Developmental, Molecular, Cell, Evolutionary, Transcriptional and Systems Biology, as well as Biochemistry, Immunology, Genetics, Population Genetics, and Neuroscience. We emailed them to aks for 4 different queries they use, each with a note on what information they are looking forWe selected two baseline queries and one personal query from each participant. For each query we took screenshots in different systems and isolated the part of the system that shows the interface feature in question (logos were removed).
During each session, which lasted 1-2 hours, we showed participants a PowerPoint presentation with screenshots for one baseline query (of their choice) and for one of their own queries (Table 1). We asked them to rate overall usefulness and aesthetics using a 5 point Likert scale for different handlings of the features. We alsoasked them to rank the systems in order of preference.
The way different systems implement autocomplete varies a lot. They display different values in different order and provide various types of information in addition to the suggestions.
We have collected A LOT of data. It took weeks to summarize and analyze it properly.
We used spreadsheets which doesn’t fit onto one slide even if I minimize it.This is the kind of information we collected about each user – first line of the table.
The remaining lines contain the responses only. Each color is a different interface feature.These are the kind of responses we have collected from the users. This one is for autocomplete.
We have summarized the results in order to find answers to our hypothesis.
This figure summarizes how participants judged the usefulness of interface features for their queries. The length of each bar equals the number of ratings. All 4 browsing participants liked autocomplete, whereas 5 out of 6 participants with info gathering queries rated it as neutral and 1 out of 6 as not useful. This shows a clear difference in usefulness of this feature. For query expansions participants expressed positive or neutral opinions for browsing and mixed opinions for the other search types. Facetted refinement was rated mostly useful for all search types with equal neutral scores for information gathering. Related searches got predominantly neutral or negative reactions, except for browsing, for which they are equally spread. Not surprisingly, it’s useful to see document previews for all search types. Comments analysis shows that snippets are better for browsing, whereas summaries with full sentences for finding specific or specialized information.The results are intuitive and participant’s comments confirm these. Based on these findings, we encourage the interface designers to give priority to “green” elements of the search display and do not bother much about related searches for biologists. Most of them told us that their searches are usually specific and even correctly suggested related searches are not of interest. Access to query expansions is important to experts, but should not be a default feature.
Here are most important findings from our study:Our results agree with those previous research that facets are well received by bioscientists, but autocomplete is less important.But while previous findings have indicated that a large number of facets are needed per query, as implemented in Semedico, we found that 9 out of 10 did not want to see a large number of facetted groups. The large number of choice and the inevitable redundancy overwhelmed them. Our results also confirm findings in that users prefer selecting multiple suggestions using checkboxes.Overall, participants told us that aesthetics are important (and need to be “good enough” to use a system) but what really matters is the content.