Presented at the PoliMedia symposium: http://www.polimedia.nl/polimedia-symposium-linking-political-debates-and-media/

1. User research in the
development of PoliMedia
Max Kemman
Erasmus University Rotterdam

2. User research
• Understanding the user
– Acceptance
– Performance
– Capabilities
– Weaknesses
• Goal
– Creating a system that is intuitive and helpful to
the users
PoliMedia symposium: User research in the
2
development of PoliMedia

3. User research in the development
process
• Review related search systems
– Staten Generaal Digitaal
– Political Mashup
– Newspaper database KB
– Academia
• Learn search behaviour from users
– Survey regarding search strategies
– Interviews
• User wishes → user requirements
• Wireframes
• Prototype
• Evaluation
PoliMedia symposium: User research in the
3
development of PoliMedia

4. Don’t
Sometimes
Very often
Often
Regularly
Never
know it
Google
• N=294
Google Images
Google Scholar
YouTube
JSTOR
Uitzending Gemist
KB
• Popular search engines
Flickr
EBSCO
Survey
Nationaal Archief
Web of Knowledge
Yahoo!
Bing
Academia.nl
General search strategies
Europeana
Scopus
Microsoft Academic Search
4
EUscreen
Arkyves

5. Survey
General search strategies
1. Keywords 4,75
2. Advanced search 3,36
3. Related terms 2,52
4. Boolean 2,42
5. Browsing subject
categories 2,29
6. Filters 2,19
7. Thesaurus 1,87
8. Visualization 1,22
PoliMedia symposium: User research in the
5
development of PoliMedia

6. Survey
Conclusions
• Google is the dominant search engine
• This has two consequences
1. People compare other search systems to their
experience with Google
2. The search task is mainly performed by using
keywords
PoliMedia symposium: User research in the
6
development of PoliMedia

7. Interviews
• N=5
• Quantitative (n=2) as well as qualitative (n=4)
• Main themes
– How do people search currently?
– What could be improved about current search systems?
– What should PoliMedia offer, given its goals?
• Results
– 39 user wishes
– Prioritized internally
• 19 user wishes deemed out of scope
• 20 user requirements
PoliMedia symposium: User research in the
7
development of PoliMedia

8. Interviews
Findings
• Key issue is to provide a good overview of data
– Why are search results retrieved
– How are search results ranked
• Assumptions of relevance
– Higher frequency of keywords indicated higher relevancy to query?
– Longer segments (speeches and articles) indicate higher importance?
• Many more or less out-of-scope wishes to make current research
easier
– Sentiment-metadata
– Context metadata
• Roles of people
• Links to external databases
• Types of documents
• Types of presentation (dramatic, humoristic, etc.)
– Ability to export to own software
PoliMedia symposium: User research in the
8
development of PoliMedia

9. Wireframes
Search interface
• Clear and
immediate
keyword-search
• Support for
Booleans and
(some) Google-
search operators
• Separate
advanced-search
PoliMedia symposium: User research in the
9
development of PoliMedia

10. Wireframes
Search results
• Keyword search
remains
prominent
• User chosen
ranking of results
• Keyword
highlighting
• Overview of
related media
• Support for
filtering
PoliMedia symposium: User research in the
10
development of PoliMedia

11. Wireframes
Debate page
• Keyword search
remains
prominent
• Overview of
people in debate
• Easy access to
related material
11

12. Evaluation
• General usability evaluation
– N=5
– Check user requirements
– General usability issues
• Search behaviour evaluation
– What role do facets play in the search behaviour?
– What is the influence of a uniform search
interface?
PoliMedia symposium: User research in the
12
development of PoliMedia