Presentation at Information Interaction in Context (IIiX) conference 2014. Best presentation award. Paper available via: humanities.uva.nl/~kamps/publications/2014/huur:from14.pdf

1. From Multistage Information - Seeking
Models to Multistage Search Systems
Hugo Huurdeman, Jaap Kamps!
University of Amsterdam
huurdeman@uva.nl, kamps@uva.nl
!
!
!
Presentation at IIiX 2014, Regensburg

2. 1. Introduction
• Search systems facilitate a rich
diversity of tasks, from simple to
complex
!
• Information seeking models:
• complex tasks involve multiple stages!
• Search systems:
• mainly one-size-fits-all approach
Search
macro
inf.
seeking
stages
micro search
systems
• Paper aim:
• bridging gap between macro level
information seeking models and concrete
search systems

3. 2. Multistage Information !
Seeking Models
macro perspective

4. 2.1 Information Seeking Models
• Information seeking modeled in
a multitude of ways:
• as behavioral patterns (Ellis)
• as nonlinear activities (Foster)
• as problem-solving (Wilson)
• as temporal stages (Kuhlthau), ..
!
• Our main focus:
• temporally based IS models
• Kuhlthau [1991] (Vakkari [2001])
• cognitively complex (work) tasks
• involving learning & construction
information
behavior
information
seeking
information
search
[Wilson99]

5. 2.2 Kuhlthau: Information Search Process [1991]
Initiation Selection Exploration Formulation Collection Presentation
+ uncertainty -
feelings
thoughts
actions
uncertainty optimism confusion clarity confidence (dis)satisfaction
doubt direction
vague focused
seeking relevant
information (exploring)
seeking pertinent
information (documenting)

6. 2.2 Vakkari’s adaptation (in [Vakkari01])
Prefocus Focus formulation Postfocus
seeking general
background information
seeking specific
information
faceted backgr.
information
relevance hard to judge relevance easier to judge
decrease of number of broader terms
information
sought
relevance
search
terms
increase of number of search terms, synonyms, narrower terms!

7. 2.3 Implications for design of search systems
Formulation
• Kuhlthau:
• so far “considerable impact” LIS,
“little impact” IR system design [Kuhlthau99]
• “applicable concepts”: process, uncertainty, uniqueness,
complexity, interest, ..
Initiation Selection Exploration Collection Presentation
Prefocus Focus Postfocus
!
• Vakkari:
• “more support needed in initial stages” [Vakkari00]
• e.g. background information
• potentially support evolving relevance, relevance criteria, search tactics, ..

8. 2.3 Implications for design of search systems
• Observation: good general
understanding of macro level
inf. seeking stages, but hard to
translate to concrete micro level
system design choices.
macro
inf.
seeking
stages
micro system
design

9. 3. Search user interfaces !
supporting seeking
micro perspective

10. 3.1 Search user interfaces supporting seeking
• Search User Interface (SUI) design:
• no straightforward task to design a UI with a high usability [Shneiderm05]
!
• A (limited) number of available frameworks, guidelines and
design pattern libraries for SUIs
• e.g. M. Wilson’s framework of SUI features [Wilson11]
Search
Input
Control
Personalizable
Informational

11. 3.2 SUI approaches: traditional search
• Streamlined
interfaces
!
• Focus on
• query formulation
• result list inspection
• Advantages: [Hearst09]
• lower cognitive load
• more accessible &
understandable
Highly optimized for lookup tasks, less for open-ended queries
[Marchionini06]

12. 3.2 SUI approaches: exploratory search
• Supporting open-ended
inf. seeking
!
• Support learning and
investigation
activities for complex
information problems
[Marchionini06]
• Many potential exploratory SUI features [White09], e.g.
• rapid query refinement, facets (input, control)
• leveraging context, visualizations (informational)
• histories/workspaces/task management (personalizable)
[Yee03]

13. 3.2 SUI approaches: sensemaking & analytics
• Support analysis & synthesis [Pirolli11]
!
• Potential functions facilitating notetaking, hypothesis
formulation & collaborative search [Hearst09]
• some overlap with exploratory search
[Hearst13]

14. [Dunne12]

15. 3.3 Implications for search stage support (1/2)
• Different approaches in support:
• streamlined interfaces (few features)
• complex and analytical interfaces (many features)
!
!
!
!
!
!
!
• Some relation
• exploratory search and early stages Kuhlthau
• sensemaking and intermediate/late stages Kuhlthau
• Few interfaces explicitly support macro level search stages

16. 3.3 Implications for search stage support (2/2)
• Observation: understanding of
search system features at the
micro level, but fragmented
understanding of how they can
support information seeking
stages at the macro level
macro
micro
inf.
seeking
stages
search!
system
features

17. Reconciling macro and micro views
• Idea: could we use the
understanding of the
information seeking models at
the macro level to understand
behavior at the micro level?
macro
micro
inf.
seeking
stages
search!
system
features

18. 4. Interface features and!
search stage
macro — micro perspective

19. 4.1 Interface features & search stage
• Discussed before: Information seeking stages
• effects on information sought, relevance, and search tactics.
!
• Hypothesis: flow of interaction also influenced by stages
Search

20. 4.2 Interface features & stage: previous work
• Approaches: log analysis, eye tracking analysis
!
• Use of specific SUI features in different stages
• e.g. relevance feedback [White05], query suggestions [Niu14]
• Use of various SUI features in different stages
• e.g. [Kules09], [Kules12], [Diriye13]
!
• Findings:
• varying use(fulness) of interface features at different moments of
(complex) search tasks
• search stage sensitive and agnostic features

21. 4.3 An experiment
• Experimental dataset [Tran & Fuhr, 2011]
• ezDL Interface with rich feature-set
• Amazon/LibraryThing book data
• 12 participants
• narrow, complex and self-defined
search tasks
• analysis: 3 complex tasks
• mean task time: 11.4m
!
• eyetracking and system data

22. 4.3 An experiment: findings
60%!
50%!
40%!
30%!
20%!
10%!
0%!
0,1! 0,2! 0,3! 0,4! 0,5! 0,6! 0,7! 0,8! 0,9! 1!
Fixation Count %!
Task Progression (0: start, 1: end)!
DetailView!
ResultView!
QueryView!
BasketView!
ResultView#01-10!
ResultView#11-20!
6!
5!
4!
3!
2!
1!
0!
0,1! 0,2! 0,3! 0,4! 0,5! 0,6! 0,7! 0,8! 0,9! 1,0!
Count!
Task Progression (0:start, 1:end)!
BasketModi"cations!
BasketMeanItems!
• Complex task 1-3: Fixation count over time
• Complex task 1: Basket use over time

23. 4.3 Findings: summary
• Beginning:
• significant focus on query (control
feature)
• low use basket (personalizable)
• Middle:
• increased detail inspections
(informational)
• increased basket activity
• End:
• changes tendencies result list
(deeper items) & detail inspections
• decreasing & increasing use
basket near the end of the task
60%!
50%!
40%!
30%!
20%!
10%!
0%!
0,1! 0,2! 0,3! 0,4! 0,5! 0,6! 0,7! 0,8! 0,9! 1!
Fixation Count %!
Task Progression (0: start, 1: end)!
DetailView!
ResultView!
QueryView!
BasketView!
ResultView#01-10!
ResultView#11-20!
6!
5!
4!
3!
2!
1!
0!
0,1! 0,2! 0,3! 0,4! 0,5! 0,6! 0,7! 0,8! 0,9! 1,0!
Count!
Task Progression (0:start, 1:end)!
BasketModi"cations!
BasketMeanItems!
15!
10!
5!
0!
0,1! 0,2! 0,3! 0,4! 0,5! 0,6! 0,7! 0,8! 0,9! 1,0!
Number of queries!
Task progression (0:start, 1:end)!

24. 4.4 Implications for search stage support
1.No clear dichotomy between
stages: gradual change feature use!
• e.g. basic informational
features, such as results list
macro
micro
inf.
seeking
stages
search!
system
features
2.However, some features useful at
specific moments!
• some input, personalizable features,
such as query and basket mainly
used in specific task stages
• Some support for hypothesis: different flow
of users’ atomic actions at different moments

25. 5. Conclusion and !
Discussion

26. 5. Conclusion and discussion
• Information seeking
behavior (macro level) &
concrete design system
(micro level)
!
• Analysis suggesting
different patterns of use
in different stages
macro
micro
inf.
seeking
stages
search!
system!
design

27. 5. Conclusion and discussion
• Open question: what is the best way to support different
stages in search interfaces?
Search Search
1. streamlined interface 2. complex interface

28. 5. Conclusion and discussion
Search Search Search
Stage 1 Stage 2 Stage 3
3. multistage interface
• Third option: differentiating search stage sensitive features &
offer customized SUI support for stages in complex search
!
• by adaptively showing SUI features
• by adjusting shown details of features
• by changing prominence, position and size of features, …

29. Explorations: INEX Interactive Social Book Search
[M. Hall, H. Huurdeman, M. Koolen, M. Skov, and D. Walsh (2014)]

30. 5. Future work
• Integrating this approach into
the user’s flow
• without being intrusive or confusing
!
• Ways to support evolving
stages, e.g.
• on the interface level (SUI feature
selection & details)
• on the system level (customized
contents and ranking)
!
• Guiding searchers in their
complex search process
Search

31. Acknowledgements
• We are very grateful to Vu Tran
and the Information Engineering
Group at the University of
Duisburg-Essen for allowing us
to analyze data collected with
the ezDL interface.
!
• This research is supported by
the Netherlands Organization
for Scientific Research (NWO
project # 640.005.001 -
WebART)

32. References (1/2)
• [Diriye13] A. Diriye, A. Blandford, A. Tombros, and P. Vakkari. The role of search interface features
during information seeking. In TPDL, volume 8092 of LNCS, pages 235–240. Springer, 2013.
• [Dunne12] C. Dunne, B. Shneiderman, R. Gove, J. Klavans, and B. Dorr. Rapid understanding of
scientific paper collections: Integrating statistics, text analytics, and visualization. JASIST, 63 (12):
2351–2369, 2012.
• [Hall14] M. Hall, H.C. Huurdeman, M. Koolen, M. Skov, and D. Walsh. Overview of the INEX 2014
interactive social book search track. CLEF 2014 Online Working Notes. Sheffield, United
Kingdom.!
• [Hearst09] M. A. Hearst. Search user interfaces. Cambridge University Press, 2009.
• [Kuhlthau91] C. C. Kuhlthau. Inside the search process: Information seek- ing from the user’s
perspective. JASIS, 42:361–371, 1991.
• [Kuhlthau99] C. C. Kuhlthau. Accommodating the user’s information search process: challenges
for information retrieval system designers. Bulletin of the ASIST, 25(3):12–16, 1999.
• [Kules09] B. Kules, R. Capra, M. Banta, and T. Sierra. What do ex-ploratory
searchers look at in a faceted search interface? In
JCDL, pages 313–322. ACM, 2009.
• [Kules12] B. Kules and R. Capra. Influence of training and stage of
search on gaze behavior in a library catalog faceted search
interface. JASIST, 63:114–138, 2012.
• [Marchionini06] G. Marchionini. Exploratory search: from finding to understanding. CACM, 49(4):
41–46, 2006. …

33. References (2/2)
• [Niu14] X. Niu and D. Kelly. The use of query suggestions during information search. IPM, 50:218–
234, 2014.
• [Pirolli11] P. Pirolli and D. M. Russell. Introduction to this special is- sue on sensemaking. Human-
Computer Interaction, 26:1–8, 2011.
• [Shneiderm05] B. Shneiderman and C. Pleasant. Designing the user in- terface: strategies for
effective human-computer interaction. Pearson Education, 2005.
• [Tran11] V. T. Tran and N. Fuhr. Quantitative analysis of search sessions enhanced by gaze
tracking with dynamic areas of interest. In Theory and Practice of Digital Libraries, volume 7489 of
LNCS, pages 468–473. Springer, 2012.
• [Vakkari01] P. Vakkari. A theory of the task-based information retrieval process: a summary and
generalisation of a longitudinal study. Journal of Documentation, 57:44–60, 2001.
• [Wilson99] T. D. Wilson. Models in information behaviour research. Journal of Documentation,
55:249–270, 1999.
• [Wilson11] M. L. Wilson. Interfaces for information retrieval. In I. Ruthven and D. Kelly, editors.
Interactive Information Seeking, Behaviour and Retrieval. Facet, 2011.
• [White05] R. W. White, I. Ruthven, and J. M. Jose. A study of factors affecting the utility of implicit
relevance feedback. In SIGIR, pages 35–42. ACM, 2005.
• [White09] R. W. White and R. A. Roth. Exploratory search: Beyond the query-response paradigm.
Synthesis Lectures on Information Concepts, Retrieval, and Services, 1:1–98, 2009.

34. From Multistage Information - Seeking
Models to Multistage Search Systems
Hugo Huurdeman, Jaap Kamps!
University of Amsterdam
huurdeman@uva.nl, kamps@uva.nl
!
!
!
Presentation at IIiX 2014, Regensburg