1. 토요인지모임 2012 2월 발표
Interactive
Information Retrieval
2012.2.18
박정아
sunseed9@gmail.com
http://twitter.com/sunseed9

2. Contents
§ Interactive Information Retrieval
§ Models of Information Seeking Behavior
§ Evaluation & Relevance
§ Search User Interfaces
§ Towards User-centered Search

3. Information Technology

4. Background
- 컴퓨터과학 (Computer Science) + 국어국문학
- 자연어처리 (Natural Language Processing)
- 인지과학 (Cognitive Science)
- 정보검색 (Information Retrieval)
Interacti
Human Computer

5. 검색

6. Information Retrieval System

7. Interactive Information
Retrieval

8. Human centered approach in Information
Retrieval
§ Cognitive viewpoint in information retrieval
§ Traditional IR model concentrates on matching not user side &
interaction
§ interactive IR
§ more than simply developing interfaces for searching
§ Shneiderman, Byrd, & Croft, 1998
§ the strength of good research in IIR comes not only from a technical knowledge of
interactive systems development but also from a knowledge of people’s search
behavior and search context, including the environmental factors that influence
behavior
§ Fidel & Pejtersen, 2004
§ HCIR (Human Computer Information Retrieval)
§ Human-Computer Interaction + Information Retrieval
§ HCI & IR
§ HCI and IR come from different traditions;
§ HCI, for example, places more emphasis on the published literature on usability
§ whereas IR emphasizes effectiveness.

9. Interactive information retrieval (IIR)
§ “ Interactive information retrieval (IIR) is, of its nature, cognitive. One
important goal for IIR research is to study user interaction with a
search system to learn about the user’s search intent and when they
encounter relevant documents. Taking account of the user and her
context has potential to improve understanding of the search process
and the user’s intent. A search system that “knows” this information
can improve its performance in retrieving documents that satisfy user’s
needs. Awareness of demands imposed on user’s cognitive
processing and levels of user’s knowledge can also contribute to
improvements in system performance”
§ from "Inferring Cognitive States from Multimodal Measures
in Information Science"
§ interactive IR is more than simply developing interfaces for searching
(Shneiderman, Byrd, & Croft, 1998) and that the strength of good
research in IIR comes not only from a technical knowledge of
interactive systems development but also from a knowledge of people’s
search behavior and search context, including the environmental
factors that influence behavior (Fidel & Pejtersen, 2004).

10. Interactive Information Research problems
§ IIR research addresses three major problem areas
§ (1) understanding information seeking needs and behaviors;
§ (2) developing retrieval systems that respond to information needs and
support information seeking behaviors and interactions;
§ (3) developing methods and measures to study and evaluate behaviors,
interactions and systems.
§ issues
§ information seeking behavior related information needs and query intent
§ Models of the Information Seeking Process
§ Design of Search User Interfaces
§ Presentation of Search Results
§ includes document surrogates, properties of results listings, summaries (snippets)
as used in search results
§ How people evaluate IR systems / Search Quality

11. Models of information
seeking behavior

12. Models of the Information Seeking Behavior
§ Bates’ berrypicking – acts in searching
§ Dervin's sense-making theory – gap, bridge
§ Ellis’s Information Seeking Process
§ Kuhlthau’s information search process
§ Ingwersen's cognitive model
§ Wilson's information-seeking behaviour model
§ Saracevic's model of stratified interaction

13. Models of the Information Seeking (1)
§ Bates' Berrypicking - “dynamic” not “static”

14. Models of the Information Seeking (2)
§ Dervin's sense-making theory
questioning that can reveal the nature of a
problematic situation, the extent to which
information serves to bridge the gap of uncertainty,
confusion, or whatever, and the nature of the
outcomes from the use of information.
Figure 3: Dervin's 'sense-making'
§ A problem-solving mode
§ The solution of the problem, the resolution of the discrepancy, the
advance from uncertainty to certainty

15. Models of the Information Seeking (3)
§ Ellis’s Information Seeking Process
Figure 5: A process model based on Ellis's 'characteristics'
§ Kuhlthau’s information search process
Figure 2.2: Kuhlthau's model of the search process

16. Models of the Information Seeking (4)
§ Ingwersen's cognitive model
§ 'traditional' model
§ represents IR as a two prong set (system and user) of elements and processes converging
on comparison or matching...';
§ Ingwersen's cognitive model [27], (cognition)
§ concentrates on identifying processes of cognition which may occur in all the
information processing elements involved.

17. Models of the Information Seeking (5)
§ Saracevic's model of stratified interaction
§ Saracevic then goes on to propose what he calls a 'stratified interaction model' developed within an overall
framework of an 'acquisition-cognition-application' model of information use.
The levels or strata posited by Saracevic
are simplified (in his words) to three:
1. surface, or the level of interaction
between the user and the system
interface;
2. cognition, or the level of interaction
with the texts or their representation
3. situation, or the context that
provides the initial problem at hand.
Figure 9: Saracevic's model of stratified

18. Evaluation & Relevance

19. Evaluation of interactive information retrieval
systems with users
§ THE EVALUATION OF SEARCH USER INTERFACES
§ Measure
§ search interfaces are usually evaluated in terms of three main aspects of usability:
effectiveness, efficiency, and satisfaction, which are defined by ISO 9241-11, 1998 as:
§ Effectiveness
§ Accuracy and completeness with which users achieve specified
goals.
§ Efficiency
§ Resources expended in relation to the accuracy and completeness
with which users achieve goals.
§ Satisfaction
§ Freedom from discomfort, and positive attitudes towards the use of
the product.

20. Evaluation of interactive information retrieval
systems with users
§ STANDARD INFORMATION RETRIEVAL
EVALUATION
§ Text REtrieval Conference (TREC), run by the U.S. National Institute of
Standards (NIST) for more than 15 years (Voorhees and Harman, 2000)
§ The most common evaluation measures used for assessing ranking
algorithms
§ Precision, Recall, F-measure, Mean Average Precision (MAP).
§ The TREC evaluation method has been enormously valuable for
comparison of competing ranking algorithms.
§ discounted cumulative gain (DCG) (Järvelin and Kekäläinen, 2000,
Kekäläinen, 2005)

21. USABILITY TESTING - the results of an eye-tracking study
Arrows indicate dominant directions of eye movement; “hotter” colors indicate more frequent eye
fixations, and X's indicate locations of clicks. From Search Engine Results: 2010, by Enquiro Research

22. The Dynamics of relevance judgment and its
order effect
§ dynamics of relevance
§ change-in-meaning hypothesis => direct impression
§ subject's relevance judgments would vary as a consequence of the order of presentation
§ first received information affected participants' impression of the following information
§ learning effect (Harter, 1992)
§ fatigue effects ( clancy & wachsler 1971)
§ information item이 어느정도로 증가하면 subject는 너무 fatigue해서 carefully하게 반응 어려움.
§ the dynamics of relevance judgment
§ (a) the changing external task situation and demand that modify a user’s
information need (Cuadra & Katter, 1967; Park, 1993);
(b) the changing cognitive state of the user as a result of encountering relevant
documents (Harter, 1992; Xu, 2007b);
(c) the different modes of document presentation
§ such as showing only the title, the abstract, bibliographic information, or the whole
document content (Ingwersen & Järvelin, 2005).
§ belief-adjustment model
§ Hogarth & Einhorn 1992

23. Summary styles compared in an experiment
by Aula, 2004.

24. Brand Awareness and the Evaluation of
Search Results
검색 엔진 브랜드가 검색 결과 품질에 미치는 영향 (The Effect of Brand Awarenes On the Evaluation of Search Engine
Results)
• 연구 내용 요약
◦ 사람들에게 같은 검색 결과가 제시되어도 검색 엔진이 무엇이냐에 따라 검색 품질 평가가 달라진다는 것을 재미
있는 실험을 통해 증명함으로써, 검색 엔진 브랜드가 검색 품질 평가의 한 요소임을 밝힌 연구
• 실험 방법 및 결과
◦ 사용한 쿼리
■ camping mexico, laser removal, manufactured home, techo music
■ 150만개 가량의 e-commerce 검색 로그로 부터 분야별로 4개의 쿼리 선정
◦ 검색 결과
■ 선정된 4개의 검색 쿼리를 구글 검색에 던져서 검색 결과 저장
◦ 검색 로고
■ 각각 Google, MSN, Yahoo의 로고들을 캡쳐하여 사용하고, AI2RS라는 새로운 검색 브랜드 로고 추가로
생성하여 사용
• 실험 대상
◦ 18~25세 사이의 미국 대학생 32명( 남자 24명, 여자 8명)
• 실험 결과
■ 결과적으로 이름 없는 AI2RS 는 평균적으로 10% 떨어지는 평가를 받았다.
■ Yahoo는 4개 쿼리 모두 평균 이상의 평가를 받으며 높은 브랜드 인지도를 나타내었다.
• 결론
◦ 검색 성능(품질) 평가에서 검색 엔진 브랜드의 영향을 살펴본 결과, 동일 한 검색 결과라도 검색 엔진 브랜드에
대한 인식에 따라 검색 품질 평가에 상당한 영향을 미칠 수 있다는 것을 알 수 있다.

26. 적합성 (Relevance)
• 시스템 중심 적합성 vs 사용자 중심 적합성
시스템 중심 사용자 중심
논리적 (Cooper, 1971) 심리적 (Wilson, 1973)
주제적 (Cooper, 1971; Park, 1994) 상황적 (Wilson, 1973; Harter, 1993)
객관적 (Swanson, 1986; Howard, 1994) 주관적 (Swanson, 1986; Howard, 1994)
적합성 분류 (출처. Maglaughlin & Sonnenwald, 2002)
• 시스템 중심 적합성
– 벡터 스페이스 적합성(vector space relevance), 확률 적합성
(probabilistic relevance), 불린 적합성(Boolean relevance)
• Borlund, 2003

27. 사용자 중심 적합성
• 사용자 중심 적합성 연구 활발
• 특히 90년대 연구 집중
– Froehlich, 1994; Green, 1995; Harter, 1992; Janes, 1994; Mizzaro, 1998; Park,
1994; Saracevic, 1996; Schamber, Eisenberg, & Nilan, 1990
• 사용자 중심 적합성에 대한 정의
– “적합성이란 다차원의 인지적 개념으로써 사용자의 정보 인식과 정
보 이용자의 정보 요구 상황에 상당 부분 의존한다” (Borlund, 2003)
– 상황 적합성(Schamber, 1990), 심리적 적합성(Harter, 1992), 과제
기반 적합성 (Cosijn, 2000; Reid 1999) 등

28. Saracevic의 적합성 분류(1996)
• 시스템 중심 적합성
– 시스템 적합성
• 검색어와 문서간의 유사도
• 사용자 중심 적합성
– 주제 적합성
• 검색어와 문서의 주제
– 인지 적합성
• 문서가 정보 이용자의 지식 상태와
인지적인 정보 요구에 얼마나 잘
부합하는지
– 상황 적합성
• 문서가 상황이나 현재 문제에 얼마
나 잘 적합한지

29. Saracevic’s stratified model of IR interaction
Context
social, cultural
Situational
tasks; work context...
TA
Affective
RA
intent; motivation ...
ST
tion
R
Cognitive
USE
pta
knowledge; structure...
Ada
Query
CE
characteristics …
N
VA
Surface level
LE
RE
INTERFACE
tion
N&
a
TIO
orm
Engineering
AC
f
of in
ER
hardware; connections...
INT
ion
R
Processing
Use
UTE
ptat
software; algorithms …
MP
Ada
Content
CO
inf. objects; representations...
Tefko Saracevic
28

30. 사용자 적합성 판단 기준 연구들
출처 도메인 참가자 판단 기준
개수
Schamber, 1991 날씨정보검색 직장인 30명 10
Park, 19913 석사논문연구주제 검색 대학원생 11명 22
Barry, 1994 온라인 정보 검색 학생 18명 23
Spink et al. 1998 연구 정보 검색 교수와 학생 11명 27
Bateman, 1998 논문검색 대학원생 35명 40
Wang & Sorel, 1998 연구 프로젝트 검색 대학원생 25명 11
Tang & Solomon, 1998 기말 논문 검색 대학원생 1명 10
Hirsh, 1999 스포츠 검색 초등학생 10명 14
Maglaughlin & 논문검색 대학원생 12명 29
Sonnenwald, 2002
Choi & Rasumuseen, 이미지 검색 학생 38명 9
2002
Savolainen & Kari, 2006 웹 검색 학생 9명 18

31. My Dissertation
정보 검색에서의 사용자 중심 적합성 판단 모형 개발 및 평가
• 개요 (Overview)
– 한국 통합 검색 환경에서의 사용자 적합성 판단에 관한 연구
– 정보 검색 과제 별 사용자가 적합성을 판단하는 기준과 적합성 유형과의
관계
• 연구1. 사용자 적합성 판단 기준에 관한 탐색적 연구
– 한국 통합 검색 환경
– 반구조(semi-structured) 인터뷰
• 연구2. 사용자 적합성 판단 모형 개발
– 정보 검색 과제별 적합성 판단 기준과 적합성 유형의 관계
– Xu & Chen 적합성 판단 기준 정량적 연구 기반
– 반제어(semi – controlled) 설문
– “적합성 유형”별, “정보 검색 과제”별
• 연구 3. 정보 검색 과제별 동적 검색 랭킹 모델 구현 및 검증
– 사용자 적합성 판단 기준 랭킹 요소로 정보 검색 시스템 반영
– 사용자 평가 비교 실험
• 정적 검색 랭킹 모델 vs 동적 검색 랭킹 모델

32. 정보 검색 과제
• 정보 검색 과제에 따른 사용자 행동 또는 반응 기존 연구
Navarro-Prieto, White, Jose,
Kelly et al. Limberg, L.
Scaife, & Ruthven Freund (2008) This research
(2002) (1999)
& Rogers (1999) (2003)
Fact finding Fact-finding fact search Fact finding 사실 검색
(fact question) ex) a named
Fact finding person’ s current
Ex)“How long does email address
it take to get a
passport?
Procedural Understanding b a c k g r o u n d Learning 문제 해결 검색
a topic search
(task question)
ex) dust allergies How to
Ex) “How do I get
a passport?”
Solve a problem
Exploratory decision search Make a decision 의사 결정 검색
assessing an
ex) find Rome’ s
issue a n d best museum
reaching a f o r
decision impressionist
art

33. Search User Interfaces

34. Search User Interfaces

35. Presentation of Search Results (Ranked
List)
Search results listings from Infoseek in 1997 (left) and Google in 2007 (right), courtesy Jan Pedersen.

36. Designing Search for Humans - Provide
Memory Aids
Suggest the Search Action in or near the Query Form
www.yelp.com, www.powerset.com

37. Memory Aids
Provide Access to Recent Actions
PubMed
amazon.com Dumais et al., Stuff I’ve Seen, SIGIR 2003

38. Memory Aids; Anchoring Aids
Dynamic Query Suggestions
http://www.daum.net
http://google.com
38

39. Suggest Next Steps: Query suggestions
Show suggestions after the query has been issued.
http://yahoo.com
http://bing.com
39

40. Suggest Next Steps: Query suggestions
PubMed
40 http://nextbio.com

41. Putting It All Together: Faceted Navigation
§ Suggests next steps
§ Helps with Vocabulary Problem and Anchoring
Problem
§ Promotes Flow
§ Show users structure as a starting point, rather than
requiring them to generate queries
§ Organize results into a recognizable structure
§ Eliminates empty results sets
41

42. A New Development: Faceted Breadcrumbs
Nudelman, http://www.boxesandarrows.com/view/faceted-finding-with
42

43. Towards
User-centered Search

44. Search is not easy
Frowns, Sighs, and Advanced Queries -- How does search behavior
change as search becomes more difficult?

45. Instant Search
43

46. http://goo.gl/p0Alw

47. Interface in Mobile (1)
청각적(voice) 정보를 이용한 검색 시각적(visual) 정보를 이용한 검색

48. Interface in Mobile (2)
초성검색 통합웹 컷오프
모바일 화면의 제약으로 인한 컴팩트한 검색 결과 제공

50. 'Natural' Search User Interfaces
§ 'Natural' Search User Interfaces | November 2011 |
Communications of the ACM
§ 사람들은 자연스러운(natural) 인터페이스 - 타이핑 입력
보다 말하기를, 텍스트 읽기 보다 동영상 보기 - 를 원하
고, 혼자가 아닌 함께(social)를 원한다.
§ "Users will speak rather than type, watch video rather than read,
and use technology socially rather than alone"
§ 따라서 검색 인터페이스는 natural 과 social 을 지원하는
방향으로 발전해 가야한다.
§ Siri / Social Search

51. Siri, artificially intelligent voice search assistant

52. 소셜 검색 (What is Social search?)

53. Trapit, Discovery Engine
[Curation]
• 인간의 요소, 인간만이 - 패턴을 인식하는 인간 고유의 능력 @ <큐레이션, 인간을 지향하다>
"프로그래머와 큐레이터로서 인간의 역할이 사라지는 일은 일어나지 않을 거에요. 컴퓨터가 절대로 따라올 수
없는 부분이 있으니까요. 그게 바로 인간의 요소, 인간만이 떠맡을 수 있는 부분이죠"

54. Beyond Search
“Contextual discovery will
take data gathered from
people’s browsing data and
location profiles and use it to
serve up interesting and
relevant results – without the
search”

55. Thank You!
sunseed9@gmail.com
http://twitter.com/sunseed9