In just a little over half a century, the field of information retrieval has experienced spectacular growth and success, with IR applications such as search engines becoming a billion-dollar industry in the past decades. Recommender systems have seen an even more meteoric rise to success with wide-scale application by companies like Amazon, Facebook, and Netflix. But are search and recommendation really two different fields of research that address different problems with different sets of algorithms in papers published at distinct conferences?
In my talk, I want to argue that search and recommendation are more similar than they have been treated in the past decade. By looking more closely at the tasks and problems that search and recommendation try to solve, at the algorithms used to solve these problems and at the way their performance is evaluated, I want to show that there is no clear black and white division between the two. Instead, search and recommendation are part of a much more fluid continuum of methods and techniques for information access.
(Keynote at "Mind The Gap '14" workshop at the iConference 2014 in Berlin, Germany)
1. Search & Recommendation: Birds of a Feather?
Toine Bogers
Aalborg University Copenhagen
Copenhagen, Denmark
âMind the Gap â14â workshop @ iConference 2014, Berlin
March 4, 2014
4. Success of search engines
⢠Search engines have had a huge impact on the information economy
- Academia
⣠Vibrant & growing research community with many dedicated conferences and journals
⣠Evaluation initiatives like TREC were shown to have a great impact on the performance
of Web search engines
- Industry
⣠Google â ~13 billion USD in proďŹt in 2013
4
6. Success of recommender systems
⢠Recommender systems have seen a meteoric rise to success in
past two decades
- Academia
⣠From specialized workshops to dedicated conference and journals
- Industry
⣠Amazon â 35% sales from recommendations
⣠NetďŹix â 75% of what its users watch comes from recommendations
⣠Google News â recommendations generate 38% more click-through
6
7. Different perspectives?
⢠Search and recommendation are commonly treated as diďŹerent (but
related) research areas
⢠Search perspective â recommendation is a special type of search
problem
- Smaller research community with few dedicated venues
- Dedicated sessions at major IR conferences
⢠Recommendation perspective â ďŹeld of its own
- Rapidly growing research community with s
- Strong industry support
- Separate data sets, experimental protocol, and evaluation
7
8. ....but are they really that different?
⢠Looking at search and recommendation in isolation can be
counter-productive in many situations!
⢠Three aspects of where both ďŹelds are (growing) close(r)
- Use cases
- Algorithms & evaluation
- Trends
8
10. Comparing use cases
⢠What are the characteristics of the information access paradigms?
- What problem are they trying to solve?
- What do we know about what the user wants?
- What do we know about the user?
- How do we know we have solved the userâs problem?
10
11. Comparing deďŹnitions
tions
âA recommender system is software that provides sugges
to users on which items could be of use to them.â
â Ricci et al. (2011)
retrieval (IR) is ďŹnding material of an
âInformation
ation
ructured nature that satisďŹes an inform
unst
need from within large collections.â
â Manning et al. (2008)
11
12. Search characteristics
⢠Information need
- Explicit representation of userâs information need as a query (and
occasionally a description or narrative)
⣠Typically at Taylorâs last two stages
Taylorâs four stages
1. Visceral
2. Conscious
3. Formalized
4. Compromised
⢠Knowledge about the user
- User characteristics traditionally abstracted away
- More focus on the user in recent years (e.g., search history)
12
13. Search characteristics
⢠Matching criteria
- Relevance
⣠Assessment of perceived topicality, pertinence, usefulness or utility of an information
source by an actor or algorithm with reference to a task at a given point in time
- Relevance is a multi-dimensional concept â many different ďŹavors!
⣠Topical relevance most common interpretation
⣠Textual similarity used as a proxy for topical relevance
Saracevicâs categories
⢠Algorithmic relevance
⢠Topical relevance
⢠Temporal relevance
⢠Situational relevance
- See Borlund (2003) for a comprehensive overview of relevance in IR
13
14. Recommendation characteristics
⢠Information need
- Implicit representation of userâs information need as a the userâs proďŹle
⣠Typically at Taylorâs ďŹrst two stages
⢠Knowledge about the user
Taylorâs four stages
1. Visceral
2. Conscious
3. Formalized
4. Compromised
- User proďŹle representing the userâs interests
- Usage patterns, past interactions with the system, requirements
⢠Matching criteria
- Interest / Usefulness
14
15. No user proďŹle
Classic IR
Explicit
need
Web search
PopularityBrowsing based
methods
Implicit
need
Information
ďŹltering
?
Recommendation
User proďŹle
15
16. Search & recommendation form a continuum
⢠Search (âShow me all books about Xâ)
⢠Focused recommendation (âShow me interesting books about X!â)
⢠Recommendation (âShow me interesting books!â)
Search
Focused
recommendation
Recommendation
16
18. How prevalent is focused recommendation?
⢠Is there evidence for such a continuum?
- Search engines see millions of pure search requests every day
- NetďŹix and Amazon proďŹt immensely from pure recommendation scenarios
- But how prevalent are these focused recommendation requests?
⢠Possible explanations for underrepresentation
- Perhaps we are looking in the wrong places?
- Interfaces offer little support for entering complex requests
18
19. INEX Social Book Search track
⢠Track running at INEX from 2011-2014 on book search
- Amazon/LibraryThing collection
⣠2.8 million book metadata records
⣠Mix of metadata from Amazon, Librarything, Library of Congress, and British Library
- Realistic book requests & information needs from LibraryThing fora
⣠Highly varied set of requests that touch upon topics, genres, authors, engagment,
reading level, personal preferences, etc.
19
21. INEX Social Book Search track
⢠Track running at INEX from 2011-2014 on book search
- Amazon/LibraryThing collection
⣠2.8 million book metadata records
⣠Mix of metadata from Amazon, Librarything, Library of Congress, and British Library
- Realistic book requests & information needs from LibraryThing fora
⣠Highly varied set of requests that touch upon topics, genres, authors, engagment,
reading level, personal preferences, etc.
⣠Collected & annotated 944 book requests from the LibraryThing fora
- Relevance judgments
⣠Member suggestions (Suggestions made by other Librarything members)
⣠Reading behavior (Has the original requester added any suggestions afterwards?)
21
22. Relevance aspects of book requests
⢠Eight LIS students annotated all requests on relevance aspects
Relevance aspects
%
Accessibility
16
Content
74
Engagement
23
Familiarity
36
Known-item
21
Metadata
28
Novelty
4
Socio-cultural
14
0
10 20
30
40
50
60
70
80
22
23. Continuum of search & recommendation
⢠How common are the different types of information needs?
Familiarity
No familiarity
Content
Focused
recommendation
(260 requests)
Search
(338 topics)
No content
Recommendation
(66 topics)
Context
(78 topics)
Sign up at https://inex.mmci.uni-saarland
.de/tracks/books/!
23
25. Whatâs next?
⢠Focused recommendation deserves more attention!
- Combines aspects of search and recommendation
⢠Open questions
- How can we best address focused recommendation requests?
⣠Likely to require a combination of both search and recommendation approaches
⣠Early indications from INEX track that a combination indeed works best
- How can we support expressing these complex needs through the UI?
25
27. Algorithms & evaluation
⢠Past decade has seen combination & mutual inspiration
- Both ďŹelds have borrowed techniques & metrics from each other
- Dedicated workshops & events
⣠CARR 2011-2014
⣠BARR 2013
⣠Mind The Gap 2014
27
28. Recommender systems â IR
⢠Collaborative ďŹltering
- Automates the process of word-of-mouth recommendations by looking for
unseen items among other users with similar interests
⢠Used in IR for
- Collaborative search
⣠I-SPY search engine by Smyth et al. (2004)
- Query suggestion
- Improving 'More like this' functionality
28
29. IR â Recommender systems
⢠Recommender systems has borrowed from many different ďŹelds
- ArtiďŹcial Intelligence (ML, CBR), IR, Natural Language Processing
⢠Inspiration from IR
- Algorithms
⣠TF¡IDF weighting scheme for CF (Breese et al., 1998)
⣠Query expansion for recommender systems (Formosa et al., 2013)
⣠Probability ranking principle in recommender systems (Wang et al., 2006)
⣠Language modeling for recommender systems (Bellojin et al., 2013)
- Evaluation
⣠Increasing use of nDCG (and MAP) as metrics for ranked list recommendation
29
32. Diversity
⢠Ensuring a diverse range of relevant results/recommendations
⢠Search
- IDR workshop (2009)
- DDR workshop (2011-2012)
- Many publications addressing diversity in search results
⢠Recommendation
- DiveRS workshop (2011)
- Many publications addressing diversity in recommender systems
32
33. Privacy
⢠Protecting user privacy when generating results/recommendations
or releasing data sets
- Hot topic in the aftermath of release of AOL and NetďŹix data sets
- Many papers on how to (de-)anonymize of recommendation data sets and
search logs
⢠Search
- PIR workshop (2014)
⢠Recommendation
- RESSON workshop (2013)
33
35. Conclusions
⢠Search & recommendation form an information access continuum
- Pure search & recommendation needs are addressed well by the respective
research ďŹelds
- But many other information needs fall through the cracks!
⣠Need to look at the whole range of information needs
⣠Both in terms of algorithms and interface design
⢠Search & recommendation are already moving closer together
- Exchange of algorithms & techniques
- Shared evaluation metrics
- Similar research trends
⢠A continuum of requests requires a continuum of solutions!
35
38. Example requests
CONTEXT REQUEST
I've just ďŹnished my undergraduate
work, and as I ďŹoat into the real
world, I ďŹnd myself missing books-and recommendations for books--in a
serious way. So, those of you in a
similar state (and those of you who
simply love reading, and sharing):
have any books that you ďŹnd essential
for living? I'll post what I've been
reading, and you can as well...I'm
especially interested in books that are
a little older, a little less known, and
more prone to ďŹying under the radar.
I read almost everything as well, a
sentiment I'm sure most of you are
familiar with.
SEARCH REQUEST
looking for heroine oriented love
triangle romances,
any recommendations appreciated.
RECOMMENDATION REQUEST
Just read and reviewed Moon in the
Water: ReďŹections on an Aging Parent.
I wonder if other early readers have
recommendations for similar
pieces...this makes me want to go
back and read The Summer of the
Great-Grandmother by Madeline
L'Engle. I glossed through it the ďŹrst
time, but now that I am closer to that
stage of life I wonder if it will have
more meaning.
38