CS410 Survey on Preference Elicitation in Recommender Systems

1. Образец заголовка
Preference elicitation in
recommender systems
by Anish Shenoy and Siddharth Atreya
Prepared as an assignment for CS410: Text Information Systems in Spring 2016

2. Образец заголовкаMotivation
• Modern consumers are inundated with choices!
– Which movie should I see?
– What web page has the information I need?
– Where should I order food from?
– Which places should I visit on my holiday?
– Which book should I buy?
• Recommender systems help the decision process by suggesting items that a user might value based on
the space of possible items – this has become a major aspect of user experience
• Collaborative filtering is the de facto mechanism for recommendation
– The recommender compares user’s ratings to other users to find similar users, and recommends
items
• When a new user comes along, the system knows nothing about the user! – this is called the cold start
problem
The winners of the Netflix Prize. Source: NYT

3. Образец заголовкаThe new user problem
• Pleasing new users is challenging, since they judge the system based on their first
experiences – this is an important issue for startups looking to acquire customers!
• A direct way to profile users is to ask new users about their opinions on a few products
• A good interview must be designed so that it increases recommendation accuracy and
decreases user interaction effort

4. Образец заголовкаThe Elicitation Process
• It is the process of gathering relevant information
from users to learn their needs/interests.
• It generates the user profile that aids the
recommendation process
• The goal: Given a set of items, how to decide which
items to elicit ratings on, and how to design the
rating process?
• User’s interests need to be explored/exploited in a
flexible manner!

5. Образец заголовкаElicitation Techniques
In this survey, we have covered recent papers which adopt various ways to design
the elicitation process such as:
• Selecting which items to show in the interview process
• Adapting the interview based on the user’s response
• Integrating decision tree techniques with matrix factorization techniques
• Selecting representative items to elicit ratings
• Selecting a group of items as well as assigning points to different categories
• Selecting images from a group of images
These techniques either develop new approximations of the ratings matrix, or
develop new ways to map the user’s response to a user profile

6. Образец заголовкаSelecting the right items to rate
Main Idea
• New users are asked to rate items from a seed set
• Seed set previously chosen using criteria such as popularity, contention
and coverage
• However, these criteria are arbitrary and different combinations might be
required for optimal predictions
• Seed set items selected in a principled manner by minimizing a cost
measure
Technique
• Given a user preference prediction algorithm A, and a cost measure F, use
a greedy algorithm to iteratively add items which minimize the cost
measure
• Specifically, A is the factorized item-item model, and cost measure is RMSE
Golbandi, N., Koren, Y. and Lempel, R. 2010. On Bootstrapping Recommender Systems. Proceedings of the 19th ACM International Conference on Information
and Knowledge Management(New York, NY, USA, 2010), 1805–1808.
RMSE =
s X
(u,i)2T
(rui ˆrui)2/(|T|)

7. Образец заголовка
Selecting the right items to rate
(contd.)
Results
• Generated 200 seed sets from the Netflix
dataset, size varying from 1 to 200
• Error rates decrease as items in the seed set
increase
• Improves test set prediction accuracy over other
techniques of selecting seed sets such as
Entropy, Coverage, and HELF
• Requires 2-3x less items to achieve similar RMSE
scores compared to closest competitor
technique (Var)
Drawbacks of method
• Current implementation of GreedyExtend is
slow, compared to other techniques for
generating seed set – takes 8 hours vs
instantaneous generation for other methods
• Authors suggest parallelizing seed set selection,
and discarding unpopular items from the set of
candidate items

8. Образец заголовкаAdaptively choosing items to rate I
Main Idea
• Static seed sets (previous slides) mean all users rate the
same set of items, and the system does not exploit the
growing knowledge about the user
• Use an adaptive interview process, where a user’s
previous response determines the next question – this
improves accuracy and user experience
Technique
• Uses a decision tree to evaluate user preferences – the
path taken by a user from the root to a leaf is used for
generating recommendations
• User evaluates each movie as ”like”, “dislike”, or
“unknown”
• Each node predicts ratings for an item by taking the mean
ratings of corresponding users, and the error for that node
and item is the squared difference between actual ratings
and predicted ratings
• Decision tree constructed in top down manner – splitting
item chosen so that the error is minimized within subtrees
Golbandi, N.,Koren,Y. and Lempel, R. 2011. Adaptive Bootstrapping of Recommender Systems Using Decision Trees. Proceedings of the Fourth ACM International Conference
on Web Search and Data Mining (New York, NY, USA,2011), 595–604.

9. Образец заголовка
Adaptively choosing items to rate I
(contd.)

10. Образец заголовка
Adaptively choosing items to rate I
(contd.)
Results
• This technique performs better
than GreedyExtend, Entropy0,
HELF, and Var
• The decision tree technique
requires 6 items to achieve the
same RMSE score that takes other
items atleast 20 items to achieve,
thus proving that the adaptive
technique is far more efficient
• Test RMSE improves if training
dataset consists of users with
more ratings
• Authors also demonstrate
blending multiple trees by asking
several questions simultaneously,
and this improved the RMSE

11. Образец заголовка
Functional matrix factorization for
cold start recommendation
Main Idea
• Initiate several trials when a new user visits the site, and at each trial ask the user to provide their opinion on a
seed item
• In Golbandi et al, WSDM 2011, a decision tree is fit to the users’ ratings – this technique extends the method by
integrating decision tree construction into the matrix factorization framework, and the authors call this method
Functional Matrix Factorization
Technique
• Compared to traditional matrix factorization, no information about the user profile is available to predict
ratings
• The user’s answers on P questions are mapped to a user’s profile using a function, and the goal is to learn both
the item profiles and the user profiles
• Alternative optimization is used for calculating the item profiles given the mapping function, and then the
mapping function is obtained by fitting a decision tree, given the item profiles
• To prevent overfitting, hierarchical regularization is used, where the coefficient of a node is shrunk towards its
parent
Zhou, K., Yang, S.-H. and Zha, H. 2011. Functional Matrix Factorizations for Cold-start Recommendation. Proceedings of the 34th International ACM SIGIR Conference on
Research and Development in Information Retrieval (New York, NY, USA, 2011), 315–324.

12. Образец заголовка
Functional matrix factorization for cold
start recommendation (contd.)
Results
• This method is compared with Tree (Golbandi et al., WSDM
2011) and TreeU (learns decision tree using the Tree algorithm
and matrix factorization through two separate steps)
• Performance improves as number of interview questions
increases for all three methods
• fMF performed better than Tree and Tree U on all three
datasets – MovieLens, EachMovie, and Netflix
• The authors studied the impact of non-responses, and model
parameters
• Compared to the warm-start matrix factorization technique,
this method worked poorly since it was constrained to ask a
few questions – however the performance is comparable if the
depth of the tree is allowed to be large

13. Образец заголовка
Representative based rating
elicitation
Main Idea
• Select representatives – those users/items whose linear combinations of profiles
accurately approximate other users’/items’ profiles
Technique
• Propose representative based matrix factorization – ratings matrix has a rank-k
approximation: Y = CX, C consists of columns of Y, X is the loading matrix with free
parameters
• The dimensionality of the column space of Y is reduced using a rank-k SVD, and the
basis vectors (representatives) are chosen using the maximal volume concept
• The loading matrix X is then obtained using regularized least squares method
• Folding in method is used to efficiently compute ratings for a new user – by asking him
to rate k representative items
Liu, N.N., Meng, X., Liu, C. and Yang, Q. 2011. Wisdom of the Better Few: Cold Start Recommendation via Representative Based Rating Elicitation. Proceedings of the Fifth
ACM Conference on Recommender Systems (New York, NY, USA, 2011), 37–44.

14. Образец заголовка
Representative based rating
elicitation (contd.)
Results
• Compared this technique to the baselines: random
selection of representatives, most popular items as
representatives and k-medoids clustering
• RBMF method is able to achieve a good balance
between coverage and diversity of representatives
compared to other techniques
• RBMF method also performed better than the
baselines on metrics Pre@10, MAP, and AUC, on the
Netflix, MovieLens, and Last.fm datasets
• RBMF method also performed better than traditional
matrix factorization

15. Образец заголовкаAdaptively choosing items to rate II
Main Idea
• Even if the system adapts to the user’s response, there is a high
probability that user is asked to rate an item which they have not seen,
leading to a large portion of users falling in the “unknown” category in
decision trees
• Ask user to rate multiple items at each trial to increase the chance that
they have seen at least one of them – it is thus an extension of the single
decision tree method proposed by Zhou et al., SIGIR 2011
Technique
• Obtain low dimensional approximation of ratings matrix as a product of
k-dimensional user profiles and item profiles
• Induce a function from the decision tree that maps the user’s response
to a profile – this function maps a node to a user’s profile
• Because of multiple questions, people with different opinions enter a
node, so cannot average their ratings to predict
• Learn a logistic function for each node that splits users into three child
nodes based on their response
• Use L1 regularization to optimize each split
• After learning the mapping function and estimating the user profile,
update item profiles using regularized least square regression
Sun, M., Li, F., Lee, J., Zhou, K., Lebanon, G. and Zha, H. 2013. Learning Multiple-question Decision Trees for Cold-start Recommendation. Proceedings of the Sixth ACM
International Conference on Web Search and Data Mining (New York, NY, USA, 2013), 445–454

16. Образец заголовка
Adaptively choosing items to rate II
(contd.)
Results
• Compared this technique with single-question
decision tree method (w1base, Zhou et al, SIGIR
2011), linear combination of k-random trees (w4base
- Golbandi et al., WSDM 2011)
• For all techniques, RMSE reduces as more questions
are asked
• Current technique with 2,3 questions per trial
outperforms the single decision tree method, since
more information is obtained
• Current technique with 3,4 questions per trial also
performs better than the linear combination of k-
random trees with 4 questions for every node
• Current technique gives same performance as the
single question method, for the same expected time
spent by the user on the interview process

17. Образец заголовка
Interactive Collaborative Filtering
(ICF)
Paper: Zhao et al., CIKM 2013
Main Idea
• Addresses cold start problem
• Probabilistic matrix factorization
Technique Recommendation-feed back loop
• Uses an objective function to select an item to recommend next:
– Maximum expected reward from target user (+ve feedback)
• Two methods of items selection –
– Sampling
– Confidence Bound
Zhao,X., Zhang, W. and Wang, J. 2013. Interactive collaborative filtering. Proceedings of the 22nd ACM international conference on Conference on information & knowledge
management (New York, NY, USA, 2013), 1411–14.

18. Образец заголовкаICF (contd.)
• Sampling
– user & feature vectors : these are random variables
• updated using Thompson sampling
– user vector is more sensitive to new ratings
• Confidence Bound
– fixed feature vectors
– upper bound confidence (item with the highest upper bound is
chosen) & SVD (singular vector decomposition) are used to estimate
function
Results:
• Proposed algorithms outperformed the baselines including greedy
algorithm, active learning and interview approaches
• Also, more exploration is needed when longer term satisfaction is
targeted

19. Образец заголовкаChoice Based Preference Elicitation
Paper:Loepp et al., CHI 2014
Main Idea
• Interactive user control with automatic recommender techniques
• Extracts latent factors from a matrix of user ratings and generates dialogs
in which the user iteratively chooses between two sets of sample items
Technique
• Uses user-item matrix and derive latent item features from it.
• Latent factors are computed using Matrix Factorization techniques
• Movies are arranged in an n-dimensional vector space according to their
feature values
• Position of target user is incrementally updated to find items having
similar latent feature characteristics
Loepp, B., Hussein, T. and Ziegler, J. 2014. Choice-based Preference Elicitation for Collaborative Filtering Recommender Systems. Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems (New York, NY, USA,2014), 3085–3094.

20. Образец заголовка
Choice Based Preference Elicitation
(contd.)
• In each step:
– two sets of movies are
shown (dialog)
– user vector (features) is
updated
– movies with shortest
distance from user vector are
shown Two movie sets that differ in a single factor [7]
Results
• This approach gave significantly better results than the other methods in
15 (out of 24) parameter comparisons
• It gives good result even for the cold start problem
Loepp, B., Hussein, T. and Ziegler, J. 2014. Choice-based Preference Elicitation for Collaborative Filtering Recommender Systems. Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems (New York, NY, USA,2014), 3085–3094.

21. Образец заголовка
Group of Items for Preference
Elicitation
Paper:Chang et al., CSCW 2015
Main Idea
Preference elicitation using a group of items instead of individual items
which was used in previous paper
Technique
• High quality clusters are created using spectral clustering
• Individual ratings contain overlapping information
• Pairwise cosine similarities are considered for groups
• Movies and tags are used to represent a group
– three unique & relevant tags
• User assigns points to movie groups (across diff. clusters)

22. Образец заголовка
Group of Items for Preference
Elicitation (contd.)
Interface for movie groups (4 are visible on this screen, there are 6 preference points for allocation) [8]
Chang, S., Harper, F.M. and Terveen, L. 2015. Using Groups of Items for Preference Elicitation in Recommender Systems. Proceedings of the 18th ACM Conference on
Computer Supported Cooperative Work & Social Computing (New York,NY, USA,2015), 1258–1269

23. Образец заголовка
Group of Items for Preference
Elicitation (contd.)
• Pseudo rating profile of user is created
– discover users having highest rating
for movies in cluster
– calculate average rating
• Ratings are generated based on pseudo profile
Results
• Only top 10 recommendations were considered
• Baseline is rating 15 movies
• Time taken by users is less than half of
the median time taken by the baselining process
• Users are more satisfied from top N
recommendations.
Survey results about recommendation quality, comparing group-
based process with the base-
line rate 15 process.
Chang, S., Harper, F.M. and Terveen, L. 2015. Using Groups of Items for Preference Elicitation in Recommender Systems. Proceedings of the 18th ACM Conference on
Computer Supported Cooperative Work & Social Computing (New York,NY, USA,2015), 1258–1269

24. Образец заголовкаPairwise Preference Elicitation
Paper:Bledaite et al., HT 2015
Main Idea
• Uses a combination of ratings and pairwise preferences for
recommendation
• Similar technique was used in choice based elicitation in which user is
asked to compare a set of movies (requires lot of comparisons)
Technique
• Slider is used in the interface: dragged towards one of the two choices
• Pearson correlation : between the choices
• More decorrelation helps in understanding users’ preference

25. Образец заголовка
Pairwise Preference Elicitation
(contd.)
• Ranking technique used for
recommendation
• Matrix contains scores (user-user similarity)
– how much user prefers one item over
another
• Two stages –
– gather user preferences; give
recommendation & elicit additional
preferences
– again present recommendations (nDCG
for evaluation)
Blédaité, L. and Ricci, F. 2015. Pairwise Preferences Elicitation and Exploitation for Conversational Collaborative Filtering. Proceedings of the 26th ACM Conference on
Hypertext & Social Media (New York, NY, USA,2015), 231–236.

26. Образец заголовка
Eliciting Users’ Unknown
Preferences
Paper:Neidhardtet al., RecSys 2015
Main Idea
• Users select a set of pictures instead of answeringsome questions / or
rate products
• This technique is focused for tourism field
• Generally, users do not know their needs & are unable to express
themselves
• Pictures reflect touristic behavioral patterns
Technique
• Questionnaire – capture 17 tourist types, 5 personality traits (rated on 5
point scale)
• factor analysis is performed on these variables
• Users select pictures (out of 102 pics) & validate association b/w pics
and factors

27. Образец заголовка
Eliciting Users’ Unknown
Preferences (contd.)
• Experts assign pictures to point
of interests (POIs)
• Multiple regression model :
quantify relation b/w pics and
factors
• Give recommendations (POIs)
to users based on their
selection of pictures
Results
84% of users agreed
that experience with this
approach is exciting
Neidhardt, J., Schuster, R., Seyfang, L. and Werthner, H. 2014. Eliciting the Users’ Unknown Preferences. Proceedings of the 8th ACM Conference on Recommender Systems
(New York, NY, USA, 2014), 309–312

28. Образец заголовкаFuture Work
• Address item cold-start problem i.e. how to target new items to a set of existing
users so that long-term feedback collected by the item is maximized
• Influence of different user characteristics, e.g. familiarity with the product domain or
user specific decision strategies
• Improving group based elicitation technique though the use of automated clustering
algorithms
• Use of active learning strategies for the pairwise preference (PPR) model and elicit
mixed preference data, i.e., both ratings and pairwise preferences
• Extension of the picture approach using other emotional objects such as sound
samples is planned. Text mining methods on object descriptions and reviews can also
be used to position the touristic object

29. Образец заголовка
Thank you!
Questions?

30. Образец заголовкаReferences
1. Golbandi, N., Koren, Y. and Lempel, R. 2010. On Bootstrapping Recommender Systems. Proceedings of the 19th ACM
International Conference on Information and KnowledgeManagement(New York, NY, USA, 2010),1805–1808.
2. Golbandi, N., Koren, Y. and Lempel, R. 2011. Adaptive Bootstrapping of Recommender Systems Using Decision Trees.
Proceedings of the Fourth ACM International Conference on Web Search and Data Mining (New York, NY, USA, 2011), 595–
604.
3. Zhou, K., Yang, S.-H. and Zha, H. 2011. Functional Matrix Factorizations for Cold-start Recommendation. Proceedings of the
34th International ACM SIGIR Conference on Research and Development in Information Retrieval (New York, NY, USA, 2011),
315–324.
4. Liu, N.N., Meng, X., Liu, C. and Yang, Q. 2011. Wisdom of the Better Few: Cold Start Recommendation via Representative
Based Rating Elicitation. Proceedings of the Fifth ACM Conference on Recommender Systems (New York, NY, USA, 2011), 37–
44.
5. Sun, M., Li, F., Lee, J., Zhou, K., Lebanon, G. and Zha, H. 2013. Learning Multiple-question Decision Trees for Cold-start
Recommendation. Proceedings of the Sixth ACM International Conference on Web Search and Data Mining (New York, NY,
USA, 2013),445–454.
6. Zhao, X., Zhang, W. and Wang, J. 2013. Interactive collaborative filtering. Proceedings of the 22nd ACM international
conference on Conference on information & knowledgemanagement(New York, NY, USA, 2013),1411–14.
7. Loepp, B., Hussein, T. and Ziegler, J. 2014. Choice-based Preference Elicitation for Collaborative Filtering Recommender
Systems. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2014), 3085–
3094.
8. Chang, S., Harper, F.M. and Terveen, L. 2015. Using Groups of Items for Preference Elicitation in Recommender Systems.
Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (New York, NY, USA,
2015),1258–1269
9. Blédaité, L. and Ricci, F. 2015. Pairwise Preferences Elicitation and Exploitation for Conversational Collaborative Filtering.
Proceedings of the 26th ACMConference on Hypertext & Social Media (New York, NY, USA, 2015),231–236.
10. Neidhardt, J., Schuster, R., Seyfang, L. and Werthner, H. 2014. Eliciting the Users’ Unknown Preferences. Proceedings of the
8th ACMConference on Recommender Systems (New York, NY, USA, 2014),309–312.