The document discusses explainability in recommender systems and decision biases. It summarizes research on how collaborative explanations from other users can influence decision making. Three studies are described that use conjoint experiments to understand how different types of explanations and user styles (maximizers vs satisficers) affect preferences. An online experiment is also discussed that collected interaction data to analyze search heuristics when rating recommendations. The document promotes transparency and scrutability in recommender systems to improve user satisfaction and effectiveness.
5. Decision Biases
H. Simon. 1959. Theories of decision making in
economics and behavioural science. American
Economic Review 49, 3 (1959), 253–283
6. Ubiquitous in online marketplaces
and e-commerce agencies
… and they have a strong
bias/influence on online decision
making
… as well as to predict business
performance
Ratings
Are the main ingredient for
algorithmic decision support
mechanism like RS
They can be used for generating
explanations over a recommended
item
8. User style - xRecSys
Similar users Users’ Feedback
Abdollahi, Behnoush, and Olfa Nasraoui. "Using explainability for
constrained matrix factorization." Proceedings of the Eleventh ACM
Conference on Recommender Systems. 2017.
10. Items’ Explainability - 1
Determine the similarity between two
users:
users
item
Catalogue
of u
rating of
user v on i
mean rating
of user v
NN(u) is the set of N users with the highest
similarity.
11. Items’ Explainability - 2
Explainability of an item i for user u:
rating threshold
neighborhood size
neighborhood
17. We want to determine how
collaborative explanations are
guiding users’ choices in the online
scenario
17
We want to determine how
collaborative explanations are
guiding users’ choices in the online
scenario
18. Decision making strategies
Interpersonal differences
18
[1] Herbert A Simon. A behavioral model of rational choice. The quarterly journal of economics, 1955
[2] Schwartz et al., 2002, Maximizing versus satisficing: Happiness is a matter of choice. Journal of Personality and Social Psychology 1983
Herbert Simon
Three subdimensions[2]:Decision Difficulty; Alternative Search; High Standards
Maximizers [1]
o Exhaustively search the
best
o Compare to others
o Spend more time and
energy
o Are less happy with their
decisions
o Feel more regret
Satisficers [1]
○ Search for an option that
is “good enough”
○ Not obsessed with other
options
○ Are happier with their
decisions
19. We consider the collaborative
explanations to be multi-
attribute objects:
Explanation
Number of ratings
Mean of the ratings
Variance
Skewness
Bimodality
We consider the collaborative
explanations to be multi-
attribute objects:
Explanation
Number of ratings
Mean of the ratings
Variance
Skewness
Bimodality
19
20. Decision Making on Multi-Attribute Items
Non-Compensatory Strategies [1]:
Compare items based on one attribute
Perform intra-dimensional comparisons
Perform less comparisons
Compensatory Strategies [1]:
All attributes meet a minimum requirement
Multiple inter-dimensional comparisons
Spend more time on items
Eye movement is an indicator of the screening of the choices [2].
20
[1] John W Payne. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human
Performance, 1976
[2] Jacob L. Orquin and Simone Mueller Loose. Attention and choice: A review on eye movements in decision making. Acta Psychologica 2013
21. Conjoint experiment to quantify users’
preferences
3 Choice/Ranking based Conjoint Experiments:
Used in product design/development
Items can be seen as a bundle of attributes
Goal to identify the utility contribution of each attribute
of the collaborative explanation separately
Data driven levels
Attribute levels were in the choice sets were
balanced, orthogonal and with minimal overlap
Order was randomized
21
22. Study 1/3 – Choice Based Conjoint
experiment
Compared user/item/social-
style: mean vs number of
ratings
3 alternatives
6 comparisons
Cohort of 77 respondents
22
23. Study 1/3 - Parameter estimates (on the
user-style)
23
L. Coba, M. Zanker, L. Rook, P. Symeonidis: Exploring Users' Perception of Collaborative Explanation Styles. CBI 2018.
24. Study 2/3 – Choice Based Conjoint
experiment
Design: explanations, mean,
nr. ratings, variance, skewness
2 alternatives
16 comparisons
Cohort of 199 respondents
24
26. Study 2/3 - Parameter estimates
26
Note: Max/Sat determined by median split on the Decision Difficulty subdimension
L. Coba, L. Rook, M. Zanker, P. Symeonidis: Decision making strategies differ in the presence of collaborative explanations: two conjoint studies. IUI 2019.
27. Study 3/3 – Rank Based Conjoint
experiment
Rank 6 items: mean, ratings,
bimodality
3 choice sets
Eye-tracked
Cohort of 44 respondents
27
43. By-product
L. Coba, P. Symeonidis, M. Zanker: Visual Analysis of Recommendation Performance. RecSys 2017.
44. New multi-objective optimization approach
trade-off between precision, novelty and explainability
Metrics to measure the explainability of the recommendations
Extensive evaluation
45. 3 conjoint experiments on the perception of
collaborative explanations
Involving 300+ respondent
Rating summary statistics influence/bias users’ choice
Maximizers and Satisficers perceptions
45
47. Questions?
Journal Papers:
L. Coba, L. Rook, M. Zanker. Decision biases in the online recommendation scenario. JITT 2019.
P. Symeonidis, L. Coba, M. Zanker . Improving Time-aware Recommendations in Open Source Packages. IJAIT 2019.
L. Coba, P. Symeonidis, M. Zanker. Personalized Novel and Explainable Matrix Factorization. DKE 2019.
L. Coba, P. Symeonidis, M. Zanker. Counteracting the Filter Bubble in Recommender Systems: Novelty-aware Matrix Factorization. Intelligenza Artificiale. 2019.
L. Coba, L. Rook, M. Zanker. The short-term causes of maximization in session-based recommendation settings. Under Review at UMAUI
Conference papers:
L. Coba, M. Zanker: Replication and Reproduction in Recommender Systems Research - Evidence from a Case-Study with the rrecsys Library. IEA/AIE 2017.
L. Coba, M. Zanker, L. Rook, P. Symeonidis: Exploring Users' Perception of Collaborative Explanation Styles. CBI 2018.
L. Coba, P. Symeonidis, M. Zanker: Replicating and Improving Top-N Recommendations in Open Source Packages. WIMS 2018.
L. Coba, P. Symeonidis, M. Zanker: Novelty-Aware Matrix Factorization Based on Items' Popularity. AIIA 2018.
L. Coba, M. Zanker, L. Rook: Decision Making Based on Bimodal Rating Summary Statistics - An Eye-Tracking Study of Hotels. ENTER 2019.
L. Coba, L. Rook, M. Zanker, P. Symeonidis: Decision making strategies differ in the presence of collaborative explanations: two conjoint studies. IUI 2019.
Demo & Posters:
Hinweis der Redaktion
Those that were supposed to be satisficers
Based on this number one could assume that maximisers go for simple decision heuristics, which would contradict theory
Trade-off while having the choice between
Gender comparison
This is according to theory
In Slide Show mode, select the arrows to visit links.
