UiPath Community: AI for UiPath Automation Developers
Social Aspects of Interactive Recommender Systems
1. Social aspects of interactive RecSys:
Bridging the gap between predictive algorithms
and interactive user interfaces
Denis Parra, Assistant Professor
CS Department
School of Engineering
Pontificia Universidad Católica deChile
SoAPS Workshop at ECIR 2018, March 26th 2018
Funded by :
2. Personal Introduction
• 2008-2013: PhD at U. of Pittsburgh
• 2013 – now: CS Department, PUC (Santiago, Chile)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 2
8,280 Km
3. Outline
• Quick intro to
– RecommenderSystems
– Visualization in RecommenderSystems
• Survey of our work on Visualization and
Interaction on RecSys
– With highlights to social aspects of the results
• Summary & Discussion
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 3
4. In Collaboration with
• Peter Brusilovsky (University of Pittsburgh, USA)
• Katrien Verbert (KU Leuven, Belgium)
• Christoph Trattner (University of Bergen, Norway)
• Chaoli Wang (Notre Dame University, USA)
• Ivania Donoso (alumni, PUC Chile)
• María Sepúlveda (MSc student, PUC Chile)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 4
6. Recommender Systems (RecSys)
Systems that help (groups of) people to find relevant items in
a crowdeditem or information space(MacNee et al. 2006)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 6
7. Why do we care about RecSys?
• Nowadays, several domains& applications require
people to make decisions among a large set of items.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 7
8. A lil’ bit of History
• First recommender systems were built at the
beginning of 90’s (Tapestry, GroupLens, Ringo)
• Online contests, such as the Netflix prize, grew the
attention on recommender systems beyond
Computer Science
(2006-2009)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 8
9. The Recommendation Problem
• A popular way of presenting the recommendation
problem was rating prediction (Netflix prize)
• How good is my prediction?
Item 1 Item 2 … Item m
User 1 1 5 4
User 2 5 1 ?
…
User n 2 5 ?
Predict!
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 9
10. Traditional Recommendation Methods
• Without covering all possible methods, the two
most typical classifications on recommender
algorithms are
Classification 1 Classification 2
- Collaborative Filtering
- Content-based Filtering
- Hybrid
- Memory-based
- Model-based
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 10
11. Collaborative Filtering (User-based KNN)
• Find like-minded people to recommend
5
4
4
2
1
5
4
4
Active
user
User_1
User_2
∑
∑
⊂
⊂
−⋅
+=
)(
)(
),(
)(),(
),(
uneighborsn
uneighborsn nni
u
nuuserSim
rrnuuserSim
riupred2
3
4
2
Item 1
Item 2
Item 3
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 11
12. Content-Based Filtering
• Can be traced back to techniques from IR, where
the User Profile represents a query.
user_profile = {w_1, w_2, …., w_3} using TF-IDF, weighting
Doc_1 = {w_1, w_2, …., w_3}
Doc_2 = {w_1, w_2, …., w_3}
Doc_3 = {w_1, w_2, …., w_3}
Doc_n = {w_1, w_2, …., w_3}
5
4
5
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 12
13. Hybridization
• Combine previous methods to overcome their
weaknesses (Burke, 2002)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 13
14. Model-based: Matrix Factorization
Latent vector of the item
Latent vector of the user
SVD ~
Singular Value
Decomposition
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 14
15. Other paradigms and techniques
• Recommendation as a graph problem:
– PersonalizedPageRank (Kamvar et al, 2010), (Santos et
al 2016), etc.
• Recommendation as a ranking problem:
– Karatzoglou et al. (2013), Shi et al. (2014), Macedo et al.
(2015), etc.
• Deep learning methods:
– MetaProd2Vecby Vasile et al. (2016), YouTube
Recommendationsby Covington et al. (2016), etc.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 15
16. Are accuracy metrics the goal of RecSys?
• Some works started pointing out that small
improvements in RMSE did not have a
proportional improvement in user satisfaction
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 16
18. PeerChooser (2008) Controllability in CF
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 18
O’Donovan et al. “PeerChooser: Visual Interactive Recommendation” (2008)
19. SmallWorlds: Expanded Explainability
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 19
Gretarsson et al. “SmallWorlds:Visualizing social recommendations” (2010)
20. TasteWeights: Hybrid Control and Inspect
Bostandjev et al. “TasteWeights:A Visual Interactive Hybrid Recommender System” (2012)
Controllability:
Sliders that let users
control the
importance of
preferences and
contexts
Inspectability: lines
that connect
recommended items
with contexts and user
preferences
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 20
21. IUI 2017
• Loepp et al. (2017)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 21
22. More Details? Check our survey
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 22
He, C., Parra, D., & Verbert, K. (2016). Interactive recommender
systems: a survey of the state of the art and future research
challenges andopportunities.Expert Systems with Applications, 56, 9-27.
23. My Take on RecSys Research (2009 ~)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 23
26. TalkExplorer – IUI 2013
• Adaptation of Aduna Visualization to CN
• Main researchquestion: Does fusion (intersection) of
contextsof relevance improve user experience with RecSys?
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 26
27. Research Platform
• The studies were conducted using Conference
Navigator, a Conference Support System
• Our goal was recommending conference talks
Program Proceedings Author List Recommendations
http://halley.exp.sis.pitt.edu/cn3/
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 27
31. Evaluation: Intersections & Effectiveness
• What do we call an “Intersection”?
• We used #explorations on intersections and their
effectiveness, defined as:
Effectiveness =
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 31
32. Our Assumptions
• Items which are relevant in more that
one aspect could be more valuable to the
users
• Displaying multiple aspects of relevance
visually is important for the users in the
process of item’s exploration
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 32
33. TalkExplorer Studies I & II
• Study I
– ControlledExperiment:Userswere asked todiscover
relevant talksby exploringthe three types of entities:tags,
recommenderagents and users.
– Conductedat HypertextandUMAP 2012(21 users)
– Subjectsfamiliarwith Visualizationsand Recsys
• Study II
– FieldStudy: Users were left free to explore the interface.
– Conductedat LAK 2012 and ECTEL 2013 (18 users)
– Subjectsfamiliarwith visualizations, but not much with
RecSys
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 33
34. Results of Studies I & II
• Effectiveness increases
with intersections of
more entities
• Effectiveness wasn’t
affected in the field
study (study 2)
• … but exploration
distribution was
affected
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 34
35. More detail on entities
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 35
36. More detail on entities
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 36
• Despite not being the most effective, the
user (social) entity attracted by far more
explorations from users.
37. Social ~ Trust in RecSys
• O'Donovan, John, and Barry Smyth. "Trust in
recommender systems," IUI, 2005.
• Golbeck, Jennifer, and James Hendler. "Filmtrust:
Movie recommendations using trust in web-based
social networks," ComNet, 2006.
• Jamali, Mohsen, and Martin Ester. "A matrix
factorization technique with trust propagation for
recommendation in social networks." RecSys, 2010.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 37
38. SetFusion – IUI 2014
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 38
39. SetFusion I
Traditional
Ranked List
Papers sorted by
Relevance.
It combines 3
recommendation
approaches.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 39
40. SetFusion - II
Sliders
Allow the user to control the importance of
each data source or recommendation method
Interactive Venn Diagram
Allows the user to inspect and to filter papers
recommended. Actions available:
- Filter item list by clicking on an area
- Highlight a paper by mouse-over on a circle
- Scroll to paper by clicking on a circle
- Indicate bookmarked papers
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 40
41. Study : iConference
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 41
A
B
C
• A and C: Social
• B: Content-based
42. Rating per method – Effect of Visuals
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 42
43. Rating per method – Effect of Visuals
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 43
No significant difference
44. Rating per method – Effect of Visuals
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 44
Hybrid algorithm + visualization
yield the only significant
difference
45. Summary & Conclusions
• The combination of several sources of relevance
has an impact on recommendation, being the social
aspect among the most relevant.
• The visual paradigm combined with social aspects
used can have a significant effect on user behavior.
• We need to keep working on visual
representations that promote exploration without
decreasing recommendation accuracy.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 45
47. Moodplay
• MoodPlay
– With Ivana Andjelkovic& John O’Donovan (UCSB)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 47
Andjelkovic,I., Parra, D., & O'Donovan,J. (2016). Moodplay:Interactive
Mood-based Music Discovery and Recommendation.In Proceedings of
the 2016 Conference on User Modeling Adaptation and Personalization (pp.
275-279).ACM.
50. Emotion Models
• Modelo de emociones de Russel
(1980)
• GEMS (2008)
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 50
51. Moods and Music: the GEMS model
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 51
52. Components
• Colors: GEMS emotions
• Dots: artists
• Zooming and Panning
• Playing &
Recommendations
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 52
53. Components
• Colors: GEMS emotions
• Dots: artists
• Zooming and Panning
• Playing &
Recommendations
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 53
54. Components
• Colors: GEMS emotions
• Dots: artists
• Zooming and Panning
• Playing &
Recommendations
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 54
55. Components
• Colors: GEMS emotions
• Dots: artists
• Zooming and Panning
• Playing &
Recommendations
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 55
56. Emotion-Aware Recommendation
• Using several Web APIs, we
collected users’ perception of mood
associated to artists.
• Then, using artists as input, we
calculate recommendations of
similarly perceived artists
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 56
57. MoodPlay
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 57
• Our results indicate that users’ reported mood and
artist mood have an effect on people satisfaction
with the system.
• Try it at
http://moodplay.pythonanywhere.com
59. March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 59
Interactive Relevance Feedback
Interface for evidence–based
Medicine
Ivania Donoso-Guzmán
Denis Parra
Honorable mention for best paper award
ACM IUI 2018
60. Belleret al (2013).
… takes from 6 to 12 meses
Process for answering a clinical
question
60
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018)
63. Future Work
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 63
- CollaborativeModel:
- Whichis the best way to providea collaborative
human-in-the-loopinterfacefor Evidence-Based
Medicine?
- … Ideas from research on collaborative web
search, for instance:
Yue et al. “Influences on query reformulation in
collaborative web search”. IEEE Computer Magazine,
2014.
64. IEEE VIS 2017 – Panel ML & Vis ?
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 64
65. Deep Learning and Social on RecSys
• Lei, Chenyi, et al. "Comparative deep learning of
hybrid representations for image
recommendations." CVPR. 2016.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 65
66. Deep Learning and Social on RecSys
• Lei, Chenyi, et al. "Comparative deep learning of
hybrid representations for image
recommendations." CVPR. 2016.
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 66
- How to make better
sense of the social
embeddings for
explaining
recommendations?
67. IEEE VIS 2017 – Panel ML & Vis ?
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 67
68. Conclusion
March 26th, 2018 D.Parra~ Invited talk at SoAPSWS(ECIR 2018) 68
• The social aspects are indeed relevant in
recommendation systems, and their effect on
prediction accuracyhas been already studied.
• The studies presented in this talk show that combining
interactive visualizations with social and other
relevance signal can have an important effect on users’
perception of recommendations.
• I invite researcher to further study the connections
between social aspects, visualization and their effect
on recommender systems.