I gave this presentation in the Recommendation and Collaboration Intelligence User Interfaces workshop at the 2008 International Conference on Intelligent User Interfaces.
Presents a data visualisation technique that is used to help users to enter their explicit preferences for their profile. In this case, users were able to enter more preferences for actors when the actors were visualised according to a similarity metric rather than randomly organised, in the same time period.
4. context
general context
recommender systems attempt to offer items to users that will be appreciated
the quality of these recommendations is largely constrained by the data
• a user’s preferences
• domain-specific ‘general knowledge'
• the items that can be recommended
• previous users’ opinions of items
the acquisition of such data is of central importance
specific context
a hybrid content-based and collaborative filtering recommendation system
the system will be implemented in the domain of cinema
the user can explicit enter their preferences
kris jack – p4
5. the problem
entering your preferences explicitly
can be boring
can be difficult
ifa system is too much trouble to use, then it
simply won't be used
how can we improve the explicit preference entry
process?
maximise the number of explicit preferences that could be
elicited within a given period of time
kris jack – p5
7. definitions
preferences, many types and definitions exist
concentrating on the elicitation of monadic preferences
represent a user’s like, dislike, or indifference towards an item or item
attribute (e.g. “i like tim burton”, “i dislike horror movies” and “i love kill
bill”)
preference acquisition strategies
explicit – user explicitly enters their preferences
implicit – learning strategies that are non-invasive (e.g. user-profiling
and collaborative filtering)
explicit preference entry (epe) interface
an interface that asks users to explicitly give their preferences
towards items and item attributes (e.g. “i love comedies”)
kris jack – p7
8. some existing epe interfaces
recommenders with epe interfaces
minekey (www.minekey.com)
stumbleupon (www.stumbleupon.com)
movielens
drawbacks
boring to use minekey
difficult to be inspired
(when guidance is lacking)
difficult to describe yourself
in their terms at times
kris jack – p8 stumbleupon
9. data visualisation
perhaps we can improve epe interfaces by visualising data
data visualisation techniques have had some success in
recommenders
music plasma (www.musicplasma.com)
amaznode (amaznode.fladdict.net/)
music plasma amaznode
kris jack – p9
11. visualising data similarity in an epe
interface
the epe interface
should encourage users to enter their preferences
may guide users based upon the preferences required
should be enjoyable to use and not boring
the epe interface creation process
must be robust and reliable
desirable if it is automated from start to finish
creation process summary
input a list of items or item descriptors (e.g. actors)
using a similarity metric, find the similarities between data elements
visualise the similarities between the data elements
kris jack – p11
12. instantiating the system's semantic
knowledge
the system's semantic knowledge describes the
similarity between descriptors in the database
the notion of similarity is necessarily subjective
how similar do you find these two actors?
(robert de niro and al pacino)
kris jack – p12
13. strategies for defining semantic
similarities
considered
instantiation by hand
differencing mechanism
co-occurrence measures
clustering algorithms
collaborative filtering techniques
opted for one based on co-occurrences of actor names
found using the google search estimates
max{log f (i1),log f (i 2)} − log f ( i1, i 2)
d (i1, i 2 ) =
log M − min{log f (i1),log f (i 2)}
where m is the total number of pages considered, f(i1) and f(i2) are the number of hits
for i1 and i2 respectively, and f(i1,i2) is the number of hits for the co-occurrence of i1 and
i2.
in essence, the more often two items appear together on the same web pages,
the more similar they are
the measure of semantic relatedness server provides free access
kris jack – p13
14. normalised google distance
actors jackie chan bruce lee jane fonda
jackie 2,420,000 965,000 145,000
chan (0.0) (0.09) (0.26)
bruce 965,000 2,630,000 46,700
lee (0.09) (0.0) (0.37)
jane 145,000 46,700 1,930,000
fonda (0.26) (0.37) (0.0)
the number of google hits for actor pairs and their
normalized google distances given in brackets (the
smaller the distance, the more similar the actors)
kris jack – p14
15. data visualisation
use of the radial tree layout to visualise data similarities
manageable linear complexity in laying out the tree
• efficient even with several hundreds of nodes
the more similar two items are, the closer their proximity in the graph
a radial layout encourages users to explore the tree in a less hierarchical
fashion as it is unclear where the root node is
tree can be focussed upon any node in the tree (implementing a smooth
transition animation)
• users have previously found this form of visualisation attractive
implementation available in prefuse (www.prefuse.com) library
strategy
each item (actor) is represented by a node in the tree
connect every node with their two closest nodes (using item similarity)
kris jack – p15
18. epe interface
the visualisation is mounted in an epe interface that allows
users to
zoom in by right clicking on a graph area (in zoomed out mode) and zoom
out by right clicking on a graph area that does not contain an actor (in
zoomed in mode);
pan within the graph by left clicking on a graph area and dragging in the
direction to pan;
search for an actor by typing the actor’s name in the search box. when the
user starts to type a name, the mode changes to zoomed out mode and all
actor nodes who’s names match the string are enlarged;
change the preference towards an actor (like, dislike, neutral, no preference)
by righting clicking on the actor’s node (in zoomed in mode);
re-organise the graph to centre upon one actor by double left clicking on
another actor node.
kris jack – p18
22. evaluating the epe interface
evaulated:
choice of similarity metric in the context of actors
type of preferences elicited
epe interface's ease of use
appreciation of the epe interface
materials
epe interface mounted with 3 different graphs:
• organised graph (nodes positions according to actor similarity)
• unorganised graph (organised graph with nodes randomised)
• demonstration graph (organised graph with nodes randomised)
each graph contained the same 500 nodes (most frequent actors from
an in-house french database)
instruction sheet
kris jack – p22
23. participants and procedure
28 participants (14 male, 14 female)
procedure
practice the functions of the epe interface using the demonstration
graph (took 10 minutes on average)
task
• enter as many actor-based preferences as possible in 5 minutes
• once with the organised graph and once with the unorganised graph
(following a within-subjects design with 2 groups of 14 participants)
• note that the graphs were not named here, the tasks were referred to as
task 1 and task 2
participants completed a questionnaire on terminating the tasks
kris jack – p23
24. hypothesis and measurements
hypothesis
participants will find it easier to declare their preferences for actors in
the organised graph task than in the unorganised graph task, within
the same time period
measuring the ease of declaring preferences
quantity of preferences entered
subjective questioning in the questionnaire
kris jack – p24
26. preference elicitation
more preferences were entered using the organised graph:
significant increase in 'like' preferences (34%)
decrease in 'dislike' and 'neutral' preferences
P ref erenc e Elic itation
60
50
40
Organised
30
Unorganised
20
10
0
All Like Dislike Neutral
kris jack – p26
27. perceived ease of entering
preferences
participants reported that it was:
easy to enter preferences using the organised graph
neither easy nor difficult to enter preferences using the unorganised
graph
ease of preference entry mean
statements
organised unorganised
"i found it easy to enter
my preferences." 3.96 3.36
"entering my preferences
demanded too much
effort" (mean reversed) 3.64 3.11
cronbach’s alpha = 0.80
kris jack – p27
28. perceived differences between graphs
22 participants (79%) reported differences between the two
graphs
commented that the organised graph
had been hand designed so that it was easier to navigate;
rearranged itself based on the actors that the participant said that they liked;
arranged the participant’s favourite actors together;
had more connections between nodes;
arranged actors together who:
• co-starred in the same films;
• shared the same nationality;
• shared the same degree of celebrity;
• were similar to one another.
commented that the unorganised graph
was less organised
kris jack – p28
29. appreciation of the epe interface
participants enjoyed using the interface and would be happy
using it again (mean = 4.11/5.00, sd = 0.99)
suggested improvements
preference changing. some participants did not like having to click
twice to register a dislike and three times to enter a neutral
preference.
zooming in. some participants would have preferred a precise
indication of the region into which they could zoom into before
zooming.
zooming out. some participants felt lost when zoomed in on the
actors as they were not sure of where they were with respect to the
entire map
kris jack – p29
31. discussion
the participants mark more 'like' preferences with the
organised graph (34%)
find more actors who they like and less who they dislike or are neutral
towards
why?
• in searching, participants tended to begin by using the search feature,
then zoom in on their desired actor
• when at the zoomed in level, they would pan around to find other
actors.
• actors who were in close proximity tended to be similar
• similar actors tend to be liked too
applications that can exploit likes better than dislikes may want to
introduce semantic similarity in an epe interface
kris jack – p31
32. discussion
how well did the notion of similarity come across?
with only 5 minutes of exposure to each graph, the majority of
participants found that one was organised and that the other was
lesser so or not at all
the word similar was repeatedly used by participants
results serve to validate the use of the google distance metric in this
area
the similarity metric thus goes some way to replacing what is
traditionally in the domain of human-design decisions
the epe interface was very much appreciated
participants liked it and wanted to use it again
they commented that it was more like playing a game and not like
entering their preferences
kris jack – p32
33. discussion
participants report that the organised graph is easier to use
an organised graph is easier to navigate than an unorganised
graph
they find more actors who they like
addressing interface issues
replace the right click to change a preference with three icons next to
the node. a single click on the item will designate the corresponding
preference
offer a 'mini-map', with the absolute position of the main map
indicated, that is always zoomed out
kris jack – p33
34. discussion
when should the epe interface appear in the recommendation
process?
from the start, users should be able to use it
• initial entry of preferences
all throughout the recommendation process also
• could be used to visualise learned or predicted preferences too, allowing the user to
correct any mistakes at a visual level
what other benefits does a similarity-based epe interface bring?
users become aware of the notion of similarity as used by the system
the logic of the system, in this case the positioning of actor nodes, becomes
learned
imagine a system that uses this form of similarity to produce non-exact results
for searches (e.g. cannot fine any jackie chan films, would you like bruce lee
films instead?)
understanding the logic of the system is very important in developing trust in
the system
kris jack – p34
36. conclusion
a new epe interface is introduced that can takes data and
organises it based on a robust similarity metric
data similarities are visualised into a pleasing tree-based graph
users can navigate through the graph and explicitly enter their
preferences for different items
interface favours elicitation of 'like' preferences
users enter 34% more 'like' preferences when the graph is organised with the
similarity metric compared to when it is left unorganised
users report a reduction in cognitive effort when using the
organised graph
the epe creation process is a robust and flexible solution to
eliciting explicit user preferences in a recommendation system
kris jack – p36
37. the end
many thanks for your attention
kris jack – p37