Integrating Context Similarity with Sparse Linear Recommendation Model

1. Integrating Context Similarity
with Sparse Linear
Recommendation Model
Yong Zheng, Bamshad Mobasher, Robin Burke
Center for Web Intelligence, DePaul University, Chicago, USA
The 23rd Conference on User Modeling, Adaptation and Personalization,
Dublin, Ireland, June 29 – July 3, 2015 (UMAP 2015)

2. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

3. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

4. RecSys and Context-aware RecSys
• Recommender Systems (RS)
The data is usually a 2D rating matrix: User × Item ―> Ratings
Task-1: Rating Predictions for <user, item> pair
Task-2: Top-N Recommendations for a specific user, i.e., provide a
list of ranked items to the user

5. • Context-aware RecSys (CARS)
Context dimension: the variable, e.g., time, location, companion
Context condition: values in dimension, e.g., weekend and weekday
Context situation: a set of conditions, e.g., <weekend, home, sister>
The data is represented in a multi-dimensional rating space.
Task-1: Rating Predictions for <user, item, contexts>
Task-2: Top-N Recommendations for a user in specific contexts,
RecSys and Context-aware RecSys

6. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

7. Contextual Recommendations
• How to build CARS algorithms?

8. Contextual Recommendations
• Contextual Modeling
There are usually two ways for contextual modeling:
1). Independent Contextual Modeling
Tensor Factorization, ACM RecSys 2010
2). Dependent Contextual Modeling
2.1). Deviation-Based Modeling
Context-aware Matrix Factorization, ACM RecSys 2011
Contextual Sparse Linear Method, ACM RecSys 2014
2.2). Similarity-Based Modeling
The proposal in this paper, UMAP 2015

9. Contextual Modeling
• Independent Contextual Modeling
Tensor Factorization (TF), ACM RecSys 2010
Assumption: context is independent with user/item dimension.
But usually, there are dependencies involved.

10. Contextual Modeling
• Dependent Contextual Modeling
Context-aware Matrix Factorization (CAMF), ACM RecSys 2011
Contextual Sparse Linear Method (CSLIM), ACM RecSys 2014
Global average
rating
User bias Item bias
Matrix Factorization:
CAMF:
Item bias in contexts

11. Contextual Modeling
• Dependent Contextual Modeling
Context-aware Matrix Factorization (CAMF), ACM RecSys 2011
Contextual Sparse Linear Method (CSLIM), ACM RecSys 2014
Those approaches are named as deviation-based modeling,
since they tried to incorporate contextual rating deviations into
recommendation algorithms by modeling dependencies or
correlations between contexts and user/item dimensions.
Any other alternatives? How about the dependencies or correlations
among contexts? We name this approach of context modeling as
similarity-based modeling.

12. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

13. SLIM and Contextual SLIM
• Why SLIM
SLIM = Sparse Linear Method, which is an effective top-N
recommendation algorithm in traditional RS.
In this paper, we choose SLIM as the base algorithm, and introduce how
to build contextual SLIM algorithms by incorporating context similarity.
SLIM was demonstrated as the most effective top-N recommendation
algorithms in previous work. Here, we focus on top-N contextual
recommendation. Other algorithms, such as matrix factorization, can
also be chosen as base algorithm.

14. • SLIM in Traditional RecSys
Matrix R = rating matrix; W = coefficient matrix
SLIM aggregates users’ ratings by coefficients between items.
It learns item coefficients by minimizing the ranking score.
Sparse Linear Method (SLIM)

15. • CSLIM in Context-aware RecSys
P is multidimensional contextual rating space; W is item coefficient matrix;
Matrix D estimates the rating deviation from one context to another.
1). By Deviation-Based Contextual Modeling, RecSys 2014, CIKM 2014
Contextual SLIM (CSLIM)

16. • CSLIM in Context-aware RecSys
Previous dependent contextual modeling approaches mainly focused on
modeling the correlations between context and user/item dimensions,
but ignore the correlation between contexts themselves;
Context similarity = similarity between two contexts, measuring inner
similarities or correlations between two contextual situations;
We propose and believe that modeling context similarities is another important
way to develop dependent contextual modeling approaches, rather than
modeling contextual rating deviations!!!
2). By Similarity-Based Contextual Modeling, UMAP 2015
Contextual SLIM (CSLIM)

17. • CSLIM in Context-aware RecSys
Original SLIM:
Deviation-Based CSLIM:
Similarity-Based CSLIM:
2). By Similarity-Based Contextual Modeling, UMAP 2015
Deviation term
Similarity term
Contextual SLIM (CSLIM)

18. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

19. Modeling Context Similarity
• Context Similarity
Context similarity can be obtained in the following ways:
1). Semantics
But, it is hard to for <holiday, cinema> & <weekday, home>;
Semantics is more useful for hierarchical or tree-based categorical data;
2). Calculation based on co-ratings in different contexts
However, contextual rating data is usually sparse, which results in unreliable
calculations for context similarity.
3). Learning methods
Instead, we can learn the similarity directly by minimizing ranking errors.
error = ranking score – predicted ranking score
Minimizing this ranking error by gradient descent in CSLIM

20. Modeling Context Similarity
• Context Similarity
Learning methods
Instead, we can learn the similarity directly by minimizing ranking errors.
error = ranking score – predicted ranking score
Minimizing this ranking error by gradient descent in CSLIM
However, the performance may directly depend on how we represent
and model context similarity. In this paper, we discuss 4 modeling:
1). Independent Context Similarity (ICS)
2). Latent Context Similarity (LCS)
3). Weighted Jaccard Context Similarity (WJCS)
4). Multidimensional Context Similarity (MCS)

21. Modeling Context Similarity
• 1).Independent Context Similarity (ICS)
Similarity-Based CSLIM:
Independent Context Similarity (ICS) can be represented as follows:
For example: Ck = {Time = Weekend, Location = Home}; Cm = {Time = Weekday, Location = Office}
is: Similarity(Weekend, Weekday) × Similarity (Home, Office)
Assumption: contextual variables are assumed as independent.
What to be learnt: each individual similarity between two conditions;

22. Modeling Context Similarity
• 2).Latent Context Similarity (LCS)
Similarity-Based CSLIM:
Latent Context Similarity (LCS) is an improvement over ICS.
For example: Ck = {Time = Weekend, Location = Home}; Cm = {Time = Weekday, Location = Office}
is: Similarity(Weekend, Weekday) × Similarity (Home, Office)
Each condition is represented by a vector;
What to be learnt: the weights in vectors for each contextual condition.
Training: <weekend, weekday> <weekday, holiday>
Testing: <weekend, holiday>
Context Sparsity

23. Modeling Context Similarity
• 3).Weighted Jaccard Context Similarity (WJCS)
Weighted Jaccard Context Similarity refers to similarity between two strs.
Assume those three context dimensions are equally weighted, w1 = w2 = w3 = 1.
= # of matched dimensions / # of all dimensions = 2/3
What to be learnt: the weight for each context dimension.
Similarity is measured by Weighted Jaccard similarity
User Movie Time Location Companion Rating
U1 Titanic Weekend Home Girlfriend 4
U2 Titanic Weekday Home Girlfriend 5
U3 Titanic Weekday Cinema Sister 4
U1 Titanic Weekday Home Sister ?

24. Modeling Context Similarity
• 4).Multidimensional Context Similarity (MCS)
Similarity-Based CSLIM:
Multidimensional context similarity utilizes the distance metric.
(NA, Home, Weekday)
(NA, Home, Weekday)
(Kids, Home, NA) (Kids, Home, NA)

25. Modeling Context Similarity
• 4).Multidimensional Context Similarity (MCS)
Similarity-Based CSLIM:
Key points in MCS:
1). Each contextual variable is represented as an axis;
2). Each contextual condition is one position in corresponding axis;
3). Thus a contextual situation is mapped as a point in the space;
4). The distance between two points is viewed as dissimilarity;
Any distance metric can be applied; here we use Euclidean distance.
What to be learnt: the positions of each condition in axises.

26. Modeling Context Similarity
• Summary
Similarity-Based CSLIM:
What to be learnt in each context similarity model:
ICS LCS
The correlation (real value) for each
individual pair of context conditions
The vector representation (weights in
factors) for each contextual condition
WJCS MCS
The weights for each context
dimension.
The positions (real values) for each
contextual condition

27. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

28. Experimental Evaluations
• Data Sets
Note: The number of context-aware data sets is really limited!!
We use 5-folds cross validation for evaluation purposes.
We use Precision and Mean Average Precision (MAP) as metrics:
- Precision: measuring the hit ratio towards relevant items;
- MAP: additional taking the rankings of items into account.

29. Experimental Evaluations
• Algorithms for Comparison
1). Baseline Algorithms
CASA = Context-aware Splitting Approaches (a pre-filtering approach)
TF = Tensor Factorization (independent contextual modeling)
CAMF = Context-aware MF (dependent contextual modeling)
Deviation Model = CSLIM using deviation-based contextual modeling
2). New Algorithms
Four algorithms using different context similarity representations:
Similarity-ICS Model, Similarity-LCS Model
Similarity-WJCS Model, Similarity-MCS Model
Note: all those models were built on SLIM.

30. Experimental Evaluations

31. Experimental Evaluations
• Summary of the results
1). Which algorithm is the best?
Answer: Similarity-Based CSLIM using Multidimensional Context Similarity
2). Which one is better? Deviation or similarity-based modeling?
Answer: we can always find a similarity-based contextual modeling outperforming the
deviation-based modeling; but, the appropriate representation for context similarity
should be selected.
3). Which representation is the best?
Generally speaking, latent context similarity always outperforms independent context
similarity; and multidimensional context similarity is the best choice. Weighted
Jaccard context similarity shows non-stable recommendation performance in the
experiments.

32. Agenda
• RecSys and Context-aware RecSys
• Contextual Modeling
• SLIM and Contextual SLIM
• Modeling Context Similarity
• Experimental Evaluations
• Conclusions and Future Work

33. Conclusions & Future Work
• Conclusions
 We propose a new way to build dependent contextual modeling – similarity-
based contextual modeling;
 We choose SLIM as the base algorithm and incorporate context similarity into
SLIM to formulate new contextual SLIM algorithms;
 We discuss different representations to model context similarity;
 We demonstrated the advantages of similarity-based CSLIM by experimental
evaluations over multiple context-aware data sets.
• Future Work
Multidimensional Context Similarity (MCS) is the best representation to model
context similarity; but it increases computational costs at the same time. In our
future work, we’d like to explore how to reduce the computational costs for MCS,
e.g., reducing context dimensions, merging contextual conditions, etc.

34. Conclusions & Future Work
• Stay Tuned
Context similarity can also be incorporated into matrix factorization.
 Yong Zheng, Bamshad Mobasher, Robin Burke. "Incorporating Context
Correlation Into Context-aware Matrix Factorization". Workshop on Intelligent
Personalization @ IJCAI 2015
 Yong Zheng, Bamshad Mobasher, Robin Burke. "Correlation-Based Context-
aware Matrix Factorization". In DePaul CDM School of Computing Research
Symposium, 2015 (Best Paper Award)
• Survey: Context-aware Movie Ratings
Welcome to fill out it: http://depaul.qualtrics.com/SE/?SID=SV_4TrIZbAnQtzaHsx
Short URL: http://tinyurl.com/surveycars

35. Integrating Context Similarity
with Sparse Linear
Recommendation Model
Yong Zheng, Bamshad Mobasher, Robin Burke
Center for Web Intelligence, DePaul University, Chicago, USA
The 23rd Conference on User Modeling, Adaptation and Personalization,
Dublin, Ireland, June 29 – July 3, 2015 (UMAP 2015)