Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo

Copyright © 2017 Criteo
Highlights of RecSys'17
Elena Smirnova, Amine Benhalloum, Lowik Chanussot
Criteo
25/09/2017

Introduction
• RecSys'17 was held in Como, Italy on August, 27-31
• +600 participants
• Increasing number of industry sessions
• Criteo presented 2 papers on Deep learning workshop

RecSys'17: Key topics
• Session-based recommendation: Elena
• Representation learning: Amine
• Scalability: Lowik

Session-based
recommendation

Session-based recommendation
Classical setup: independent user-item observations
link prediction
matrix factorization
New setup: sequences of user-item interactions in time
next event prediction
Time

Recurrent Neural Networks
RNNs for session-based recommendation introduced in 2015
Learns sequence embedding (aka internal state) that represents the sequence
of user-item interactions
Performs the same computation at each time step
Hidasi et al. Session-based Recommendations with Recurrent Neural Networks.

RecSys’17: Stronger baselines
Session-based kNN
• Find k most similar past sessions
• Cosine similarity of bit vectors
• Score items by the sum of session similarities
D. Jannach and M. Ludewig. When Recurrent Neural Networks meet the Neighborhood for Session-Based
Recommendation.

RecSys’17: Hierarchical Extension
Hierarchical RNNs model long-term user behavior across sessions
2 RNNs: user and session representation
M. Quadrana et al. PersonalizingSession-basedRecommendations with Hierarchical RecurrentNeuralNetworks.

RecSys’17: Contextual Extension
Condition RNN on contextual information (event type, timestamp)
Integrate at 2 levels:
Input and Output layers
Hidden Dynamics
E. Smirnova and F. Vasile. Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks.

Wrap up
• Session-based recommendation has now it own track
• Stronger baselines have been introduced
• Multiple extensions to Recurrent Neural Networks to better model user
behavior

Representation learning

Learning to represent items, users
and their relationships in an
appropriate space (as a real valued
vector)

Leveraging available content
(images, descriptions, reviews …)
Helping with the cold start
problem

Leveraging content: Review texts
R. Catherine et al. Transnets: Learning to transform for recommendation
• Review texts are available, how
can we use them ?
• Learn a representation of the
review and then predict the
associated rating

Leveraging content: Item features
T. Nedelec et al. : Content2Vec: Specializing jointrepresentationsof productimagesand text for the task of product
recommendation
• How can we combine
heterogeneous product
representations ?
• Specialize feature
representations (text,
image, …) for the task of
item-item similarity and
merge them

Cold start problem: Attribute to feature mapping
• Can we use item characteristics to initialize an new item's latent
representation ?
• Learning attribute (item characteristics) to feature (latent space representation)
mapping.
D. Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation

Cold start problem: Attribute to feature mapping
• We learn a linear mapping between attribute vectors {𝒂𝒊} and latent
representation {𝒗𝒊}
𝑣𝑖 ≈ 𝑊𝑎𝑖
• For a new item 𝒋 we initialize its latent representation
𝑣𝑗
0
≈ 𝑊𝑎𝑗
D.Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation

Folding
• “Folding” effect of embedding can
lead to spurious recommendations
• Model has to take into account data
Missing Not At Random, introduce
metric to measure the severity of
folding
D. Xin et al. Folding: Why Good Models Sometimes Make Spurious Recommendations.

Wrap up
• Embed all the things !
• A Deep MultimodalApproach for Cold-start Music Recommendation(Oramas et al.)
• Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation
(Dominguez et al.)
• Translation based recommendation (He et al.)
• Use content and reviews
• InterpretableConvolutional Neural Networks with Dual Local and Global Attention for
Review Rating Prediction (Seo et al.)
• Recommendation of High Quality Representative Reviews in e-commerce (Paul et al.)
• Sequential recommendations
• Sequential User-based Recurrent Neural Network Recommendations (Donkers et al.)

Scalability

Large scale constraints
Many products, many users
Online latency
Training time
Offline storage

Convolution at character level for session-based reco
Id
Name
Category
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2 Item #n
view view basket

Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1 Id: 263N
Item #1 Item #2
view view basket
Item #n

Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1 Id: 263
Name: “iPhone”
Category:
“Phones/Apple”
V
N
N x 3V
Item #1
Item #1 Item #2
view view basket
Item #n

Item #1 Item #2
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1
3*V = 56
N=150
Compact Input 3V x N x D
Id: 263
Name: “iPhone”
Category:
“Phones/Apple”
V
N
N x 3V
Item #1
view view basket
Item #n

Scaling deep nets
Your favorite
deep net
D x L1 D x Ln
- Compact
- Fast to compute
- For input and output
- Reversible output
- No change in the deep-net
- Appropriate loss
- Same accuracy
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks

Bloom filters embeddings
1
1
1
H1
H2
Hk
1
D
m < D
p

1
1
1
H1
H2
Hk
1
Your favorite
deep net
0.1
0.3
0.2
D
m < D
m x L1 m x Ln
p
v

1
1
1
H1
H2
Hk
1
Your favorite
deep net
0.1
0.3
0.2
D
m < D
m x L1 m x Ln
1 q
p
H1
Hk
y
v
H2

Wrap up
Dedicated workshop
Preselection of products
Scaling deep nets
Online
Ranking
Candidates
Items
Offline
Selection
N products
K products (K << N)

Conclusion
RecSys’18 - Vancouver, Canada, 2nd-7th October 2018

Thank you!

Join Criteo!

Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo

Recommended

Recommended

More Related Content

Similar to Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo

Similar to Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo (20)

More from recsysfr

More from recsysfr (17)

Recently uploaded

Recently uploaded (20)

Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo