Presentation of the Phd. thesis defense of Gabriel de Souza Pereira Moreira at Instituto Tecnológico de Aeronáutica (ITA), on Dec. 09, 2019, in São José dos Campos, Brazil.
Abstract:
Recommender systems have been increasingly popular in assisting users with their choices, thus enhancing their engagement and overall satisfaction with online services. Since the last decade, recommender systems became a topic of increasing interest among machine learning, human-computer interaction, and information retrieval researchers.
News recommender systems are aimed to personalize users experiences and help them discover relevant articles from a large and dynamic search space. Therefore, it is a challenging scenario for recommendations. Large publishers release hundreds of news daily, implying that they must deal with fast-growing numbers of items that get quickly outdated and irrelevant to most readers. News readers exhibit more unstable consumption behavior than users in other domains such as entertainment. External events, like breaking news, affect readers interests. In addition, the news domain experiences extreme levels of sparsity, as most users are anonymous, with no past behavior tracked.
Since 2016, Deep Learning methods and techniques have been explored in Recommender Systems research. In general, they can be divided into methods for: Deep Collaborative Filtering, Learning Item Embeddings, Session-based Recommendations using Recurrent Neural Networks (RNN), and Feature Extraction from Items' Unstructured Data such as text, images, audio, and video.
The main contribution of this research was named CHAMELEON a meta-architecture designed to tackle the specific challenges of news recommendation. It consists of a modular reference architecture which can be instantiated using different neural building blocks.
As information about users' past interactions is scarce in the news domain, information such as the user context (e.g., time, location, device, the sequence of clicks within the session), static and dynamic article features like the article textual content and its popularity and recency, are explicitly modeled in a hybrid session-based recommendation approach using RNNs.
The recommendation task addressed in this work is the next-item prediction for user sessions, i.e., "what is the next most likely article a user might read in a session?". A temporal offline evaluation is used for a realistic offline evaluation of such task, considering factors that affect global readership interests like popularity, recency, and seasonality.
Experiments performed with two large datasets have shown the effectiveness of the CHAMELEON for news recommendation on many quality factors such as accuracy, item coverage, novelty, and reduced item cold-start problem, when compared to other traditional and state-of-the-art session-based algorithms.
[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems
1. CHAMELEON: A Deep Learning
Meta-Architecture for News
Recommender Systems
Gabriel de Souza Pereira Moreira
Advisor: Prof. Dr. Adilson Marques da Cunha
Doctoral Thesis Defense
Instituto Tecnológico de Aeronáutica
09/12/2019
2. Introduction
"We are leaving the Information Age and entering the
Recommendation Age.".
Cris Anderson, "The long tail"
2
3. News consumption is the majority of web traffic (TREVISIOL et al. , 2014b)
3
Introduction
4. 4
Research trends on News Recommender Systems (KARIMI et al. , 2018)
Introduction
7. 7
1. Preferences shift
2. Sparse user profiles
Introduction
News Recommender Systems Challenges
# sessions by user
(G1 dataset, 16 days)
Q3 = 3 sessions
Q2 = 2 sessions
Q1 = 1 session
8. 8
1. Preferences shift
2. Sparse user profiles
3. Fast growing number of items
Introduction
News Recommender Systems Challenges
# fresh articles by day
(G1 dataset)
9. 9
1. Preferences shift
2. Sparse user profiles
3. Fast growing number of items
4. Accelerated item’s value decay
Introduction
News Recommender Systems Challenges
Permanent User and Item Cold-Start Problem
Articles age at click time distribution
(G1 dataset)
Percentile of
# clicks
Article Age
10% < 4 hours
25% < 5 hours
50% < 8 hours
75% < 14 hours
90% < 26 hours
10. Objectives
10
1. To investigate the quality of news recommendation
using session-based algorithms
2. To design a Deep Learning meta-architecture for
hybrid and contextual recommendation, based on
Recurrent Neural Networks, to improve the quality of
recommendations provided by news portals.
11. Research Questions
● How does the proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news
domain, in terms of recommendation quality (accuracy, item coverage, novelty, and diversity),
compared to existing approaches for session-based recommendation?
11
● What is the effect on news recommendation quality factors of leveraging different types of
information in a neural-based hybrid recommender system?
● What is the effect on news recommendation quality of using different textual representations,
produced by statistical NLP and Deep NLP techniques?
● Is the proposed hybrid RNN-based architecture able to reduce the problem of item cold-start in
the news domain, compared to other existing approaches for session-based recommendation?
● Is the proposed approach effective to balance the competing quality factors of accuracy and
novelty?
1
2
3
4
5
12. Scope (1/2)
● Session-based news recommendation
● Task: Next-click prediction for user sessions in a news portal
12
13. Scope (1/3)
● Session-based news recommendation
● Task: Next-click prediction for user sessions in a news portal
13
User session clicks
C1
C2
C3
C4
Next-click prediction
model
Article B
Article A
Article C
Article D
...
Ranked articles
Recommendable articles (sampled)
14. Scope (2/3)
● The proposed Deep Learning meta-architecture should be:
○ Session-based
○ Hybrid
○ Contextual
14
● Recommendation quality factors to be evaluated:
○ Accuracy
○ Item Coverage
○ Novelty
○ Diversity
○ Robustness against the Item Cold-Start Problem
15. Recommender System
Content-based filtering Collaborative filtering
Model-based filteringMemory-based filtering
Item-basedUser-based
ML-based: Clustering, Association Rules,
Matrix Factorization, Neural Networks
Hybrid filtering+ =
Focus of this
research
15
Scope (3/3)
17. Deep Learning adoption in RecSys
2007
2015
2016
2017-2019
Deep Boltzmann Machines
for rating prediction
calm before the
storm
A few seminal papers
First DLRS workshop and
papers on RecSys, KDD,
SIGIR
Continued increase
17
18. Research directions in DL-RecSys
And their combinations...
Session-based recommendations with RNNs
Feature Extraction directly from the content
Learning Item embeddings
Deep Collaborative Filtering
Multi-view Learning
Focus of this research
18
19. Related work
19
Main inspirations:
● GRU4Rec (HIDASI, 2016) - The seminal work on the usage of Recurrent Neural
Networks (RNN) on session-based recommendations, and subsequent work
(Hidasi,2017).
● MV-DNN (ELKAHKI, 2015) - Adapted Deep Structured Semantic Model (DSSM) for the
recommendation task. MV-DNN maps users and items to a latent space, where
the cosine similarity between users and their preferred items is maximized. That
approach makes it possible to keep the neural network architecture static, rather
than adding new units into the output layer for each new item.
20. CHAMELEON: A Deep Learning
Meta-Architecture for News
Recommendation
21. The conceptual model of news relevance factors
2121
News
relevance
Topics Entities Publisher
News static properties
Recency Popularity
News dynamic properties
News article
User
TimeLocation Device
User current context
Long-term
interests
Short-term
interests
Global factors
Season-
ality
User interests
Breaking
events
Popular
Topics
Referrer
22. Meta-Architecture Requirements (1/2)
● RR1 - to provide personalized news recommendations in extreme
cold-start scenarios, as most news are fresh and most users cannot be
identified
● RR2 - to automatically learn news representations from textual
content and news metadata, minimizing the need of manual feature
engineering
● RR3 - to leverage the user session information, as the sequence of
interacted news may indicate user's short-term preferences for
session-based recommendations
● RR4 - to leverage users' contextual information, as a rich data source
in such information scarcity about the user
22
23. Meta-Architecture Requirements (2/2)
● RR5 - to model explicitly contextual news properties (popularity and
recency), as those are important factors on news interest life cycle
● RR6 - to support an increasing number of new items and users by
incremental model retraining (online learning), without the need to
retrain on the whole historical dataset
● RR7 - to provide a modular structure for news recommendation,
allowing its modules to be instantiated by different and increasingly
advanced neural network architectures and methods
23
27. The CHAMELEON Meta-Architecture for News RS
27
Dynamic
Article
Context
Article
Content
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
Content Embeddings Training
(CET)
Next-Article Recommendation (NAR)
Time
Location
Device
When a news article is published...
User context
User interaction
past read articles
Popularity
Recency
Article context
Article
Content
Embedding
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
User-Personalized Contextual Article Embedding
Recommended
articles
Contextual Article Representation (CAR)
Content word embeddings
New York is a multicultural city , ...
News Article
Active User Session
Module Sub-Module EmbeddingInput Output Data repositoryAttributes
Article Content Embedding (ACE)
Legend:
Word
Embeddings
28. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Article Content Representation
(ACR) module
Responsible for learning distributed
representations (embeddings) from news’
contents.
Sub-modules:
● Textual Features Representation (TFR)
● Content Embeddings Training (CET)
28
Article
Content
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
Content Embeddings Training
(CET)
When a news article is published...
Content word embeddings
New York is a multicultural city , ...
News Article
Article Content Embedding (ACE)
Word
Embeddings
29. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 29
Inputs
Article text represented by Word
embeddings
Word embeddings pre-trained in larger
corpus:
● Word2Vec
● GloVe
Article
Content
Embeddings
When a news article is published...
Content word embeddings
New York is a multicultural city , ...
News Article
Word
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
Content Embeddings Training
(CET)
Article Content Embedding (ACE)
30. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Textual Features Representation
(TFR) sub-module
● Learns relevant features from article
textual content (RR2).
● Can be instantiated by a Deep NLP
architecture like:
○ CNN
○ RNN
○ QRNN
○ Attention mechanisms
30
Article
Content
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
When a news article is published...
Content word embeddings
New York is a multicultural city , ...
News Article
Word
Embeddings
Article Content Embedding (ACE)
Content Embeddings Training
(CET)
31. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Article Content Embedding
● The learned article distributed
representation
● After training, it is stored in a
repository for further usage by NAR
module
31
Article
Content
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
When a news article is published...
Content word embeddings
New York is a multicultural city , ...
News Article
Article Content Embedding (ACE)
Word
Embeddings
Content Embeddings Training
(CET)
32. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Content Embeddings Training (CET)
● Responsible to train Article Content
Embeddings.
○ Supervised task: e.g. predict
article metadata attributes
○ Unsupervised task: e.g.
reconstruct a compressed
version of the input
(autoencoder)
● May be instantiated as a sequence of
feed-forward Fully Connected layers.
32
Article
Content
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
Content Embeddings Training
(CET)
When a news article is published...
Content word embeddings
New York is a multicultural city , ...
News Article
Article Content Embedding (ACE)
Word
Embeddings
33. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Next-Article Recommendation
(NAR) module
● Provide news articles
recommendations for each interaction
(I) in active user sessions.
● For each recommendation request,
NAR module generates a ranked list
of the most likely articles user might
read in a given session.
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
Time
Location
Device
User context
User interaction
past read articles
Popularity
Recency
Article context
Article
Content
Embedding
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
User-Personalized Contextual Article Embedding
Recommended
articles
Contextual Article Representation (CAR)
Active User Session
33
34. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
User-Personalized Contextual Article Embedding
Recommended
articles
Contextual Article Representation (CAR)
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Inputs
1 - Article Content Embedding
Pre-trained by the ACR module (RR2)
2 - User Context
Properties of user current context (RR4):
○ Time (Period of day, Weekday /
Weekend, Month, Quarter)
○ Location (City, State, Country)
○ Device (eg. Desktop / Smartphone)
○ Device OS (eg. Android / iOS)
34
35. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
User-Personalized Contextual Article Embedding
Recommended
articles
Contextual Article Representation (CAR)
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Inputs
3 - Article context
Contextual properties of the article (RR5):
● Recent Popularity - Popularity of the
article within the last hour
● Recency - Time elapsed since the
article was published
35
36. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 36
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
Recommended
articles
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
Contextual Article Representation
(CAR) sub-module
● Merges the inputs – article content,
article context and user context –
and generates the
User-Personalized Contextual
Article Embedding
● This sub-module might be
instantiated as Fully Connected (FC)
layers or using Factorization
Machines (FM), for example.
37. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 37
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
Recommended
articles
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
Sessions in a batch
I1,1
I1,2
I1,3
I1,4
I1,5
I2,1
I2,2
I3,1
I3,2
I3,3
38. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 38
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
Recommended
articles
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
Sampling strategy
Negative samples:
Articles read by other users in the last hour
Positive sample:
Next article read by the user in his session
Samples
39. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
SEssion Representation
(SER) sub-module
● Models short-term user’s
preferences in their active sessions
(RR3)
● Must be instantiated by an
architecture that is able to model the
sequence of user clicks, such as
RNN, CNN, attention mechanisms.
39
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
Recommendations Ranking (RR)
Recommended
articles
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
SEssion Representation (SER)
40. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Predicted Next-Article Embedding
● Represents the expected
representation of a news content the
user would like to read next in the
active session.
40
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Recommendations Ranking (RR)
Recommended
articles
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
SEssion Representation (SER)
Predicted Next-Article Embedding
41. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Recommendations Ranking
(RR) sub-module
● Recommends articles to a user session
based on embeddings similarity,
supporting unseen items during
inference (RR6).
● The loss function maximizes the
similarity between “Predicted
Next-Article Embedding” and the
“User-Personalized Contextual Article
Embedding” corresponding to the
positive sample, and minimize
similarities with negative samples.
41
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Recommended
articles
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
SEssion Representation (SER)
Predicted Next-Article Embedding
Recommendations Ranking (RR)
42. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Recommendations Ranking
(RR) sub-module
42
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
SEssion Representation (SER)
Predicted Next-Article Embedding
Recommended
articles
Recommendations Ranking (RR)
Eq. 1 - Relevance Score of an item for a user session
Eq. 2 - Matching function
Eq. 3 - Softmax over Relevance Score
Eq. 4 - Accuracy Loss function
43. The CHAMELEON Meta-Architecture for News RS
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Recommendations Ranking
(RR) sub-module
43
Dynamic
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
User interaction
past read articles
Article context
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Active User Session
Time
Location
Device
Article
Content
Embedding
Popularity
Recency
Contextual Article Representation (CAR)
User-Personalized Contextual Article Embedding
SEssion Representation (SER)
Predicted Next-Article Embedding
Recommended
articles
Recommendations Ranking (RR)
Eq. 5 - Novelty Loss function
Eq. 6 - Novelty
Eq. 7 - Recent Normalized Popularity
Eq. 8 - Multi-objective Loss Function
44. The CHAMELEON Meta-Architecture for News RS
44
Dynamic
Article
Context
Article
Content
Embeddings
Article Content Representation (ACR)
Textual Features Representation (TFR)
Content Embeddings Training
(CET)
Next-Article Recommendation (NAR)
Time
Location
Device
When a news article is published...
User context
User interaction
past read articles
Popularity
Recency
Article context
Article
Content
Embedding
candidate next articles
(positive and neg.)
active article
Active
Sessions
When a user reads a news article...
Predicted Next-Article Embedding
SEssion Representation (SER)
Recommendations Ranking (RR)
User-Personalized Contextual Article Embedding
Recommended
articles
Contextual Article Representation (CAR)
Content word embeddings
New York is a multicultural city , ...
News Article
Active User Session
Module Sub-Module EmbeddingInput Output Data repositoryAttributes
Article Content Embedding (ACE)
Legend:
Word
Embeddings
45. 45
Comparison among Deep Architectures for News RS
Characteristics (SONG, 2016) (KUMAR, 2017) (PARK, 2017) (OKURA, 2017) (WANG, 2018) (ZHANG, 2018) CHAMELEON
Session-based
Multiple quality
objectives
Temporal articles
relevance decay (exponential
function)
(decay learned by
the model)
Time-aware negative
sampling
N/A - Does not use
negative sampling
Article Context
(Recency, Recent Pop)
User Context
Textual Content
Features
Letter trigrams,
processed by
MLP
doc2vec PV-DBoW Denoising
auto-encoder
Entities enrichment
of title with a
Knowledge Graph
Character-level
CNN
Sup. and Unsup.
ACR instantiations
(CNN, GRU)
DNN architecture for
sequence modeling
RNN RNN RNN RNN DNN with attention RNN RNN
Loss function Listwise ranking
based on DSSM
Listwise ranking
based on DSSM
Pairwise
ranking BRP,
TOP1
Pairwise ranking
with position bias
Pointwise ranking:
Log loss
Pairwise ranking:
BRP, TOP1
Listwise ranking
based on DSSM
(custom)
47. ACR module - Supervised - CNN-based
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend:
Output: Article Metadata Attributes
● If more than one attribute is available
for classification, Multi-Task Learning
(MTL) is used.
TASK A TASK B TASK C
Task-specific
layers
Shared
layers
47
Article
Content
Embeddings
Article Content Representation
(ACR)Textual Features Representation (TFR)
Content Embeddings Training (CET)
Content word embeddings
New York is a multicultural city , ...
News Article
Article Content Embedding (ACE)
Word
Embeddings
Convolutional Neural Network (CNN)
conv-3 (128)
max-pooling
conv-4 (128)
max-pooling
conv-5 (128)
max-pooling
Fully Connected layers
Category
Target Article Metadata Attributes
Fully Connected layers
When a news article is published...
48. ACR module - Supervised - CNN-based
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 48
Article
Content
Embeddings
Article Content Representation
(ACR)Textual Features Representation (TFR)
Content Embeddings Training (CET)
Content word embeddings
New York is a multicultural city , ...
News Article
Article Content Embedding (ACE)
Word
Embeddings
Convolutional Neural Network (CNN)
conv-3 (128)
max-pooling
conv-4 (128)
max-pooling
conv-5 (128)
max-pooling
Fully Connected layers
Category
Target Article Metadata Attributes
Fully Connected layers
Output: Article Metadata Attributes
● The last layer of ACR module is
responsible to classify one or more
article metadata attributes
● Each attribute might be single-label
(e.g. category) or multi-label (e.g.
tags), in Eq. 9 and 10, respectively
Eq. 9 - Softmax Eq. 10 - Sigmoid
Eq. 11 - Cross-entropy
When a news article is published...
49. ACR module - Supervised - RNN-based
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 49
Article
Content
Embeddings
Article Content Representation
(ACR)Textual Features Representation (TFR)
Content Embeddings Training
(CET)
When a news article is published...
Content word embeddings
New York is a multicultural city ,
...
News Article
Article Content Embedding (ACE)
Word
Embeddings
Fully Connected layers
Category
Target Article Metadata Attributes
Fully Connected layers
GRU encoderinput word embeddings
hidden
state
RNN based of TFR sub-module
● Process the sequence of word
embedding using an RNN
● P.s. Not as computationally efficient as
CNNs
50. ACR module - Unsupervised
Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 50
Article Content Representation (ACR)
Textual Features Representation (TFR)
Content Embeddings Training (CET)
When a news article is published...
Content word embeddings
New York is a multicultural city , ...
News Article
Article Content Embedding (ACE)
GRU encoder
Fully Connected layer
Reconstructed word embeddings
Fully Connected layer
input word embeddings
hidden
state
GRU decoder
New York is a multicultural city , ...
hidden state
Article
Content
Embeddings
Word
Embeddings
Autoencoder based instantiation of the
ACR module
● A Sequence Denoising Autoencoder
● The objective is to reconstruct the
sequence of corrupted input word
embeddings from a compressed
representation (Article Content
Embedding)
● P.s. The sequence of input words is
reversed to improve reconstruction
accuracy
51. 51
NAR module instantiation
Article
Context
Article
Content
Embeddings
Next-Article Recommendation (NAR)
User context
past read articles
Popularity
Recency
Article context
Article
Content
Embedding
candidate next articles
(positive and neg.)
active articleActive
Sessions
Predicted Next-Article Embedding
Session Representation (SER)
Recommendations Ranking (RR)
User-Personalized Contextual Article Embedding
Recommended
articles
Contextual Article Representation (CAR)
Fully Connected layers
UGRNN + Fully Connected layers
Time
Location
Device
Referrer
User interaction
When a user reads a news article...
Active User Session
NAR module instantiation
● Learnable parameters for centering and
scale input features
● CAR sub-module was instantiated by 2
FC layers (Leaky ReLU + tanh)
● SER sub-module was instantiated by a
2-layer UGRNN cell (COLLINS, 2017)
followed by 2 FC layers (Leaky ReLU +
tanh).
52. CHAMELEON Implementation
52
● CHAMELEON’s instantiations are implemented using TensorFlow
https://github.com/gabrielspmoreira/chameleon_recsys
● Training and evaluation performed in Google Cloud Platform ML Engine
54. Evaluation Protocol
54
Task: For each item within a session, predict the next-clicked item from a set
composed by the positive sample (correct article) and 50 negative samples.
Hours
Input
I1
I2
I3
I4
I2
I3
I4
I5
Expected Output (labels)
55. Metrics (1/2)
55
Accuracy HR@10 Hit Rate - Checks whether the positive item is among the top-N
ranked items
MRR@10 Mean Reciprocal Rank - Ranking metric which assigns higher
scores at top ranks
Item
Coverage
COV@10 # recommended articles @ top-n
# recommendable articles
58. Datasets (1/2)
58
G1 dataset
● Provided by Globo.com, the most popular news portal in Brazil
● Made publicly available by this research for the community
https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom
59. 59
G1 (globo.com) Adressa
Language Portuguese Norwegian
Period (days) 16 16
# users 322 k 314 k
# sessions 1,048 k 982 k
# clicks 2,988 k 2,648 k
# articles 46 k 13 k
Avg. session length * 2.84 2.70
* After removing single-click sessions and sessions with more than 20 clicks.
Datasets (2/2)
60. Baseline algorithms
60
Neural methods
GRU4Rec A landmark neural architecture using RNNs for session-based recommendation
SR-GNN A recently state-of-the-art architecture for session-based recommendation based on GNN.
Frequent patterns methods
CO A simple method of co-occurrence based on association rules of length two.
SR Sequential rules of size two.
kNN methods
Item-kNN Most similar articles to the last read one, in terms of the cosine similarity between the vector of their occurrence in sessions.
V-SkNN Compares the entire active session with past (neighboring) sessions to determine items to be recommended. The similarity function
emphasizes items that appear later within the session.
Non-personalized methods
RP Recently Popular - Most viewed articles during the last hour.
CB Content-Based - Most similar articles (content embedding) to the last user click.
62. Research Questions
How does the proposed hybrid RNN-based architecture – the
CHAMELEON – perform in the news domain, in terms of
recommendation quality factors (accuracy, item coverage, novelty,
and diversity), compared to existing approaches for session-based
recommendation?
62
DLRS @ RecSys, 2018 IEEE Access, 2019
(Qualis A2, Eng. IV)
1
69. News recommendation quality of session-based algorithms
69
Evaluation of other quality factors
1Adressa dataset
70. Research Questions
70
What is the effect on news recommendation quality factors of
leveraging different types of information in a neural-based hybrid
recommender system?
IEEE Access, 2019
(Qualis A2, Eng. IV)
2
71. 71
The effect of different types of information
2
Input
Configuration
Feature Sets
IC1 Article Id
IC2 IC1 + Article Context (Novelty and Recency)
IC3 IC2 + the Article Content Embeddings (ACE) learned by the supervised
instantiation of the ACR module
IC4 IC3 + Article Metadata
IC5 IC4 + User Context
73. Research Questions
73
What is the effect on news recommendation quality of using different
textual representations, produced by statistical NLP and Deep NLP
techniques?
INRA @ RecSys, 2019
3
74. 74
The effect of different content representations
3ACRmodule
instantiations
75. 75
Recommender HR@10 MRR@10
No-ACE 0.6281 0.3066
Supervised
CNN 0.6585 0.3395
GRU 0.6585 0.3388
Unsupervised
SDA-GRU 0.6418 0.3160
W2V*TF-IDF 0.6575 0.3291
LSA 0.6686* 0.3423
doc2vec 0.6368 0.3119
Comparison of different content encodings for CHAMELEON
The effect of different content representations
G1 dataset
3
76. 76Comparison of different content encodings for CHAMELEON
The effect of different content representations
Adressa dataset
3
Recommender HR@10 MRR@10
No-ACE 0.6816 0.3252
Supervised
CNN 0.6860 0.3333
GRU 0.6856 0.3327
Unsupervised
SDA-GRU 0.6905 0.3360
W2V*TF-IDF 0.6913 0.3402
LSA 0.6935 0.3403
doc2vec 0.6898 0.3402
77. Research Questions
Is the proposed hybrid RNN-based architecture able to reduce the
problem of item cold-start in the news domain, compared to other
existing approaches for session-based recommendation?
77
4
78. 78
The effect on item cold-start
4
# Batches before First Recommendation (BFR@n)
G1 dataset
79. 79
The effect on item cold-start
4
Adressa dataset
# Batches before First Recommendation (BFR@n)
80. Research Questions
Is the proposed approach effective to balance the competing quality
factors of accuracy and novelty?
80
IEEE Access, 2019
(Qualis A2, Eng. IV)
5
83. Main Contributions
83
1. The investigation of the stated research questions related to the
effectiveness of session-based recommendation algorithms on the
news domain
2. The design, implementation, and evaluation of a Deep Learning
meta-architecture – the CHAMELEON – for hybrid and contextual
session-based news recommendation
84. Complementary Contributions
84
1. The elaboration of a Conceptual Model of factors that affect news relevance
2. The four instantiations of ACR and NAR modules of CHAMELEON
3. The open-source implementation of CHAMELEON 1
4. The temporal offline evaluation protocol for news recommendation
5. The adaptation of rank- and relevance-sensitive novelty and diversity metrics
Additional Contribution
6. The preprocessing and sharing of two novel datasets for evaluation of hybrid
and contextual RS for news (G1) 2
and articles (CI&T Desktop) 3
1
https://github.com/gabrielspmoreira/chameleon_recsys
2
https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom
3
https://www.kaggle.com/gspmoreira/articles-sharing-reading-from-cit-deskdrop
85. Recommendations
85
1. Online evaluation of CHAMELEON in a live news portal
2. To balance more than two quality factors (including diversity)
3. To adapt CHAMELEON for session-aware news recommendation
4. To explore different negative sample approaches (e.g., by geographic
region)
5. Explore other Deep NLP techniques (e.g. BERT) to produce better article
content embeddings
6. To deepen the investigation of the RQ4 about the item cold-start problem
(next paper)
86. Suggestions for Future Works
86
1. Instantiate the SER sub-module with a neural network architecture other
than RNNs (e.g., CNN, QRNN, GNN)
2. Investigate attention mechanisms as a way to improve accuracy and
provide recommendation explanations
3. Investigate “noisy outliers” in the user profiles
4. Adapt CHAMELEON for contextual and hybrid session-based
recommendations in other domains like e-commerce, entertainment and
media
87. Published Articles
87
1. MOREIRA, Gabriel de Souza Pereira; SOUZA JUNIOR, G. J. A. ; CUNHA, A. M. . Comparing offline and online recommender
system evaluations on long-tail distributions. In: ACM Recommender Systems Conference, 2015, Viena, Austria. Poster
Proceedings of ACM RecSys 2015, 2015. v. 1441. p.
2. MOREIRA, GABRIEL DE SOUZA P.; SOUZA, GILMAR . A Recommender System to tackle Enterprise Collaboration. In: the
10th ACM Conference, 2016, Boston. Proceedings of the 10th ACM Conference on Recommender Systems - RecSys '16
(Demo paper). New York: ACM Press, 2016. p. 395. DOI https://doi.org/10.1145/2959100.2959115
3. MOREIRA, GABRIEL. CHAMELEON: a Deep Learning Meta-Architecture for News Recommender Systems. In: 12th ACM
Conference on Recommender Systems - RecSys'18, 2018, Vancouver. Proceedings of the 12th ACM Conference on
Recommender Systems RecSys'18 - Doctoral Symposium. New York: ACM Press, 2018. DOI
https://doi.org/10.1145/3240323.3240331
4. MOREIRA, GABRIEL; FERREIRA, FELIPE ; DA CUNHA, ADILSON MARQUES . News Session-Based Recommendations
using Deep Neural Networks. Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems - DLRS 2018.
New York: ACM Press, 2018. p.15. DOI https://doi.org/10.1145/3270323.3270328
5. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES . On the Importance of News Content
Representation in Hybrid Neural Session-based Recommender Systems. Proceedings of the 7th Workshop on News
Recommendation and Analytics (INRA 2019), in conjunction with RecSys 2019, September 19, 2019, Copenhagen
6. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES .. "Contextual Hybrid Session-Based News
Recommendation With Recurrent Neural Networks." IEEE Access 7 (2019): 169185-169203.
88. CHAMELEON: A Deep Learning
Meta-Architecture for News
Recommender Systems
Gabriel de Souza Pereira Moreira
Advisor: Prof. Dr. Adilson Marques da Cunha
Doctoral Thesis Defense
Instituto Tecnológico de Aeronáutica
09/12/2019