A look behind L’Oréal’s tool for consumer feedback analysis. We will discuss how NLP and Computer Vision have been applied to analyse large volumes of product reviews. Specifically, we’ll talk about topic extraction, sentiment analysis and topic enrichment in NLP, and siamese neural networks with triplet loss in Computer Vision.
Gauging Business Disruption with the Disruptability Index | Accenture
Ähnlich wie “NLP and Computer Vision Applications in Consumer Feedback Analysis” by Olesia Khrapunova - Data Scientist at l'Oreal Beauty Tech Accelerator
Democratizing Data Science on Kubernetes John Archer
Ähnlich wie “NLP and Computer Vision Applications in Consumer Feedback Analysis” by Olesia Khrapunova - Data Scientist at l'Oreal Beauty Tech Accelerator (20)
“NLP and Computer Vision Applications in Consumer Feedback Analysis” by Olesia Khrapunova - Data Scientist at l'Oreal Beauty Tech Accelerator
1. 1
C1 - Internal use
Consumer Loop
Actionable consumer
feedbacks
in an AI powered tool
R&I
2. 2
C1 - Internal use
Olesia Khrapunova
Data Scientist at
Beauty Tech Accelerator Paris
@L’Oréal
UVA
Marketing &
Business Analytics
Clarabridge
(now Qualtrics)
Business
Consultant
ESSEC +
CentraleSupélec
Data Science &
Business Analytics
L’Oreal
Data Scientist
3. 3
C1 - Internal use
TECHACCELERATOR
THE TECH
ACCELERATOR IS
OUR BEAUTY TECH
TRANSFORMATION
CATALYST BRINGING
STRATEGIC & TECH
IDEAS TO LIFE
4. 4
C1 - Internal use
TECHACCELERATOR
KEY PURPOSE
• Catalyse our tech transformation
• Deliver data science use cases,
validated by top level, at scale
• Augment our employees with tech
solutions that play a strategic role
for the group
5. 5
C1 - Internal use
TECHACCELERATOR
UX
DESIGN
DATA
SCIENCE
TEAM
PRODUCT
MANAGEMENT
6. 6
C1 - Internal use
Promo AI
BetIQ
DATA POWERED BUSINESS
Consumer Loop
CONSUMER VOICE
TrendSpotter SkinDoctor Fast & Clean
Color
DIGITAL FORMULATION
F1 Digital
Formulation
GAAH
• Strategic priorities
• CPP validated
• At scale
• Products that
people love
• Drive change
PRODUCT
PORTFOLIO
TECH ACCELERATORS
TECH ENABLER
iClosing
7. 7
C1 - Internal use
TECHACCELERATOR
CONSUMER LOOP IS AN AI WEB APPLICATION THAT TURNS
CONSUMER REVIEWS INTO ACTIONABLE INSIGHTS
8. 8
C1 - Internal use
TECHACCELERATOR
CONSUMER LOOP COVERS THE MAIN BEAUTY CATEGORIES
ACROSS PRIORITIZED MARKETS
Four categories
Make-up Skincare
Perfume
Haircare
Nine geographies
USA
Mexico
Brazil
China
Indonesia
India
France
UK
Germany
9. 9
C1 - Internal use
TECHACCELERATOR
LET’S TAKE A LOOK AT CONSUMERLOOP.BEAUTY.TECH!
#
10. 10
C1 - Internal use
TECHACCELERATOR
LET’S TAKE A LOOK AT CONSUMERLOOP.BEAUTY.TECH!
#
11. 11
C1 - Internal use
TECHACCELERATOR
LET’S TAKE A LOOK AT CONSUMERLOOP.BEAUTY.TECH!
13. 13
C1 - Internal use
TECHACCELERATOR
Sentiment Analysis
Topic Enrichment
Topic Extraction
SEVERAL ALGORITHMS HELP ANALYZE FEEDBACK
‘‘
’’
The mascara does not flake,
but dries out too quickly.
I prefer the old formula.
Topic Extraction Topic Enrichment Sentiment Analysis
Flakes
Product dryness
Discontinued
formula
Flakes
Product dryness
Discontinued
formula
15. 15
C1 - Internal use
TECHACCELERATOR
THE GOAL OF PRODUCT DEFINITION IS TO GROUP TOGETHER
PRODUCT COMING FROM DIFFERENT SOURCES
Same platform
Different platforms in
the same country
Platforms from
different countries
16. 16
C1 - Internal use
TECHACCELERATOR
WE RECEIVE MULTIPLE DATA POINTS THAT CAN BE USED FOR
MATCHING
17. 17
C1 - Internal use
TECHACCELERATOR
PRODUCT GROUPS ARE BASED ON A GRAPH WHERE EDGES
ARE BUILT USING MULTIPLE DATA POINTS
Edges based on UPC,
Syndicated Reviews
Edges based on UPC,
Syndicated Reviews
+
Image and Title Similarity
18. 18
C1 - Internal use
TECHACCELERATOR
SIMILARITY PREDICTION IS OFTEN BASED ON TWO STEPS
Get feature vector
representation of the
inputs
Calculate distance
between the vectors
19. 19
C1 - Internal use
TECHACCELERATOR
WE CHOSE SIAMESE NEURAL NETWORK ARCHITECTURE FOR
TRAINING EMBEDDINGS
Distance
Base model
Base model
Architecture image source: https://omoindrot.github.io/triplet-loss
INPUT A
INPUT B
20. 20
C1 - Internal use
TECHACCELERATOR
WE TRAINED THE NETWORK USING TRIPLET LOSS
Architecture image source: https://omoindrot.github.io/triplet-loss
Base model
Base model
Base model
INPUT A
INPUT B
INPUT C
21. 21
C1 - Internal use
TECHACCELERATOR
FIRST, WE APPLIED THE APPROACH TO IMAGES
Architecture image source: https://omoindrot.github.io/triplet-loss
Base model
Base model
Base model
MobileNet2 /
ResNet /
InceptionV3
MobileNet2 /
ResNet /
InceptionV3
MobileNet2 /
ResNet /
InceptionV3
Cosine
Similarity
22. 22
C1 - Internal use
TECHACCELERATOR
MODEL TRAINING SIGNIFICANTLY IMPROVED PERFORMANCE
1.0 precision
Weights pre-trained on
ImageNet
~0.60 recall
Weights fine-tuned with
Triplet Loss
~0.70 recall*
+0.10
Base Model: ResNet50
*model didn’t reach 100% precison, but came very close to it (99.8%)
0.98 precision
~0.70 recall
~0.86 recall
+0.16
23. 23
C1 - Internal use
TECHACCELERATOR
WE ALSO USED THE APPROACH WITH PRODUCT TITLES
Architecture image source: https://omoindrot.github.io/triplet-loss
Base model
Base model
Base model
CountVectorizer
with trainable
weights
CountVectorizer
with trainable
weights
CountVectorizer
with trainable
weights
Advanced
Genifique
Radiance
Activating
Serum
20ml/0.67oz
Advanced
Génifique
Radiance
Boosting
Face Serum
L’Oréal Paris
Revitalift
Face Serum
20ml
Precision:
0.98
Recall:
0.68
Cosine
Similarity
24. 24
C1 - Internal use
TECH ACCELERATOR
Join the team!
theo.leccia@loreal.com
audrey.castillo@loreal.com
Hinweis der Redaktion
Each review is split into quotes
Interesting points:
Text classification model, one model per topic, allows for easier adjustment later if problems arise witha a specific category
LDA, where prior is based on keyword – topic associations we already know about
Trained to predict review rating based on text, custom,