Mario Rodriguez gave a presentation at Stanford on LinkedIn's talent recommendation products. He discussed how LinkedIn uses data on members, jobs, skills and companies to match talent to opportunities through products like TalentMatch and JYMBII. He also explained how LinkedIn predicts job transitions and migration patterns to improve recommendations. Finally, he briefly described how the economic graph maps the global economy using LinkedIn data.

1. Talent, Opportunity, and Data @ LinkedIn
Mario Rodriguez
Senior Data Scientist and Team Lead
Reputation – Recommendations – Search
LinkedIn Confidential ©2014 All Rights Reserved
E145 – Technology Entrepreneurship
Stanford
February 13, 2014

3. LinkedIn Confidential ©2013 All Rights Reserved 3
Members Worldwide
2 new
Members Per Second
100M+Monthly Unique Visitors
225 M+ 2M+
Company Pages
...

4. LinkedIn Confidential ©2013 All Rights Reserved 4
Members Worldwide
2 new
Members Per Second
100M+Monthly Unique Visitors
225 M+ 2M+
Company Pages
...

5. Some LinkedIn products (Agenda)
JYMBII
– Recommend opportunities to talent
TalentMatch
– Recommend talent to opportunities
Job Search
– Let talent find opportunities
People Search
– Let opportunities find talent
Bonus: Economic Graph
LinkedIn Confidential ©2013 All Rights Reserved 5

6. LinkedIn Confidential ©2013 All Rights Reserved 6
Members Worldwide
2 new
Members Per Second
100M+Monthly Unique Visitors
225 M+ 2M+
Company Pages
...

7. Some LinkedIn products (Agenda)
JYMBII
– Recommend opportunities to talent
TalentMatch
– Recommend talent to opportunities
Job Search
– Let talent find opportunities
People Search
– Let opportunities find talent
Bonus: Economic Graph
LinkedIn Confidential ©2013 All Rights Reserved 7

8. Job
Posting

9. Member
Profile

10. 10
Corpus StatsJob
User Base
Filtered
title
geo
company
industry
description
functional area
…
Candidate
General
expertise
specialties
education
headline
geo
experience
Current Position
title
summary
tenure length
industry
functional area
…
Similarity
(candidate expertise, job description)
0.56
Similarity
(candidate specialties, job description)
0.2
Transition probability
(candidate industry, job industry)
0.43
Title Similarity
0.8
Similarity (headline, title)
0.7
Transition probabilities
Connectivity
yrs of experience to reach title
education needed for this title
…
Interaction Features
Binary
Exact matches:
geo, industry,
…
Soft
transition
probabilities,
similarity,
…
Text
Source/Target Features
- Job Popularity
- Job Seeking Propensity
Find, Engage & Hire
Jobs You May be Interested In

11. Find, Engage & Hire
Jobs You May be Interested In

12.  Open to relocation ?
- Region similarity based on profiles or network
- Region transition probability
- predict individuals propensity to migrate and most
likely migration target
 Impact on job recommendations
 Lift in views/viewers/applications/applicants
Find, Engage & Hire
Jobs You May be Interested In

13. 13
What would you transition to .. and when ?
Months since graduation
Probabilityofswitch
Find, Engage & Hire
Jobs You May be Interested In

14. Where are you likely to stay ?
14
Find, Engage & Hire
Jobs You May be Interested In

15. Power of aggregation..
Before
employees worked at
Yahoo! (247)
Google (139)
Microsoft (105)
Oracle (93)
eBay (68)
Before
employees worked at
Microsoft (1379)
IBM (939)
Yahoo! (608)
Oracle (558)
15
Find, Engage & Hire
Jobs You May be Interested In

16. Some LinkedIn products (Agenda)
JYMBII
– Recommend opportunities to talent
TalentMatch
– Recommend talent to opportunities
Job Search
– Let talent find opportunities
People Search
– Let opportunities find talent
Bonus: Economic Graph
LinkedIn Confidential ©2013 All Rights Reserved 16

17. 17
Real Time Talent Match

18. Job Seeker Intent Model
• Propensity Score
o Indicates receptiveness to new opportunities
o p(switch jobs in next time period)
• Model
o Survival Analysis of Positions
o Accelerated failure time (AFT) model
log Ti = Σkβkxik+σεi
ACTIVE
PASSIVE
NON-JOB-
SEEKER

19. Connecting Talent to Opportunity
Job Seeking Propensity
Review of Survival Analysis
 is the time of death/event/purchase
 is the survival time
 Probability density distribution of event
 Survival function
 Hazards function

20. Job-Seeker
Feature
Example:
Industry
Attrition
Probability
Time

21. Tenure-based recommendations
 1 million job applications over 5 years as training data
– Different job transitions happen at different times
– Transition to a higher position usually takes longer time
– Various factors affect transition time
– It matches with our motivation
Connecting Talent to Opportunity
Job Seeking Propensity
time time

22. Job-Seeking Intent:
actives & passives
16x reply rate on
career-related mail
Reply
Rate
Increase TalentMatch Utility
fn(booking rate, email rate, reply rate)
Connecting Talent to Opportunity
Multi-Objective Optimization

23. Talent Match ranking
Match Score
1, Item X, 0.98, Non-Seeker
2, Item Y, 0.91, Non-Seeker
---------------------------------------
3, Item Z, 0.89, Active
Perturbed ranking
Match Score, Perturbed Score
1, Item X, 0.98, 0.98, Non-Seeker
2, Item Z, 0.89, 0.93, Active
------------------------------------------------
3, Item Y, 0.91, 0.91, Non-Seeker
Perturbation
Function f()
Divergence
Function Δ()
Divergence
score
Objective
Function g()
Objective
score
Match Score
Distributions
Connecting Talent to Opportunity
MOO
How: Controlled Perturbation

24.  Perturbation Function
 Divergence Function
 Objective Function
Connecting Talent to Opportunity
MOO

25.  Loss Function
 Objective and divergence depend on a sort/rank,
so gradient-based optimization not directly
applicable
 Lambda value?
Connecting Talent to Opportunity
MOO

26. Pareto
Optimization
Connecting Talent to Opportunity
MOO

27. 0 27
54 100
Match Score Histogram Divergence
Connecting Talent to Opportunity
MOO

28. Experiments
 A/B Test
– Treatment 1: 1.15 boost (8/12)
– Treatment 2: 1.07 boost (6/12)
– Control: 1.0 boost (4/12)
 Expectations
– 50% increase in reply rate for 1.07 boost
– 100% increase in reply rate for 1.15 boost
– Expected booking rate and email rate to remain
unchanged or minimally affected
Connecting Talent to Opportunity
MOO

29. Booking rate
α = β = 1.07 0%
α = β = 1.15 -0.4%
Email rate
α = β = 1.07 31%
α = β = 1.15 25%
Reply rate
α = β = 1.07 22%
α = β = 1.15 42%
Connecting Talent to Opportunity
MOO

30. Some LinkedIn products (Agenda)
JYMBII
– Recommend opportunities to talent
TalentMatch
– Recommend talent to opportunities
Job Search
– Let talent find opportunities
People Search
– Let opportunities find talent
Bonus: Economic Graph
LinkedIn Confidential ©2013 All Rights Reserved 30

31. LinkedIn Data (Endorsements)

32. ©2013 LinkedIn Corporation. All Rights Reserved. 32
LinkedIn Data
?
Human Evaluation
Crowd LinkedIn’s Recruiters
Machine Learning
Rankings for every Skill
…
Product
Usage
Professional Reputation - Search

34. Some LinkedIn products (Agenda)
JYMBII
– Recommend opportunities to talent
TalentMatch
– Recommend talent to opportunities
Job Search
– Let talent find opportunities
People Search
– Let opportunities find talent
Bonus: Economic Graph
LinkedIn Confidential ©2013 All Rights Reserved 34

35. Economic graph
 Digitally map the global economy:people, jobs, skills, companies,
and schools.
 Spot in real-time the trends pointing to economic opportunities.
LinkedIn Confidential ©2013 All Rights Reserved 35

36. Economic graph
LinkedIn Confidential ©2013 All Rights Reserved 36

37. LinkedIn Confidential ©2013 All Rights Reserved 37

39. Credits
Engineering : Anmol Bhasin, Abhishek Gupta, Adam
Smyczek, Adil Aijaz, Alan Li, Baoshi Yan, Bee-Chung
Chen, Deepak Agarwal, Ethan Zhang, Haishan Liu, Igor
Perisic, Jonathan Traupman, Liang Zhang, Lokesh Bajaj,
Mario Rodriguez, Mitul Tiwari, Mohammad Amin, Monica
Rogati, Parul Jain, Paul Ogilvie, Sam Shah, Sanjay Dubey,
Tarun Kumar, Trevor Walker, Utku Irmak, Andrew Hill,
Christian Posse, Gyanda Sachdeva, Mike Grishaver,
Parker Barrile, Sachit Kamat, and many more…
Alphabetically sorted 

40. Contact:
mrodriguez@linkedin.com
http://data.linkedin.com/
http://engineering.linkedin.com/