This talk details patterns and processes at LinkedIn for building real-time machine learning systems to provide our members with actionable, relevant opportunities to nurture their professional networks. Featuring the Connected mobile app as an in-depth case study of how to combine compute-intensive features describing billions of relationships with information that isn’t known until the moment a user opens the app, in this talk we’ll discuss the architectural, modeling, and experimentation patterns leveraged by the Connected relevance team to rank and serve mobile content. Additionally, this session will touch on the human element of machine learning product development, outlining collaboration and communication patterns for working effectively across the organization – from reporting and documentation to evangelism, skills transfer and user experience research. Taken together, these insights will provide a detailed picture of some of LinkedIn’s best practices for building data products at a global scale.

1. Real Time Relevance for Mobile
Mike Conover, Ph.D.
Senior Data Scientist, LinkedIn
Friday, February 20, 15

2. People Organizations Jobs Skills Education Knowledge
Friday, February 20, 15

3. Sample Corporate Network
Friday, February 20, 15

4. How do industries evolve in time?
Which companies are positioned for explosive growth?
Which jobs will employ the workforce in ten years?
Industrial
Structure
Environmental Consulting
Friday, February 20, 15

5. Technology
Entrepreneurship
2013 Connection Graph
lnkd.in/report
Friday, February 20, 15

6. Crunchbase Entrepreneurial Demographics
lnkd.in/report
Friday, February 20, 15

7. “Experience is the teacher of all things.”
Julius Caesar
lnkd.in/report
Friday, February 20, 15

8. lnkd.in/report
Predicting
VC Investment
Friday, February 20, 15

9. Network Intelligence
Connected
Friday, February 20, 15

10. The Strength of Weak Ties
Fundamental Value Proposition
Zero Marginal Cost
New Information
Friday, February 20, 15

11. Relationship
Opportunities
Job Changes Work Anniversaries
Birthdays Content Creation
Recent Meeting
Friday, February 20, 15

12. Friday, February 20, 15

13. Prioritization Exercise
Not Every Event is Equally Relevant
Target Market
Friday, February 20, 15

14. Mobile
Dimensions
Accuracy
Push Notifications
Implement and Iterate
Friday, February 20, 15

15. AAABABBCBCCC
Heuristic Ranking + Time Decay
The Power
of Intuition
Friday, February 20, 15

16. The Power
of Intuition
“In god we trust, all others bring data.”
Friday, February 20, 15

17. Relevant Dimensions
Framework Must Support
Personalization
Demographics & Engagement
Relationship Quality
Sensor Data
Profile Similarity
Card Features
Friday, February 20, 15

18. Computational Aspects
Offline Online
Personalization
Relationship Quality
Profile Similarity
Sensor Data
Impression Counts
Demographics
Card Features
Friday, February 20, 15

19. βi (Offline Features + Online Features)
Architectural Diagram
Friday, February 20, 15

20. Experimentation
OBSCURE
LinkedIn PlatoXLNT
Friday, February 20, 15

21. Network Intelligence
Connected
Friday, February 20, 15

22. Casidhe
Lee
Mathieu
Bastian
Akhilesh
Gupta
Vinodh
Jayaram
David
Brubacher
Friday, February 20, 15

23. Real Time Relevance for Mobile
Mike Conover, Ph.D.
Senior Data Scientist, LinkedIn
Friday, February 20, 15

24. Gatekeeper
Default Landing Experience
Friday, February 20, 15

25. Impressions @ Position
CTR @ Position
20x Dropoff
80% Avg
Raw Materials
Principled
Decision Making
Weighing Tradeoffs
Friday, February 20, 15

26. Tracking Data
0,0 0,1 0,2
1,0
2,0
1,1
2,1
1,2
2,2
Friday, February 20, 15