Cross-Device Tracking Performance and Challenges
•
3 gefällt mir•383 views
1. The document describes Adform's efforts to perform cross-device tracking by linking cookies and devices that belong to the same user. 2. Adform analyzes over 1.5 billion events per day from deterministic sources and bid responses to build a cross-device graph and cluster cookies into user profiles. 3. A gradient boosted decision tree classifier is used to classify cookie pairs as belonging to the same user or different users based on features like location, device type, and behavior over time.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Cross-Device Tracking Performance and Challenges
Ähnlich wie Cross-Device Tracking Performance and Challenges (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Cross-Device Tracking Performance and Challenges
- 1. Cross-Device Tracking September 2017 Morten Arngren Lead Data Scientist Vlad Sandulescu Senior Data Scientist Jan Kremer Data Scientist
- 2. a lot of experimentation 📷 Snapshot Still under development
- 4. 👻 time 📱 📞💻 Cookie Churn Cross-Device
- 8. ❓Ground Truth? External Deterministic Providers ⏲ LocationTimeCookie DeviceOS 📞 📈data
- 9. (Deterministic) >700K events/day 1.5B events/day ⏲ LocationTimeCookie DeviceOS 📞 (Adform bid responses)
- 10. 1.5B events/day ⏲ LocationTimeCookie DeviceOS 📞 94% 74K Filtering Remove cookies with multiple clusters clusters Cookie Churn Cross-Device 6% Data set may not be representative. (Deterministic) >700K events/day 2M events 1 Months of data (Adform bid responses)
- 11. 300M 300M Training Validation time More of it… 2.236.222 events 4.776.974 events 2017-01-21 2017-02-21 2017-03-10 2017-04-27
- 12. Cross-Device Graph Cookie User-AgentIP Device Cluster cookies into user profiles. ⌖ 👽 👻 Data ⏲ 📞 Location Time Cookie Device OS
- 13. Cross-Device Graph Cookie User-AgentIP Device B nodes B edges Graph cutting algorithms • Min-Cut • Markov Random Fields • Community Detection • Graph Clustering… Production Scale B events
- 14. Cookie IP (Pre-Clustering) Divide Graph Ignored 1 2 3 n-1 n (Connected Components Algorithm) …by location 4 Scale….what to do??
- 15. by Pair-wise classification 👽👻 A B C D …with ground truth A B C D Positive Negative A C A D B C B D Create cookie pairs as classification data set E A E B E C E D E Cookie association
- 16. cookie_id Location Device device_type_id os_id browser_id browser_language_codes mobile_app_id ip_v4 country_id region_id city_id zip_code_id log_time ID Time Crafting Features Observations for each cookie pair… A BA B Aggregation 0 12 24 hours Pageviews 0 31 days set a set b
- 17. Crafting Features for each cookie pair… Aggregation Jaccard Index Cosine Similarity Correlation Similarity Hellinger Distance Max Overlap Similarity cookie_id Location Device device_type_id os_id browser_id browser_language_codes mobile_app_id ip_v4 country_id region_id city_id zip_code_id log_time ID Time Observations A B 0 12 24 hours Pageviews 0 31 days set a set b Features
- 18. Crafting Features for each cookie pair… cookie_id Location Device device_type_id os_id browser_id browser_language_codes mobile_app_id ip_v4 country_id region_id city_id zip_code_id log_time ID Time Observations A B Features (http://arogozhnikov.github.io/2016/06/24/gradient_boosting_explained.html) 🌴 0.78 A B Classifier Gradient Boosted Decision Trees 🌴 🌴 PC 1+2 Vectorial Sets Numerical 25 dim. feature vector
- 19. A C B D E F G A B C D E F G Classification A B C DE F G Clustering (Connected Components Algorithm) Pruning Degrees of edges 1 3 2222 2 95% percentile
- 20. Classification Clustering (Connected Components Algorithm) Pruning A C B D E F G A B C D E F G A B C DE F G Pre-Clustering (Connected Components Algorithm) Full Graph Components Sub-Graph A C B D E F G Clusters Users (XGBoost)
- 21. 📈performance
- 23. PerformanceGraphs (PreliminaryYouGovdataset…) (Dasgupta et al. “Overcomming Browser Cookie Churn with Clustering” 2012)
- 25. PerformanceGraphs (Preliminary YouGov data set…) (PreliminaryYouGovdataset…) 91% 32% Precision Recall F1-score 47%
- 26. PerformanceGraphs (PreliminaryYouGovdataset…) (Preliminary YouGov data set…) 93% 43% Precision Recall F1-score 59%
- 27. production
- 28. Scheduled tasks Training Train Classifier Model parameters Pre- Processing Pre- Clustering Predict Cross Device ID’s Cross-Device Cookies Ready XD Deterministic Data Pre- Processing Pre- Clustering XD Data Model ready S3 Kafka Predicting Serving Serve… S3 Kafka( ( ))
- 29. 300M 300M 175M 138M 210M time More of it… 2017-05-01 2017-05-08 2017-06-25 2017-07-24 7 days ⏲🌐 Germany 1.3B events (10%) 30 days ⏲🌐 Germany 56B events
- 30. time More of it… 2017-05-01 2017-05-08 7 days ⏲🌐 Germany 1.3B events (10%) 30 days ⏲🌐 Germany Predicting Model parameters Pre- Processing Pre- Clustering Predict Cross Device ID’s Cross-Device Cookies Ready XD Data 30 min. 2 hours 2017-06-25 2017-07-24 56B events
- 31. ☢challenges
- 32. One IP to rule them all…. Large pre-clusters 176.22.252.??? IP Obscurity Cookie IP Noisy pre-clusters
- 33. ⏲future
- 34. Predicting Model parameters Pre- Processing Pre- Clustering Predict Cross Device ID’s Cross-Device Cookies Ready XD Data Streaming Topic modelling on website content Data 📈 Features More of it… ContextMore… Contextual Information Validate… ✈ 📹 Large pre-clusters IP Obscurity
- 35. Tran et al. “Classification and Learning-to-rank Approaches for Cross-Device Matching” CIKM Cup 2016 Dasgupta et al. ”Overcoming Browser Cookie Churn with Clustering” CIKM Cup 2016 Blondel et al. “Fast unfolding of communities in large networks” 2008 Malloy et al. “Internet Device Graphs” KDD 2017 [1] [2] [3] [4] References
- 36. thank you
- 37. Profile-cookie space Precision / Recall Estimated FN Class A Class B TN TP FP Recall = TP TP + FP Precision = = TP TP + FN = F1 = Precision • Recall 2 • Precision + Recall
- 38. (http://arogozhnikov.github.io/2016/06/24/gradient_boosting_explained.html) >1 <=1 <0.5 >=0.5 <=0 >0 <=6<0.8 >=0.8 >6 >1 <=1 + + Boosted Decision Trees 🌴 🌲 12172 12410 3 F1 F4 F3 F8 F5 F2 1 + 0.78 A B