Suche senden
Hochladen
Sidi chang demo
•
0 gefällt mir
•
84 views
Sidi Chang
Folgen
Insight Data Science
Weniger lesen
Mehr lesen
Ingenieurwesen
Melden
Teilen
Melden
Teilen
1 von 15
Jetzt herunterladen
Downloaden Sie, um offline zu lesen
Empfohlen
Probabilistic Data Structures and Approximate Solutions
Probabilistic Data Structures and Approximate Solutions
Oleksandr Pryymak
The Very ^ 2 Basics of R
The Very ^ 2 Basics of R
Winston Chen
R statistics with mongo db
R statistics with mongo db
MongoDB
Tech talk Probabilistic Data Structure
Tech talk Probabilistic Data Structure
Rishabh Dugar
From Trill to Quill: Pushing the Envelope of Functionality and Scale
From Trill to Quill: Pushing the Envelope of Functionality and Scale
Badrish Chandramouli
Sidi chang week_4.3
Sidi chang week_4.3
Sidi Chang
Ppt pkn endah
Ppt pkn endah
Tri_Endah_Sulistiani
Ppt Tri Endah
Ppt Tri Endah
Tri_Endah_Sulistiani
Empfohlen
Probabilistic Data Structures and Approximate Solutions
Probabilistic Data Structures and Approximate Solutions
Oleksandr Pryymak
The Very ^ 2 Basics of R
The Very ^ 2 Basics of R
Winston Chen
R statistics with mongo db
R statistics with mongo db
MongoDB
Tech talk Probabilistic Data Structure
Tech talk Probabilistic Data Structure
Rishabh Dugar
From Trill to Quill: Pushing the Envelope of Functionality and Scale
From Trill to Quill: Pushing the Envelope of Functionality and Scale
Badrish Chandramouli
Sidi chang week_4.3
Sidi chang week_4.3
Sidi Chang
Ppt pkn endah
Ppt pkn endah
Tri_Endah_Sulistiani
Ppt Tri Endah
Ppt Tri Endah
Tri_Endah_Sulistiani
El estado colombiano
El estado colombiano
camilo charris
IMI_ebook_GuidetoInfluencerMarketing
IMI_ebook_GuidetoInfluencerMarketing
Kellen Dieterich
Base de datos
Base de datos
Yessica Yuliana Montealegre Amado
Creative commons
Creative commons
etevago lopez dofus
Riesgos laborales según las normas convenin
Riesgos laborales según las normas convenin
johalmy
Teori pendekatan gestalt
Teori pendekatan gestalt
Tri_Endah_Sulistiani
Suresh BIM HVAC Portfolio.
Suresh BIM HVAC Portfolio.
Suresh Babu G
Renata Salátová: Chcete vést časopis? Aneb slasti a strasti redaktorské práce
Renata Salátová: Chcete vést časopis? Aneb slasti a strasti redaktorské práce
ÚISK FF UK
Vít Richter: Veřejné knihovny a jejich prostor. Výsledky celostátního průzkum...
Vít Richter: Veřejné knihovny a jejich prostor. Výsledky celostátního průzkum...
ÚISK FF UK
Lukáš Kolek: Jak se vyvíjejí výukové simulace
Lukáš Kolek: Jak se vyvíjejí výukové simulace
ÚISK FF UK
Jakub Fiala: Quantified Self
Jakub Fiala: Quantified Self
ÚISK FF UK
Ralf Herbrich - Introduction to Graphical models in Industry
Ralf Herbrich - Introduction to Graphical models in Industry
Bayes Nets meetup London
Large scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log mining
itstuff
IEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slides
Nish Parikh
Big data
Big data
canara engineering college
Big data
Big data
Harshit Namdev
Hadoop PDF
Hadoop PDF
1904saikrishna
Skillwise Big data
Skillwise Big data
Skillwise Group
Machine Learning and AI: Core Methods and Applications
Machine Learning and AI: Core Methods and Applications
QuantUniversity
Big data
Big data
Zeeshan Khan
Outlier and fraud detection using Hadoop
Outlier and fraud detection using Hadoop
Pranab Ghosh
IBANK - Big data www.ibank.uk.com 07474222079
IBANK - Big data www.ibank.uk.com 07474222079
ibankuk
Weitere ähnliche Inhalte
Andere mochten auch
El estado colombiano
El estado colombiano
camilo charris
IMI_ebook_GuidetoInfluencerMarketing
IMI_ebook_GuidetoInfluencerMarketing
Kellen Dieterich
Base de datos
Base de datos
Yessica Yuliana Montealegre Amado
Creative commons
Creative commons
etevago lopez dofus
Riesgos laborales según las normas convenin
Riesgos laborales según las normas convenin
johalmy
Teori pendekatan gestalt
Teori pendekatan gestalt
Tri_Endah_Sulistiani
Suresh BIM HVAC Portfolio.
Suresh BIM HVAC Portfolio.
Suresh Babu G
Renata Salátová: Chcete vést časopis? Aneb slasti a strasti redaktorské práce
Renata Salátová: Chcete vést časopis? Aneb slasti a strasti redaktorské práce
ÚISK FF UK
Vít Richter: Veřejné knihovny a jejich prostor. Výsledky celostátního průzkum...
Vít Richter: Veřejné knihovny a jejich prostor. Výsledky celostátního průzkum...
ÚISK FF UK
Lukáš Kolek: Jak se vyvíjejí výukové simulace
Lukáš Kolek: Jak se vyvíjejí výukové simulace
ÚISK FF UK
Jakub Fiala: Quantified Self
Jakub Fiala: Quantified Self
ÚISK FF UK
Andere mochten auch
(11)
El estado colombiano
El estado colombiano
IMI_ebook_GuidetoInfluencerMarketing
IMI_ebook_GuidetoInfluencerMarketing
Base de datos
Base de datos
Creative commons
Creative commons
Riesgos laborales según las normas convenin
Riesgos laborales según las normas convenin
Teori pendekatan gestalt
Teori pendekatan gestalt
Suresh BIM HVAC Portfolio.
Suresh BIM HVAC Portfolio.
Renata Salátová: Chcete vést časopis? Aneb slasti a strasti redaktorské práce
Renata Salátová: Chcete vést časopis? Aneb slasti a strasti redaktorské práce
Vít Richter: Veřejné knihovny a jejich prostor. Výsledky celostátního průzkum...
Vít Richter: Veřejné knihovny a jejich prostor. Výsledky celostátního průzkum...
Lukáš Kolek: Jak se vyvíjejí výukové simulace
Lukáš Kolek: Jak se vyvíjejí výukové simulace
Jakub Fiala: Quantified Self
Jakub Fiala: Quantified Self
Ähnlich wie Sidi chang demo
Ralf Herbrich - Introduction to Graphical models in Industry
Ralf Herbrich - Introduction to Graphical models in Industry
Bayes Nets meetup London
Large scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log mining
itstuff
IEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slides
Nish Parikh
Big data
Big data
canara engineering college
Big data
Big data
Harshit Namdev
Hadoop PDF
Hadoop PDF
1904saikrishna
Skillwise Big data
Skillwise Big data
Skillwise Group
Machine Learning and AI: Core Methods and Applications
Machine Learning and AI: Core Methods and Applications
QuantUniversity
Big data
Big data
Zeeshan Khan
Outlier and fraud detection using Hadoop
Outlier and fraud detection using Hadoop
Pranab Ghosh
IBANK - Big data www.ibank.uk.com 07474222079
IBANK - Big data www.ibank.uk.com 07474222079
ibankuk
Machine Learning Crash Course by Sebastian Raschka
Machine Learning Crash Course by Sebastian Raschka
PawanJayarathna1
Approximate "Now" is Better Than Accurate "Later"
Approximate "Now" is Better Than Accurate "Later"
NUS-ISS
Chapter 6. Mining Frequent Patterns, Associations and Correlations Basic Conc...
Chapter 6. Mining Frequent Patterns, Associations and Correlations Basic Conc...
Subrata Kumer Paul
Applied Stochastic Processes, Chaos Modeling, and Probabilistic Properties of...
Applied Stochastic Processes, Chaos Modeling, and Probabilistic Properties of...
e2wi67sy4816pahn
Big data serving: Processing and inference at scale in real time
Big data serving: Processing and inference at scale in real time
Itai Yaffe
Clickstream data with spark
Clickstream data with spark
Marissa Saunders
Summit EU Machine Learning
Summit EU Machine Learning
MapR Technologies
Automated Hyperparameter Tuning, Scaling and Tracking
Automated Hyperparameter Tuning, Scaling and Tracking
Databricks
Big data
Big data
Mohammad Reza Gerami
Ähnlich wie Sidi chang demo
(20)
Ralf Herbrich - Introduction to Graphical models in Industry
Ralf Herbrich - Introduction to Graphical models in Industry
Large scale Click-streaming and tranaction log mining
Large scale Click-streaming and tranaction log mining
IEEE.BigData.Tutorial.2.slides
IEEE.BigData.Tutorial.2.slides
Big data
Big data
Big data
Big data
Hadoop PDF
Hadoop PDF
Skillwise Big data
Skillwise Big data
Machine Learning and AI: Core Methods and Applications
Machine Learning and AI: Core Methods and Applications
Big data
Big data
Outlier and fraud detection using Hadoop
Outlier and fraud detection using Hadoop
IBANK - Big data www.ibank.uk.com 07474222079
IBANK - Big data www.ibank.uk.com 07474222079
Machine Learning Crash Course by Sebastian Raschka
Machine Learning Crash Course by Sebastian Raschka
Approximate "Now" is Better Than Accurate "Later"
Approximate "Now" is Better Than Accurate "Later"
Chapter 6. Mining Frequent Patterns, Associations and Correlations Basic Conc...
Chapter 6. Mining Frequent Patterns, Associations and Correlations Basic Conc...
Applied Stochastic Processes, Chaos Modeling, and Probabilistic Properties of...
Applied Stochastic Processes, Chaos Modeling, and Probabilistic Properties of...
Big data serving: Processing and inference at scale in real time
Big data serving: Processing and inference at scale in real time
Clickstream data with spark
Clickstream data with spark
Summit EU Machine Learning
Summit EU Machine Learning
Automated Hyperparameter Tuning, Scaling and Tracking
Automated Hyperparameter Tuning, Scaling and Tracking
Big data
Big data
Kürzlich hochgeladen
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
9953056974 Low Rate Call Girls In Saket, Delhi NCR
IVE Industry Focused Event - Defence Sector 2024
IVE Industry Focused Event - Defence Sector 2024
Mark Billinghurst
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
wendy cai
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
ssuser2ae721
An experimental study in using natural admixture as an alternative for chemic...
An experimental study in using natural admixture as an alternative for chemic...
Chandu841456
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
Asst.prof M.Gokilavani
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptx
KartikeyaDwivedi3
young call girls in Rajiv Chowk🔝 9953056974 🔝 Delhi escort Service
young call girls in Rajiv Chowk🔝 9953056974 🔝 Delhi escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
POWER SYSTEMS-1 Complete notes examples
POWER SYSTEMS-1 Complete notes examples
Dr. Gudipudi Nageswara Rao
Application of Residue Theorem to evaluate real integrations.pptx
Application of Residue Theorem to evaluate real integrations.pptx
959SahilShah
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
srsj9000
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
Asst.prof M.Gokilavani
8251 universal synchronous asynchronous receiver transmitter
8251 universal synchronous asynchronous receiver transmitter
ShivangiSharma879191
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
jennyeacort
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
null - The Open Security Community
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
Dr SOUNDIRARAJ N
Arduino_CSE ece ppt for working and principal of arduino.ppt
Arduino_CSE ece ppt for working and principal of arduino.ppt
SAURABHKUMAR892774
Call Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile service
rehmti665
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
NikhilNagaraju
Kürzlich hochgeladen
(20)
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
IVE Industry Focused Event - Defence Sector 2024
IVE Industry Focused Event - Defence Sector 2024
What are the advantages and disadvantages of membrane structures.pptx
What are the advantages and disadvantages of membrane structures.pptx
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
Why does (not) Kafka need fsync: Eliminating tail latency spikes caused by fsync
An experimental study in using natural admixture as an alternative for chemic...
An experimental study in using natural admixture as an alternative for chemic...
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptx
young call girls in Rajiv Chowk🔝 9953056974 🔝 Delhi escort Service
young call girls in Rajiv Chowk🔝 9953056974 🔝 Delhi escort Service
POWER SYSTEMS-1 Complete notes examples
POWER SYSTEMS-1 Complete notes examples
Application of Residue Theorem to evaluate real integrations.pptx
Application of Residue Theorem to evaluate real integrations.pptx
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
8251 universal synchronous asynchronous receiver transmitter
8251 universal synchronous asynchronous receiver transmitter
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
Gurgaon ✡️9711147426✨Call In girls Gurgaon Sector 51 escort service
young call girls in Green Park🔝 9953056974 🔝 escort Service
young call girls in Green Park🔝 9953056974 🔝 escort Service
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
Exploring_Network_Security_with_JA3_by_Rakesh Seal.pptx
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
UNIT III ANALOG ELECTRONICS (BASIC ELECTRONICS)
Arduino_CSE ece ppt for working and principal of arduino.ppt
Arduino_CSE ece ppt for working and principal of arduino.ppt
Call Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile service
main PPT.pptx of girls hostel security using rfid
main PPT.pptx of girls hostel security using rfid
Sidi chang demo
1.
Sidi Chang Insight Data
Science Data Engineering Fellow Jul 2016 JustBid
2.
Sealed/blind second price
auction Item Bidder
3.
• Demo
4.
Data Pipeline Simulated Data
5.
Data • 10K bidders •
Nearly 15 million bidding
6.
Recommendation—Jaccard Similarity Jaccard Similarity: D_i
= user_i C_i = items(user_i)
7.
Recommendation For 𝑵 = 𝟏𝟎 million, it takes more than a year(AWS m4.large cluster)… Then we will need to use minHash Algorithm which can be easily distributed… Do an unbiased estimation by Chernoff Bounds and Markov Inequality: The expected error is
8.
MinHash Example Item Row
User 1 User 2 User 3 User 4 x+1 mod 5 3x+1 mod 5 1 0 1 0 0 1 1 1 2 1 0 0 1 0 2 4 3 2 0 1 0 1 3 2 4 3 1 0 1 1 4 0 5 4 0 0 1 0 0 3 U1 U2 U3 U4 Hash 1 Hash 2
9.
MinHash Example Item Row
User 1 User 2 User 3 User 4 x+1 mod 5 3x+1 mod 5 1 0 1 0 0 1 1 1 2 1 0 0 1 0 2 4 3 2 0 1 0 1 3 2 4 3 1 0 1 1 4 0 5 4 0 0 1 0 0 3 U1 U2 U3 U4 Hash 1 Hash 2
10.
MinHash Example Item Row
User 1 User 2 User 3 User 4 x+1 mod 5 3x+1 mod 5 1 0 1 0 0 1 1 1 2 1 0 0 1 0 2 4 3 2 0 1 0 1 3 2 4 3 1 0 1 1 4 0 5 4 0 0 1 0 0 3 U1 U2 U3 U4 Hash 1 Hash 2
11.
MinHash Example Item Row
User 1 User 2 User 3 User 4 x+1 mod 5 3x+1 mod 5 1 0 1 0 0 1 1 1 2 1 0 0 1 0 2 4 3 2 0 1 0 1 3 2 4 3 1 0 1 1 4 0 5 4 0 0 1 0 0 3 U1 U2 U3 U4 Hash 1 1 Hash 2
12.
MinHash Example Item Row
User 1 User 2 User 3 User 4 x+1 mod 5 3x+1 mod 5 1 0 1 0 0 1 1 1 2 1 0 0 1 0 2 4 3 2 0 1 0 1 3 2 4 3 1 0 1 1 4 0 5 4 0 0 1 0 0 3 U1 U2 U3 U4 Hash 1 1 3 0 1 Hash 2 0 2 0 0
13.
Performance
14.
Challenges • MinHash Algorithm
implemented in distributed system • Jaccard Similarity Tested in distributed system • Use right data structures to faster computation • Use both Scala and Python
15.
About me • MS
in CS and Operations Research
Jetzt herunterladen