Efficient Approximate Thompson Sampling for Search Query Recommendation in SAC'15
•Als PPTX, PDF herunterladen•
4 gefällt mir•664 views
This document discusses using Thompson sampling for search query recommendation. It introduces the multi-arm bandit problem and how Thompson sampling can be applied to solve it. The key aspects covered are: 1) Thompson sampling frames query recommendation as a multi-arm bandit problem to balance exploring new queries and exploiting popular ones. 2) It models the success probability of each query as a beta distribution and randomly selects queries based on these distributions to decide the next query to recommend. 3) An experiment on real search log data tested Thompson sampling for query recommendation with different numbers of queries to identify, showing it can quickly find the most popular queries.
Empfohlen
Weitere ähnliche Inhalte
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Empfohlen
Empfohlen (20)
Efficient Approximate Thompson Sampling for Search Query Recommendation in SAC'15
- 1. Efficient Approximate Thompson Sampling for Search Query Recommendation Chu-Cheng Hsieh 1
- 5. 5 Before vs. After Xmas
- 6. 6 Multi-arm bandit Problem (MAB) Thompson Sampling Query recommendation Experiments
- 7. The k-arm Bandit Problem 7 A B C
- 8. 8 Play the right slot machine that maximize your profit #Goal#
- 9. 9 1. 30% (play each evenly) 2. 70% play the best one #Simple Strategy#
- 10. 10 Related Search => MAB problem (M=1) A B C D E
- 11. 11 Multi-arm bandit Problem Thompson (TS) Sampling Query recommendation Experiments
- 13. x = random.random() #0<=x<1 y = 0.49 if (x < y): return true else: return false (Observed) 13 (Estimate) I played 10 times -- win 5 and lose 5 I played 100 times -- win 45 and lose 55 #Learn# Chanceofhavingμ μ ``prior’’
- 14. 14 Data Observed Action (Machine) (Refienments being displayed) Reward (Win/Loss) (CTR, BBOWA, ...)
- 15. 15 #Question# Learn or Earn? Red 19 win 9 lose Blue 59 win 39 lose
- 16. 16 Thompson Sampling Selecting a candidate based on the following formula: Assuming:
- 17. 0.0 0.2 0.4 0.6 0.8 1.0 0.60.81.01.21.4 x PDF 17 Beta(0+1,0+1)
- 18. 0.0 0.2 0.4 0.6 0.8 1.0 0.00.51.01.52.02.5 x PDF 18 Beta(5+1,5+1)
- 19. 0.0 0.2 0.4 0.6 0.8 1.0 02468 x PDF 19 Beta(45+1,55+1)
- 20. 20 #Example# Learn or Earn? Red 19 win 9 lose Blue 59 win 39 lose 75% 25%
- 21. 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF The motivation of Thompson-S (2) 21 Beta(20,10) Beta(60,40) See a good one; “learn more”
- 22. 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF Intuition (Underdog, but worth to learn) 22 Beta(4,6) Beta(60,40)
- 23. 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF The motivation of Thompson-S (1) 23 Beta(10,15) Beta(60,40) avoid exploring “low potential” arm early on
- 24. Intuition (Equal exploration) 24 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF 0.0 0.2 0.4 0.6 0.8 1.0 02468 PDF Beta(40,60) Beta(60,40)
- 25. 25 Init: a=1, b=1, Sx=Fx=0 for all x each arm corresponds to Beta(Sx+a, Fx+b) prior 1. Draw a random number from each arm based on Beta(Sx+a, Fx+b) 2. Play the arm (x’) with the highest number 3. If (see a reward) Sx’ += 1 else Fx’ += 1 Algorithm
- 26. 26
- 27. 27 Multi-arm bandit Problem Thompson Sampling Query Recommendation Experiments
- 28. 28 Ugly truth #1 M > 1 (M=5 here)
- 29. 29
- 30. Ugly truth # 2 No response => failure ?
- 31. 31
- 32. 32 Multi-arm bandit Problem Thompson Sampling Query Recommendation Experiments
- 33. 33 #Experiments# Target: popular 100 (queries) Date: 2 weeks (Nov. 2013) Goal: identify top M Measurement: Regret -> Best -> Picked
- 34. 34 M=1M=2 #Goal# Identify top M quickly.
- 36. 36 M=2, γ=0~2
- 38. 38 Hsieh, C. Neufeld, J. Holloway King, T. Cho, J.J. Efficient Approximate Thompson Sampling for Search Query Recommendation The 30th ACM/SIGAPP Symposium On Applied Computing (SAC 2015) About the author & download: http://oak.cs.ucla.edu/~chucheng/
Hinweis der Redaktion
- Small training period? => low recall
- One related keyword is allowed
- (2) We have more items than we can displayed.
- Find an old problem so that you can stand on the shoulder of giants
- In each time, you can pick a slot machine to play. You are given choice of selecting which slot machine to play.
- You are assuming info you learn from 30% is reliable
- One related keyword is allowed
- 用Slot Machine當例子，暫時別用Related Search William R. Thompson
- “Learn” is more intersting, earn is simple You learn from observing “rewriting”
- 用80%當example You don’t know the code. You only observe what happens after. So you need to make assumption. The simple assumption is to assume the prior is “normal distribution”
- a_x: which\;machine\;is\;played r: 1\;if\;win;\;otherwise,\;0
- beta is conjugate prior of binomial likelihood function \documentclass[10pt]{article} \usepackage[usenames]{color} %used for font color \usepackage{amsfonts} \usepackage{amsmath} \usepackage{amssymb} \usepackage{bm} \usepackage[utf8]{inputenc} %useful to type directly diacritic characters \newcommand{\defined}{\stackrel{{\rm def}}{=}}
- Beta(1,1) is uniform. If you randomly draw a number from Beta, the chance you see 0 or 0.5 or 1 are the same.
- When alpha and beta is large enough, it’s a bell curve (normal distribution)
- 75.46%
- 10.9%
- 3.5%
- 0.188%
- TS is a strategy of MAP
- Jello
- Multiple choice to dispaly
- TS is a strategy of MAP
- M=1, how quickly we can find the best?
- Gamma = “when no response, how much discount you would like to apply to penalty (beta)”?