Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Running Stan in Production: Bayesian Revenue Estimation
1. Running Stan in
Production: Bayesian
Revenue Estimation
2018-08-29 StanCon
Markus Ojala
PhD, Chief Data Scientist
Smartly.io
@MarkusOjala
2. Implemented in early 2017:
https://www.smartly.io/blog/tutorial-how-we-productized-bay
esian-revenue-estimation-with-stan
Referred by Andrew Gelman in early 2018:
http://andrewgelman.com/2018/01/21/smartly-io-productize
d-bayesian-revenue-estimation-stan/
“It’s BDA come to life!“
“This sort of thing is exactly what we were hoping to see.”
Blog posts
3. 1 billion spend
Managed yearly
We Make Online Advertising
Easy, Effective, And
Enjoyable
Facebook &
Instagram Partner
7. The number of pulls for a given lever should match
its actual probability of being the optimal lever
Sample from the posterior for the mean of each lever
Bayesian Bandits / Thompson sampling
9. Separate revenue model into two parts
ROAS = revenue / cost
= revenue / conversions * conversions / cost
= revenue / conversions * 1 / CPA
ROAS = return on ad spend
CPA = cost per action
Existing model
● Lot of data
● Varies fast
● Big differences
New model
● Little data
● Varies slowly
● Small real differences
● Lot of random variation
10. ● Use hierarchical model: Account -> Campaign -> Ad set
● Revenue follows usually a long-tailed distribution, use log-normal
● We observe only hourly aggregates: approximate by using
Fenton-Wilkinson log-normal approximation
Modeling the revenue per purchase
16. ● Easy way to write Bayesian models and do inference
● But hard to get stable in production: limit the scope of the model
● Reparametrize, use reasonable informative priors and custom initialization
○ Solves most of the convergence issues
● Sampling does not scale: we use ADVI / variational inference approximation
○ Validate that you get same results with sampling and ADVI
○ Random initialization issues were solved by fixing ADVI parameters
● About 1000 daily runs and $1M scheduled with Celery, PyStan, monitoring
Learnings in productizing Stan
21. We observe only aggregates
Goal: estimate log-normal parameters for ad sets
Challenge: observation i is aggregate of multiple events ni
Solution: Estimate by another log-normal