Weitere ähnliche Inhalte Kürzlich hochgeladen (20) Attribution Management Forum 3.0: How To Build Accurate Models To Solve Attribution1. Attribution Management Forum 3.0:
How To Build Accurate Models
To Solve Attribution
Tuesday May 5, 2009
1 PM EDT
Speakers: Adam Goldberg,
Dr. PurushPapatla
©2009 Third Door Media, Inc.
2. Viewing Tips
Turn Off Pop-Up Blockers
Technical difficulties?
Click on “Help?” link
Use Q+A box
Submitting questions to speaker
Q+A session at end of webcast
Use “Ask a Question” box to submit
questions
Send questions at any time
©2009 Third Door Media, Inc.
3. Adam Goldberg,
Chief Innovation Officer
Co-Founded ClearSaleing Inc. in 2006
Columbus, OH
Started Google’s inside sales organization in
2003-2006
New York City, NY
Started Actuate’s inside sales organization in
2000-2003
San Francisco, CA
Worked for Oracle Corp. in 1998-2000 in
Major Account Sales
Redwood Shores, CA
Speaker and Trainer at events such as:
(SMX), (SES), (DMA)
©2009 Third Door Media, Inc.
4. Dr. PurushPapatla
Ph.D. from Kellogg School of Management at
Northwestern University
Associate Professor, Marketing
Sheldon B. Lubar School of Business
President and Founder; Vetra Analytics
Published in top-tier marketing journals
» Marketing Science
» Journal of Marketing Research
» Journal of Business Research
» Journal of Retailing
» Journal of Interactive Marketing
©2009 Third Door Media, Inc.
5. EVOLUTION OF ONLINE ADVERTISING
Number of Online Offline Attribution Portfolio
Clicks Conversions Conversions Management Management
©2009 Third Door Media, Inc.
10. Poll Question 1
• Currently, how are you
attributing conversion credit to
your various ad sources?
1. Last click
2. Other attribution method
©2009 Third Door Media, Inc.
11. Recap of Modeling Framework From
“Measuring the Immeasurable”
www.AttributionManagement.com
©2009 Third Door Media, Inc.
13. DECISION INFLUENCER
What we know What we don’t know yet
Our Communications Communications
┼Competitor
Search search
┼Paid ┼Consumer
U
┼Banner Ads Site visits to competitors
┼
n
Product trials
┼e-mail ┼
c
Promotions …….
┼Onsite ┼
┼Comparison Shopping e
┼Affiliate ad
r
sources
┼Other
Social Media t
┼
Consumer Search
Word of mouth
┼
a
┼Organic search
Opinion sites
┼
i
┼Site visits to us
Expert opinions
┼
n
Traditional Mass Media
┼
t
y
©2009 Third Door Media, Inc.
14. MODELING CONSUMER DECISIONS
Build a mathematical model to predict consumer decisions
┼
Using data on influencers that we are able to track and measure
┼
Representing data on influencers that we can’t yet track and measure - our
┼
uncertainty - through a statistical distribution
Calibrate the model on observed consumer decisions
┼
Purchase - yes/no
┼
Purchase size - dollar volume, # of units
┼
Repeat purchases
┼
Word of mouth
┼
Etc.
┼
Test the model’s quality by comparing predicted and actual behavior
┼
©2009 Third Door Media, Inc.
15. CONSUMER DECISION MODEL
Consumer’s Decision = f (Our Communications, Consumer Search,
Competitor Communications, Other Sources)
= f ([Paid Search, Banner Ads, e-mail,
Onsite Promotions, Comparison
Shopping, Affiliate ads],
[Organic search, Site visits to us],
[uncertainty])
©2009 Third Door Media, Inc.
16. MEASURING THE EFFECTS OF KNOWN FACTORS?
We assume that each of the known
influencers
has an influence potential
©2009 Third Door Media, Inc.
17. MATHEMATICAL MODEL FOR CONSUMER’S
DECISION
* The β’s are the attribution weights
©2009 Third Door Media, Inc.
18. GETTING THE ATTRIBUTIONS
We calibrate the model on data from the
ClearSaleingplatform
The data includes but is not limited to:
Purchase Path™ data
Record of consumer’s decisions
• Purchase/non-purchase
•Product(s) purchased
• Amount spent
• Repeat visits and purchases
©2009 Third Door Media, Inc.
19. GETTING THE ATTRIBUTIONS
Calibrate the model on the ClearSaleing data
• Find the values of β’s which will help us predict
consumer decisions as accurately as possible
Model is calibrated using:
•Maximum Likelihood
•Bayesian methods
Theβ’s are the attribution weights!
©2009 Third Door Media, Inc.
20. MODELING THE INFLUENCE POTENTIAL
Influence potentialof an influencer = f (# of exposures,
when each of the exposures occurred,
decay rate of the effect of exposures)
©2009 Third Door Media, Inc.
21. Poll Question 2
• What challenges have you run into when trying to build an
attribution model?
1. Our technology cannot track beyond the last ad clicked
2. We cannot build a sound mathematical model
3. We cannot incorporate offline, social media, and word of
mouth advertising
4. All of the above
5. We haven’t tried to build an attribution model
©2009 Third Door Media, Inc.
22. PROGRESS SINCE LAST WEBINAR
1. Selection of businesses for the first round
of model testing
2. Identification of unique influencers
3. Set up the data for calibrating and testing
the model
4. Calibrate and test multiple versions of
the model
©2009 Third Door Media, Inc.
23. SELECTING BUSINESSES
Selecting businesses that:
•Have a high level of ad spend
•Wide array of advertising sources (paid search,
email, banner, etc)
•We have a least 6 month of data
We have 2+ years of data in some cases
•Seasonal variations
©2009 Third Door Media, Inc.
24. SELECTED BUSINESS VERTICALS
We will be developing and testing the model on nine
businesses in the following verticals
• Retail – web only
• Retail – multi-channel
• Insurance
• Financial Services
©2009 Third Door Media, Inc.
25. PROGRESS SINCE LAST WEBINAR
Identification of unique influencers
©2009 Third Door Media, Inc.
26. INFLUENCER CATEGORIES
We organized the influencers into the following categories:
•Direct
• Organic Referrers (e.g., Google)
• Paid Search
• Comparison Shopping
•e-mail
• Display advertising
• Affiliate
• Social Media
• Video
©2009 Third Door Media, Inc.
27. UNIQUE INFLUENCERS
Each category was further sub-categorized into a number of
unique influencers:
•Direct - 1
• Organic Referrers – 11 (Google, Yahoo, MSN, etc)
• Paid Search Engine – 11 (Ex: Brand vs. Non-Brand)
• Comparison Shopping – 3 (Ex: Model Number vs. Product Name)
• e-mail - 3 (Ex: Direct Response vs. Brand)
• Display advertising - 4
• Affiliate - 2
• Social Media - 1
• Video - 1
©2009 Third Door Media, Inc.
28. OVERALL
We have:
• 9 categories of influencers
•37 types of unique influencers across the nine
categories
Our model develops attributions for these 37 unique
influencers across the nine businesses.
©2009 Third Door Media, Inc.
29. PROGRESS SINCE LAST WEBINAR
Set up the data for calibrating
and testing the model
©2009 Third Door Media, Inc.
30. PROGRESS SINCE LAST WEBINAR
135 predictors
64,653 Purchase Paths™
•11,353 paths resulting in a purchase
•53,300 abandoned paths that did not end in a purchase
oA path was defined as abandoned based on some
proprietary criteria
Model can explain abandonment too
Another frontier: Attributions for abandonment
©2009 Third Door Media, Inc.
31. RESULTS TO DATE
To date, we have calibrated over 70 versions of the model
We plan to calibrate and test the model at least 500 more
times in various forms before firming up our
conclusions
•45,000 models run across the nine data sets
Testing
• Do the estimated attributions make intuitive sense?
• Is the model able to predict consumer behavior?
o Can it predict purchases?
o Can it predict non-purchases?
©2009 Third Door Media, Inc.
32. RESULTS TO DATE
Intuitive assessment of attributions
• Findings: not yet firmed up since we have 37 unique
influencers to assess across hundreds of model runs
Predictive Testing
• 85% or more of the purchases being predicted correctly
• 95% or more of the non-purchases predicted correctly
• Lift charts for calibration and prediction samples
o Top decile indices average between 450 and 500
©2009 Third Door Media, Inc.
33. TYPES OF NON-CLICK/
PASSIVE INFLUENCERS
Consumer ratings and reviews
Social networks
Blogging
Social commerce
Instant messaging
Twitter
You Tube
RSS and multiple feeds
©2009 Third Door Media, Inc.
34. ATTRIBUTIONS FOR NON-CLICK INFLUENCERS
Why do we need to?
• Benefits of including non-click influencers in
attribution models
• Risks of not-including non-click influencers in
attribution models
Challenges
• How do we include them, if they don’t click and we
don’t track??
©2009 Third Door Media, Inc.
35. VETRA PASSIVE SURVEY™
Use survey data on the use of non-click influencers
• Statistically infer the likelihood of the use of each type
of passive influencers by different demographic,
lifestyle and psychographic segments
•Vetra is currently working on this approach
oVetra Passive Survey™
•Vetra Passive Survey™ can be used to include passive
influencers in attribution models
©2009 Third Door Media, Inc.
36. VETRA PASSIVE PROPORTIONS™
Monitor sources of passive influence
• YouTube
•Facebook
•Myspace
• epinions.com
Statistically infer the proportions of buyers who engage in
discussions and exchanges regarding products
•Vetra has completed preliminary work on a model for this
inference
Vetra Passive Proportions™ can also be used to include
passive influencers in attribution models
©2009 Third Door Media, Inc.
37. ONGOING RESEARCH ON ATTRIBUTION
AND NEXT STEPS
Vetra and ClearSaleingwill:
•Continue to test a number of attribution models and
influencers
• Analyze the performance of models across different
verticals
• Identify the best attribution models for different verticals
• Expand attribution models to include passive influencers using
oVetra Passive Survey™
oVetra Passive Proportions™
©2009 Third Door Media, Inc.
38. Poll Question 3
• What is your timeframe in switching from a
last click model to an advanced attribution
model?
1. Less than 6 months
2. Within a year
3. Within 2 years
4. More than 2 years
5. No timeframe
©2009 Third Door Media, Inc.
40. QUESTIONS?
Adam Goldberg- www.attributionmanagement.com
info@attributionmanagement.com
www.ClearSaleing.com
www.searchmarketingnow.com
webcasts@searchmarketingnow.com
Upcoming SMN webcast:May12: Ask iProspect: Strategies &Tactics
for World-Class SEO
May 14: Search Marketing for Small Business:
The Basics for Online Success
©2009 Third Door Media, Inc.