Presented internally at Yahoo

1. Executive Summary
Campaign Overview
• Campaign Goals & Key Business Metrics
• Operational Details of the Campaign
Key Results
• Competitive Landscape & Evolution of the Campaign Strategy
• Week-by-week Performance: Aggregate | Publisher-wise | Customer Segment-
wise | Platform (Device)-wise | Geography-wise
Future Recommendations
• Actionable Recommendation 1…(Segmentation)
• Actionable Recommendation 2…(Allocation)
• Actionable Recommendation 3…(Optimization)
Learning Component
• Learning Objectives
• Learning Outcomes
• Published White Paper/s (if any)
3/11/2015 1

2. Executive Summary
Campaign Overview
• Campaign Goals & Key Business Metrics
• Operational Details of the Campaign
Key Results
• Competitive Landscape & Evolution of the Campaign Strategy
• Week-by-week Performance: Aggregate | Publisher-wise | Customer Segment-
wise | Platform (Device)-wise | Geography-wise
Future Recommendations
• Actionable Recommendation 1…(Segmentation)
• Actionable Recommendation 2…(Allocation)
• Actionable Recommendation 3…(Optimization)
Learning Component
• Learning Objectives
• Learning Outcomes
• Published White Paper/s (if any)
3/11/2015 2

3. Campaign Goals: Business Needs & Success Metrics
Goal
#
Strategic Goals of the Campaign
(SMART: specific, measurable, attainable,
relevant & time-bound)
Key Metrics to be Measured in the
Campaign Window
1. To attract new customers Change in the percentage of new visitors to
advertiser’s website
2. To share business value proposition Visitor Loyalty: Repeat Visits by number
3. To persuade people to buy more Increment in the average order size
4. To drive offline action Likelihood to recommend or likelihood to
make an offline purchase (Captured through
quick on-exit surveys)
5. To introduce a new business or a new
product
# of leads received | # of requests for catalogs
| # of coupons printed | # of free downloads
6. To destroy competition Percentage share of voice stolen from
competitors | Traffic differentials
7. Brand Embossing Measuring ‘Rising Searches’ for the campaign
keywords/brands
Choosing the Metrics
3/11/2015 3

4. Metrics Should Cover the Entire Purchase Funnel
(Helps in conducting the ‘end-to-end’ deeper analysis irrespective of the
campaign or conversion goals)
ACQUISITION
• Clicks (Entrances)
• Unique Visitors
BEHAVIOR
• Bounces
• Page Views
• Average time on page
OUTCOME
• Per visit goal value
• Total goal completions
3/11/2015 4

5. 3/11/2015 5

6. Additional Metrics for Ideal Post Campaign Report
Compliance Analysis
• Ensuring all purchased
impressions are actually
delivered by analyzing eye-
tracking data (Fired pixels
vs. viewed by real people;
Invalid user & fraud
detection)
Attribution Analysis
(Online<->offline, across multiple screens,
across digital channels)
• Quantifying cross-channel
impact using ‘carryover’
and ‘spillover’ of visits *
(Going beyond first click, last
click, even distribution, and time
decay models)
• Identifying the most and
least common paths to
conversion
* Attribution Modeling: Understanding the Influence of Channels in the Online Purchase Funnel, Marketing Science Institute,
Hongshuang (Alice) Li and P. K. Kannan, 2012 [12-115]
3/11/2015 6

7. 3/11/2015 7
Identifying the ‘Path to Conversion’

8. Operational Details of the Campaign/s
Campaign
# & Title
Campaign
Window
(Start &
End Date)
Campaign
Timing
During
the Day
Display
Ad Size ,
Creative,
and
Position
Target
Audience
(Geography,
Demography,
Psychography
)
Publishers Platforms
or
Devices
Total
Spent
($)
Thank you
Mom!
Truthful
Confession!
Four
o'clock
flop!
Take a
load off!
3/11/2015 8

9. Executive Summary
Campaign Overview
• Campaign Goals & Key Business Metrics
• Operational Details of the Campaign
Key Results
• Competitive Landscape & Evolution of the Campaign Strategy
• Week-by-week Performance: Aggregate | Publisher-wise | Customer Segment-
wise | Platform (Device)-wise | Geography-wise
Future Recommendations
• Actionable Recommendation 1…(Segmentation)
• Actionable Recommendation 2…(Allocation)
• Actionable Recommendation 3…(Optimization)
Learning Component
• Learning Objectives
• Learning Outcomes
• Published White Paper/s (if any)
3/11/2015 9

10. Competitive Landscape & Evolution of the Campaign
Strategy
Insights from Controlled Experiments | Failure is not fatal, but failure to change might be…
• Ad creative & banner variations
• Audience segmentationWhat Worked?
• Campaign timing & smooth delivery during purchased
time period
• Relationship with publishers
What Did Not
Work?
• Identifying patterns in user receptiveness and creating
the right Ad schedule
• Negotiating with publishers & Reallocating impressions
Campaign
Optimization
• Optimized Ad cost and clear call-to-actions
• Higher ROI for the advertiser
Impact of Change
Made
3/11/2015
10

11. Summary of Key Results
► Aggregate | ► Publisher-wise | ► Customer-Segment-wise | ► Platform (Device)-wise
<For
each
Campaig
n>
#Impres
sions
#Clicks
#Conver
sions
View-
through
Rate
(VTR)
Click-
through
Rate
(CTR)
Conversi
on Rate
Task
Complet
ion Rate
Avg.
Cost per
‘000
Impressi
ons
(CPM)
Avg.
Cost per
Click
(CPC)
Avg.
Cost per
Acquisiti
on (CPA)
Return
on Ad
Dollars
spent
(ROA)
Week
#1
Week
#2
Week
#3
Week
#4
3/11/2015 11

12. Summary of Key Results
►Aggregate | ► Publisher-wise | ► Customer-Segment-wise | ► Platform (Device)-wise
)<For each
Campaign
>
Entrances Unique
Visitors
Bounces Page
Views
per Visit
Visit
Duration
Per Visit
Goal Value
(judgment
call)
Total Goal
Completio
ns
Week #1
Week #2
Week #3
Week #4
3/11/2015 12
Sample “Data Visualization Charts”

13. Data-Driven Visual Analysis : Graphs & Charts
Tell a story
through
ANNOTATED
GRAPHS!
Use of
Heat Maps
3/11/2015 13

14. Data-Driven Visual Analysis : Graphs & Charts
3/11/2015 14

15. 3/11/2015 15

16. 3/11/2015 16

17. Motion Chart
3/11/2015 17

18. 3/11/2015 18

19. Executive Summary
Campaign Overview
• Campaign Goals & Key Business Metrics
• Operational Details of the Campaign
Key Results
• Competitive Landscape & Evolution of the Campaign Strategy
• Week-by-week Performance: Aggregate | Publisher-wise | Customer Segment-
wise | Platform (Device)-wise | Geography-wise
Future Recommendations
• Actionable Recommendation 1…(Segmentation)
• Actionable Recommendation 2…(Allocation)
• Actionable Recommendation 3…(Optimization)
Learning Component
• Learning Objectives
• Learning Outcomes
• Published White Paper/s (if any)
3/11/2015 19

20. Recommendations for Future Campaigns
(SEGMENTATION | ATTRIBUTION | ALLOCATION | OPTIMIZATION)
• Ad formats should match both the form and
the function of the user experience in which it
is placed.
Ad Format & Media
Placement
• Good content for each behavioral pattern
should be established using multivariate
testing.
Ad Content & Creative
• Ads should be scheduled to show more often
during the morning and evening period when
there are more traffic and low CPC.
Ad Schedule
• Target audience should be optimized so that
campaign can go large scale without hurting
the performance.
Audience
Segmentation &
Campaign Optimization
3/11/2015 20

21. Executive Summary
Campaign Overview
• Campaign Goals & Key Business Metrics
• Operational Details of the Campaign
Key Results
• Competitive Landscape & Evolution of the Campaign Strategy
• Week-by-week Performance: Aggregate | Publisher-wise | Customer Segment-
wise | Platform (Device)-wise | Geography-wise
Future Recommendations
• Actionable Recommendation 1…(Segmentation)
• Actionable Recommendation 2…(Allocation)
• Actionable Recommendation 3…(Optimization)
Learning Component
• Learning Objectives
• Learning Outcomes
• Published White Paper/s (if any)
3/11/2015 21

22. Learning Objectives
We hoped to learn how to
increase Client’s
• Brand awareness
• Sales
• Customers
• Profits
Predictive Analytics
Translating online consumer behavior into
actionable intelligence is the foundation of
behavioral targeting.
Training Data:
Week 1,2 & 3 data
was used for
“Predictive
Model” building
Testing Data: The
model was tested
on Week 4 data.3/11/2015 22

23. Learning Outcomes
3/11/2015 23
White Papers
Our campaigns were very general and broad because we didn’t have historical
sales data. We could have been more effective if we had more historical sales
analytics from the outset.
We had limited access to Client’s website developer, which made customizations
to page content cumbersome. We could have increased our quality score and
lowered our CPC if we had the ability to make Client’s content more relevant
Retargeting with different personalized calls-to-action at different phases of
purchase funnel (i.e. research, selection, and shipping) is pivotal to greater
conversion.
We learnt that demographics, content, and time of day influence consumers’ cross-
screen behavior.

24. Improvement in prediction using
clicks in conjunction with
conversions.
Improvement in prediction accuracy by
using conversions for training instead of
clicks. Testing is done using conversions
in both cases. => Targeting users based
on clicks does not necessarily mean
maximizing for conversions.
3/11/2015 24

25. Longer user history leads to better
performance.
There is a trade-off between
recency and history length. While
the former might help in predicting
short-term conversions, the latter is
needed for long-term conversions.
• By setting Decay Factor to a larger constant, we can
give more weight to recent user activities in
comparison to the older ones.3/11/2015 25

26. We need to balance the precision and
recall in by selecting a proper number of
user segments for ads delivery.
How past exposure, along with
interest match, can help in significantly
improving the targeting
systems.
How can we do privacy-friendly
targeting?
How can we detect click frauds?
3/11/2015 26

27. 3/11/2015 27