Sprint's Next Generation Campaign Validation & Optimization
1. Next-Generation Campaign
Section Title
Validation & Optimization:
A Sprint Case Study
VictorSilva Andrea Molette
2. In 2007, comScore’s First Post-Buy Analysis Across 8 Digital US
Campaigns Showed Execution Left a Lot to be Desired …
70%
Percent of Ad Impressions for 8 Campaigns
60%
50%
40%
30% 61%
20%
10% 19%
8% 12%
0%
In US But Not Target Hit Target Hit Target Outside US
Frequency >=5 Frequency <=4
3. Branding advertisers on TV are accustomed to audience
guarantees and expect the same in digital
Accuracy of cookie-based digital plan delivery is
problematic:
Cookie Deletion Cookie Proliferation
X Cookies Are Not People
Source: comScore 2011
4. Some Things We’ve Learned About Digital
Media Plan Delivery
• The negative impact of cookie deletion
– Cookie deletion inflates ad frequency and deflates ad reach
by as much as a factor of 2.5X
• Targeting accuracy using cookies:
– 70% for 1 demo (e.g. women)
– 48% for 2 demos (e.g. women age 18-34)
– 11% for 3 demos (e.g. women age 18-34 with kids)
– 36% for behavioral targeting (e.g. people visiting travel sites)
5. Cookies Can’t Accurately Identify Who is Using a Computer at any
Given Point in Time due to Multiple Users
Over 64% of home users share a computer with other users
3+ users 1 user
32% of the36%
30%
time, someone other
than the Facebook
logged-in person is
actually using the
2 users
computer
32%
6. Cookie Deletion is a Global Reality
…and a Global Challenge
Accurately counting reach with cookies is not possible,
yet is currently the method used in most ad servers and analytics systems
Ad Server Cookies
Percent of Average # of cookies
Country computers deleting per computer for same
campaign
Australia 37% 5.7
Brazil 40% 6.6
U.K. 35% 5.9
U.S. 35% 5.4
7. Arguably the Most Important Digital Advertising Initiative To Date:
Making Measurement Make Sense (3MS) Mission
Reduce costs of doing business due to complexity of
digital advertising ecosystem
‘Single Tag’ solution to reduce complexity
Improve reporting of ad exposure
Bolster confidence that ads delivered are actually
visible
8. What is a vGRP?
• validated Gross Rating Points (GRP) based on validated
impressions delivered to the Total Census Population for selected
Geographic Market, or 100 * % Pop Reach * Average Frequency
for the reporting period
• Comparable to GRPs used in television because it used the same
calculations
• vGRPs must:
– Deliver in the target geography
– Not be fraudulent deliveries
– Be in brand safe content
– Be viewable
– Hit the target audience
11. vCE US Charter Study:
12 Major Branded Advertisers Came Together to Lead & Learn
18 campaigns
2 billion impressions
400,000 sites
Allstate
12. Charter Study replicated in Europe with similar results
15 campaigns
640 million impressions
213,000 sites
13. Study Objective
Quantify incidence of sub-optimal ad delivery across key
ad delivery dimensions…
… to better understand sources of waste and identify
opportunities to extract more value for all players in the online
advertising ecosystem
14. Importantly, all impressions in the study were
delivered in iframes, including the notoriously difficult-
to-measure cross-domain iframes
Directory on site Same-domain iframe Cross-domain iframe
(friendly) (un-friendly)
ad ad ad
web web web
site site site
61+% of
iframed ads
use this
method
15. Across all campaigns, the average in-view rate was
69%, meaning 3 out of 10 ads weren’t seen
Percentage of Ads In-View by Campaign
16. vCE Charter Study:
In-View Rates Need to Be Improved
US EU
69% 67%
AVERAGE AVERAGE
Campaign In-View ad rates ranged from:
US 55% to 93% EU 64% to 72%
17. In-view rates by site ranged from 7% to
100%, suggesting the importance of validation across all
sites
Percentage of Ads In-View by Site
18. Large sites scored better than long-tail sites
Percentage of Ads Served In-View
77% 74% 70%
61%
66% 63% 61%
55%
US EU
19. The Classic Leaderboard delivered the strongest in-view rates but
there was significant variance across all sites with a range of
7% to 93% using this size
Percent of Ads Delivered In-View by Ad Size
one potential
cause?
the relationship between ad
sizes and their typical
placement on a web page
20. Digital Ad Economics:
The Good Guys Aren’t Necessarily Winning
Low correlation of In-View Rates & CPM
R²=0.0373
An equally as weak correlation was also observed between CPM and ability to
hit a primary demographic target
21. On a campaign-by-campaign basis, one performed
flawlessly, while another delivered 15% of its impressions to
the wrong geography
Percent of Ads Delivered In Geography by Campaign
Sub-optimal geographic delivery is often a result of communication or human
error, and it can be remedied
with in-flight alerting and blocking
22. Of those ads delivered outside of their target, a good
portion of them landed in countries where English is
not even the primary language
% of Ads Delivered to Geographic Market Among All Impressions Delivered Outside of N.A.
23. Brand safety should be of the utmost importance to advertisers.
Even one poorly delivered ad can leave a very bad impression
Percent of Campaigns with Impressions Delivered
Next to Content Deemed “Not Brand Safe”
26. There is good news for advertisers and publishers
Analogous to TV audience guarantees
Eliminating unseen online inventory supply results
in:
More effective / efficient campaigns and less waste for
advertisers
More accurate metrics for market mix models
Better proof of digital ad effectiveness
Increased transparency/accountability means
increased confidence in digital
To get some market reaction, we introduced a charter program of some really large advertisers.We were delighted by the response to the program, and let me take the opportunity to say to them, Thanks you for participatingNow let us talk about what we learned.
Prior to the introduction of vCE, there hasn’t been a proven standard technology in the marketplace that could measure all three types of iframes to get a complete read of visibilityMeasuring the first two – directly on the site iframes and friendly iframes is relatively easy, and there have been solutions for these. However, unfriendly iframes, or cross domain iframes, are hard to measure in terms of location on the screen. Unfortunately, a large percent of ads are delivered this way, meaning that missing this piece of the equation, means not covering the entire story. vCE accounts for ALL THREE providing insight into total visibility.
Difference of in-view rates between Top 50 sites and long tail sites in their category was a full 16-percentage points in the USEuropean Results:Top 50 66%Top 100 63%Top 500 61%501+ 55% The only minor difference with the US is that we’ve taken the full range of sites rather than just a category We had half the publisher volume we saw in the US, and a lot have very small numbers of impressions which, when visible, skew upwards the long tail visibility average. The overall data set tells the best story and is most consistent with the original US slide.
To clarify, being in-view has nothing to do with being above or below the fold. In fact, there are some myths the charter study helped to debunk. For some campaigns less than half of the above the fold placements where actually seen.
On the other hand some placements below the fold where in almost 70% of the time.