Do More with Less: Navigating Customer Acquisition Challenges for Today's Ent...
Â
2015 Bot Baseline Report - White Ops & ANA
1. ANA / White Ops
2015 Ad Fraud Study
and 2016 Threat Models
2. In 2015, We Found:
• Bots are getting caught, eventually, but they make most of their money in the “profit
window”
• Sourced traffic and ad injection are still threatening advertisers and publishers
• Hispanic targeting and other targeting increases bot exposure
• The estimated loss in 2015 to bot fraud for the average participant was $10 million
• The threat models for mobile fraud are something to watch closely in 2016
• Awareness of ad fraud has improved among advertisers, but effective action is still rare
• Technologies that detect fraud are necessary, but not sufficient, to lower the bot rate;
advertisers also need rigorous policies to reduce the impact of ad fraud in their media
2
In 2015, White Ops and the ANA found:
4. The range of bots was 3 to 37% in 2015 compared
to 2 to 22% in 2014
4
General bots are detectable
using the industry spiders and
bots list, while sophisticated
bots require more complex
techniques to detect.
The overall bot rate did not
budge much, but bot rates
shifted among participants in
2015 (top) and 2014 (bottom).
5. Sourced traffic and ad injection still threaten
advertisers and publishers
5
Sourced traffic (at right)
contained more than
three times the bots of
unsourced traffic.
A case study of a single
publisher found that ad
injection generated 6%
of their total impressions.
6. Hispanic targeting increases bots
6
Programmatic
Hispanic-targeted
media had 70%
higher bot rates than
non-targeted media.
Direct buy Hispanic-
targeted media had
20% higher bot rates
than non-targeted
direct media.
7. Programmatic buys had higher bot rates
7
Direct Display media had 2-40%
bots with 14% lower bots than
average.
Programmatic video media had
1-70% bots with 73% higher bots.
Programmatic display media had
2-30% bot rates with 14% higher
bots on average.
The small amount of direct video
media that was measurable had
59% lower bot rates than average.
8. Re-targeting increases bots
8
Bots are able to infiltrate
retargeting segments and
reap the higher CPMs
advertisers pay to reach
them.
An advertiser’s re-targeting
campaigns drove bots to its
own e-commerce site at up
to 12 times the rate of bots
in their non-retargeted
campaigns.
9. The majority of bots come from residential internet
addresses
9
In 2015, small number of
residences accounted for a
significant amount of the bot
traffic that originates from
Residential IPs.
10. How does ad fraud continue to be a problem?
10
I.
12. 12
If you are…
AND there is malware
on your computer…
Logged into Facebook, checking
Gmail, buying items on Amazon…
13. 13
If you are…
The malware is also doing all
of those things... as you.
AND there is malware
on your computer…
Logged into Facebook, checking
Gmail, buying items on Amazon…
14. Thanks to your cookies…
14
Your malware clone is a bona fide,
targetable consumer.
When the malware runs a browser in
the background, it becomes a valuable
website visitor.
Authentication by requiring cookies
does not mean authentic visitors.
15. 15
The entire ad ecosystem implicitly trusts
the client endpoint, relying on persistent
identifiers.
Usually the identifier is a cookie, but
anything tied to the device –device
IDs, browser fingerprints, anything – is
readable by the malware, too, and is
therefore vulnerable.
16. 16
This undermines a basic,
pervasive assumption that if,
for instance, you know a user
bought something, you can be
certain that, when you serve
that user an ad, you’re
definitely serving a human.
17. 17
That's why digital ad fraud is such
a thorny problem, even for
platforms with massive amounts of
first-party identity data.
18. Kerkhoff’s Principle: The Enemy knows the system
Here's how our adversaries have overcome all the
defenses in place
II.
18
19. Bot detection is
19
not a Turing Test.
Bots successfully mimic human
browsers, and their operators
reverse engineer detection systems.
20. uses two forms of mimicry:
20
Acting human by copying the behaviors of the owner of
the computer (example: much better diurnal patterns)
Copying the traffic between lots of real human browsers
and the fraud detection services to learn the right answers
The Adversary
21. More bot operators are keeping human daytime
hours
21
The regular pattern
of computer use — with
most computers off at
night — is likely
responsible for bots
mimicking a normal
human’s waking hours.
22. Bots are still fooling Viewability measures
22
The average viewable
rate of sophisticated bot
traffic is 43 percent,
closely mimicking the
average human viewable
rate of 47 percent.
23. is reverse engineering the detection thresholds
23
Bot operators do A/B testing just like the good guys
By segmenting a botnet into parts and seeing which
ones get blocked (real-time oracle) or seeing which
ones pay out (slow oracle).
The Adversary
25. Botnets make money in the “profit window”
between newly infecting a computer and getting caught.
And publishers can buy bot traffic that they can be certain
won't get caught
25
III.
26. Bots on infected machines are a moving target for
advertisers
26
The newest bots on
newly infected machines
are unknown to general
blocking mechanisms.
Blacklisting these bots is
not possible without
using evidence-based
sophisticated detection
methods.
27. Monetization of the profit window emerges from
natural market forces
27
The platforms and services that broker traffic use the same
services that advertisers use, to only sell “the good stuff.”
29. Bots in the early part of the profit window
affect the most expensive media
29
Video media with over
$15 CPM had 173%
higher bot rates than
lower-CPM media
Display media with
over $10 CPM had
39% higher bot rates
than lower-CPM media
30. Estimated annual bot impacts in 2015 ranged from
$250,000 to $42 million
30
The estimated average
annual loss to bots
among ANA 2015
study participants was
$10 million.
31. Bots shifted among prominent exchanges and
platforms
31
Ad tech platforms which
purged bots from their
supplies were not able to
purge the most expensive
bots that are in the profit
window unless they were
using “sophisticated”
detection and prevention.
33. Being aware and involved reduces fraud exposure
33
One participant relied on
their agency and list-
lookup-based prevention
to eliminate bots and had
32% bots in their media,
while the other participant
successfully reduced fraud
to 3% by carefully
selecting providers and
looking into where their
providers’ audiences
came from.
34. Our survey showed that awareness of ad fraud
has improved
34
Last year, we often encountered
surprise that ad fraud was a
problem.
This year, 43 percent of study
participants stated that either all
parties or the advertiser themselves
should be responsible for
combatting ad fraud.
35. In 2015, advertisers with the lowest cost of
fraud:
35
Used legal language to remove fraud
during the billing stage
Leveraged the watchdog effect by
announcing anti-fraud policies to
partners
Required transparency about traffic
sourcing
Combined sophisticated anti-fraud
technology with anti-fraud policies to
reduce fraud at all levels
36. 36
• Authorize and approve third-party traffic validation technology
• Require clarity from vendors on how they combat fraud
• Protect against fraud that Is in the profit window
• Use sophisticated fraud detection to block bots in programmatic media
• Follow MRC guidelines for invalid traffic detection and filtration
• Support the Trustworthy Accountability Group
Recommendations for all stakeholders
37. 37
• Be aware and involved
• Equip your organization to fight ad fraud: budget for security
• Request transparency for sourced traffic and audience extension practices
• Include language on non-human traffic in Terms and Conditions
• Use third-party monitoring
• Use frequently updated blacklists
• Announce your anti-fraud policy to all external partners
• Involve procurement
Recommendations for media buyers
38. 38
• Continuously Monitor Sourced Traffic
• Purge the Fraud; Increase Your Prices
• Protect Yourself from Content Theft and Ad Injection
• Allow Third-Party Traffic Assessment Tools
Recommendations for publishers,
platforms, and exchanges