ACM Internet Measurement Conference (IMC) Amsterdam, Netherlands, October 2019

1. IMC 2019: A Longitudinal Analysis of the ads.txt Standard
A Longitudinal Analysis
of
the ads.txt Standard
October 22, 2019
Muhammad Ahmad Bashir Sajjad Arshad Engin Kirda
William Robertson Christo Wilson

2. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 2
What is ads.txt Standard?
Introduced by Interactive Advertising Bureau (IAB)
to combat ‘domain spoofing’ fraud
in Real Time Bidding (RTB) auctions

3. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 3
Online Advertising
Website / Publisher Advertising & Analytics (A&A) Companies

4. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 3
Online Advertising
Website / Publisher Advertising & Analytics (A&A) Companies
Users’ Profile
Users’ Profile

5. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 3
Online Advertising
Website / Publisher Advertising & Analytics (A&A) Companies
Users’ Profile
Users’ Profile

6. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
Website / Publisher Advertising & Analytics (A&A) Companies
Observable via browser
Not observable

7. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
Website / Publisher Advertising & Analytics (A&A) Companies
Observable via browser
Not observable
Ad Exchange

8. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
RTB Auction
Website / Publisher Advertising & Analytics (A&A) Companies
Observable via browser
Not observable
Ad Exchange
Demand Side
Platform (DSPs)

9. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
RTB Auction
Website / Publisher Advertising & Analytics (A&A) Companies
Observable via browser
Not observable
Advertisers
Ad Exchange
Demand Side
Platform (DSPs)

10. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
$0.05
$0.10
$0.20
RTB Auction
Website / Publisher Advertising & Analytics (A&A) Companies
Observable via browser
Not observable
Advertisers
Ad Exchange
Demand Side
Platform (DSPs)

11. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
$0.05
$0.10
$0.20
RTB Auction
Website / Publisher Advertising & Analytics (A&A) Companies
Observable via browser
Not observable
Advertisers
Ad Exchange
Demand Side
Platform (DSPs)

12. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 4
Shift Towards Real Time Bidding (RTB)
$0.05
$0.10
$0.20
RTB Auction
Website / Publisher Advertising & Analytics (A&A) Companies
RTB currently holds 42% share of
US Digital Advertising.
Observable via browser
Not observable
Advertisers
Ad Exchange
Demand Side
Platform (DSPs)

13. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
UserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
UserX=123
Observable via browser
Not observable

14. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
UserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
UserX=123
Observable via browser
Not observable

15. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
UserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
GET, DoubleClick’s Cookie = 123
UserX=123
Observable via browser
Not observable

16. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
RTB AuctionUserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
GET, DoubleClick’s Cookie = 123
UserX=123
Observable via browser
Not observable

17. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
RTB AuctionUserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
GET, DoubleClick’s Cookie = 123
UserX=123
Bid for User 123 ?
Bid for User 123 ?
Observable via browser
Not observable

18. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
RTB AuctionUserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
GET, DoubleClick’s Cookie = 123
UserX=123 $0.15
$0.40
Bid for User 123 ?
Bid for User 123 ?
Observable via browser
Not observable

19. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
RTB AuctionUserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
GET, DoubleClick’s Cookie = 123
UserX=123 $0.15
$0.40
Bid for User 123 ?
Bid for User 123 ?
GET, RightMedia’s Cookie = ABC
Observable via browser
Not observable

20. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 5
An Example of RTB Auction
RTB AuctionUserX Publisher Ad Exchange
UserX=xo$
UserX=ABC
Demand Side
Platforms (DSPs)
GET, CNN’s Cookie
GET, DoubleClick’s Cookie = 123
UserX=123 $0.15
$0.40
Bid for User 123 ?
Bid for User 123 ?
DSPs don’t receive the
request directly from
the browser.
GET, RightMedia’s Cookie = ABC
Observable via browser
Not observable

21. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 6
Potential for Fraud in RTB Auctions
Observable via browser
Not observable
Publishers Ad Exchanges

22. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 6
Potential for Fraud in RTB Auctions
Observable via browser
Not observable
Publishers Ad Exchanges DSPs (Bidders)

23. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 6
Potential for Fraud in RTB Auctions
Observable via browser
Not observable
Publishers Ad Exchanges DSPs (Bidders)

24. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 6
Potential for Fraud in RTB Auctions
Observable via browser
Not observable
Publishers Ad Exchanges DSPs (Bidders)

25. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 6
Potential for Fraud in RTB Auctions
Observable via browser
Not observable
Publishers Ad Exchanges DSPs (Bidders)
DSPs will think the user
is on CNN, rather than
FNN

26. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 7
Ad Fraud = Loss in $$$

27. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 7
Ad Fraud = Loss in $$$

28. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 8
Programmatic Buying & Selling via ads.txt Standard
Authorized Digital Sellers (ads.txt)
• Introduced by Interactive Advertising Bureau (IAB) in May 2017
• Enables publishers to list authorized exchanges
• At the root of the top-level domain

29. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 8
Programmatic Buying & Selling via ads.txt Standard
Authorized Digital Sellers (ads.txt)
• Introduced by Interactive Advertising Bureau (IAB) in May 2017
• Enables publishers to list authorized exchanges
• At the root of the top-level domain
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt

30. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 8
Programmatic Buying & Selling via ads.txt Standard
Authorized Digital Sellers (ads.txt)
• Introduced by Interactive Advertising Bureau (IAB) in May 2017
• Enables publishers to list authorized exchanges
• At the root of the top-level domain
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
File Format:
<FIELD #1>, <FIELD #2>, <FIELD #3>
#1: Exchange domain *
#2: Publisher’s account ID
#3: Type of account/relationship
* Not always an exchange

31. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 9
Potential for Fraud in RTB Auctions
Publishers Observable via browser
Not observable
Ad Exchanges DSPs (Bidders)

32. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 9
Potential for Fraud in RTB Auctions
Publishers Observable via browser
Not observable
Ad Exchanges DSPs (Bidders)
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt

33. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 10
Why Study ads.txt standard?

34. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 10
Why Study ads.txt standard?
Potential to combat domain spoofing fraud
• Check for any shortcomings.
• Lessons can be learnt for upcoming standards.

35. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 10
Why Study ads.txt standard?
Potential to combat domain spoofing fraud
• Check for any shortcomings.
• Lessons can be learnt for upcoming standards.
Might be useful for privacy researchers
• Means of bringing transparency in the advertising ecosystem
• Publishers are disclosing their relationships with ad exchanges

36. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 11
Goals / Research Questions
1. Are publishers adopting the ads.txt standard?
2. Are major A&A companies embracing the standard?
• Which A&A companies have adopted?
• Do these companies appear to be complying with the standard?
3. Can this data be useful for transparency of the ad ecosystem?

37. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 12
Talk Overview
1. RTB background & motivation for the ads.txt standard
2. Data collection
3. Adoption trends
4. Compliance of the ads.txt standard
5. Means of providing transparency

38. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology

39. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology
Alexa top-100K
publishers

40. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
Alexa top-100K
publishers
Using Python’s requests
module
Fetched /ads.txt URL

41. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
Alexa top-100K
publishers
Using Python’s requests
module
Fetched /ads.txt URL
Stored output
as base64

42. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
Alexa top-100K
publishers
Using Python’s requests
module
Fetched /ads.txt URL
Stored output
as base64
Crawled 15 URLs per
publisher using
Chrome Debugging

43. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
Alexa top-100K
publishers
Using Python’s requests
module
Fetched /ads.txt URL
Stored output
as base64
Crawled 15 URLs per
publisher using
Chrome Debugging
Stored
Inclusion Chains
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example Inclusions

44. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 13
Data Collection Methodology
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
Alexa top-100K
publishers
Using Python’s requests
module
Fetched /ads.txt URL
Stored output
as base64
• Collected 26 snapshots between January 2018 - April 2019
Crawled 15 URLs per
publisher using
Chrome Debugging
Stored
Inclusion Chains
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example Inclusions

45. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange

46. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange Increase: 860 —> 1400
Total unique exchanges: 2381

47. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
Mistakes in ads.txt files
Increase: 860 —> 1400
Total unique exchanges: 2381

48. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
Mistakes in ads.txt files
A. Syntax Errors
google.com pub−7439281311086140, DIRECT
openx.com , pub−7439281311086140
Increase: 860 —> 1400
Total unique exchanges: 2381

49. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
Mistakes in ads.txt files
A. Syntax Errors
google.com pub−7439281311086140, DIRECT
openx.com , pub−7439281311086140
B. Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381

50. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
Mistakes in ads.txt files
A. Syntax Errors
google.com pub−7439281311086140, DIRECT
openx.com , pub−7439281311086140
B. Semantic Errors
rubicnprject.com, 11078, DIRECT
Misspelled
Increase: 860 —> 1400
Total unique exchanges: 2381

51. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
Mistakes in ads.txt files
A. Syntax Errors
google.com pub−7439281311086140, DIRECT
openx.com , pub−7439281311086140
B. Semantic Errors
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com
Increase: 860 —> 1400
Total unique exchanges: 2381

52. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 14
Authorized Exchanges Over Time
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
Mistakes in ads.txt files
A. Syntax Errors
google.com pub−7439281311086140, DIRECT
openx.com , pub−7439281311086140
B. Semantic Errors
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com
Increase: 860 —> 1400
Total unique exchanges: 2381

53. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

54. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
Fetch WHOIS record
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

55. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
Fetch WHOIS record
Exists
1035 domains
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

56. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
Fetch WHOIS record
Exists
1035 domains
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

57. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
Fetch WHOIS record
Exists Not Found
1346 domains
Perform DNS lookup
1035 domains
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

58. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
Fetch WHOIS record
Exists Not Found
1346 domains
Perform DNS lookup
1035 domains
No
Resolution
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

59. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 15
Fixing Semantic Errors
Increase: 860 —> 1400
Total unique exchanges: 2381
Number of authorized exchange
domains over time
Fetch WHOIS record
Exists Not Found
1346 domains
Perform DNS lookup
1035 domains
No
Resolution
NumberofExchanges
All Exchanges
Valid Exchange
rubicnprject.com, 11078, DIRECT
doubleclick.net, 3159, DIRECT
Misspelled
Should be
google.com

60. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 16
Talk Overview
1. RTB background & motivation for the ads.txt standard
2. Data collection
3. Adoption trends
4. Compliance of the ads.txt standard
5. Means of providing transparency

61. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 17
How Have Publishers Adopted?
ads.txt adoption by Alexa Top-100K
publishers over time

62. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 17
How Have Publishers Adopted?
ads.txt adoption by Alexa Top-100K
publishers over time
Adoption has been modest
Increase: 12% —> 20%

63. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 17
How Have Publishers Adopted?
ads.txt adoption by Alexa Top-100K
publishers over time
Publishers that run RTB ads
Adoption is pretty decent. ~64%
Adoption has been modest
Increase: 12% —> 20%

64. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 18
Authorized Exchanges Discovered
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
1035 valid domains
Number of unique publishers per
authorized exchange domain

65. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 18
Authorized Exchanges Discovered
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
1035 valid domains
Number of unique publishers per
authorized exchange domain

66. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 18
Authorized Exchanges Discovered
Number of authorized exchange
domains over time
NumberofExchanges
All Exchanges
Valid Exchange
1035 valid domains
Number of unique publishers per
authorized exchange domain
Out of 1035 domains, 20% appear only
on a single ads.txt file

67. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 19
Adoption Trends are Encouraging
• 64% of the publishers from Alexa top-100K who run RTB ads.
• Major A&A domains support ads.txt

68. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 19
Adoption Trends are Encouraging
• 64% of the publishers from Alexa top-100K who run RTB ads.
• Major A&A domains support ads.txt

69. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 20
Talk Overview
1. RTB background & motivation for the ads.txt standard
2. Data collection
3. Adoption trends
4. Compliance of the ads.txt standard
5. Means of providing transparency

70. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 21
A Simple Check for Compliance
Publisher
Observable via browser
Not observableAd Exchange DSP (Bidder)

71. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 21
A Simple Check for Compliance
Publisher
Observable via browser
Not observableAd Exchange DSP (Bidder)
google.com, pub−787, DIRECT
rubiconproject.com, 11078, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt

72. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 21
A Simple Check for Compliance
Publisher
Observable via browser
Not observableAd Exchange DSP (Bidder)
google.com, pub−787, DIRECT
rubiconproject.com, 11078, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
✔

73. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 21
A Simple Check for Compliance
Publisher
Observable via browser
Not observableAd Exchange DSP (Bidder)
google.com, pub−787, DIRECT
rubiconproject.com, 11078, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
✔
google.com, pub−787, DIRECT
rubiconproject.com, 11078, DIRECT
https://cnn.com/ads.txt

74. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 21
A Simple Check for Compliance
Publisher
Observable via browser
Not observableAd Exchange DSP (Bidder)
google.com, pub−787, DIRECT
rubiconproject.com, 11078, DIRECT
openx.com, 183753,RESELLER
https://cnn.com/ads.txt
✔
google.com, pub−787, DIRECT
rubiconproject.com, 11078, DIRECT
https://cnn.com/ads.txt
✗

75. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 22
Checking Compliance at Scale
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example InclusionsAdvertisements
(Detected Using EasyList)

76. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 22
Checking Compliance at Scale
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example InclusionsAdvertisements
(Detected Using EasyList)

77. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 22
Checking Compliance at Scale
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example InclusionsAdvertisements
(Detected Using EasyList)

78. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 23
Checking Compliance at Scale
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example InclusionsAdvertisements
(Detected Using EasyList)

79. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 23
Checking Compliance at Scale
pub/
index.html
ads/
script.js
ads/
frame.html
logo.jpg
ads/
img_a.jpgadnet/
data.js
Example InclusionsAdvertisements
(Detected Using EasyList)
Chain must have at least 3 elements

80. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 24
Extracting Tuples For Compliance Analysis
Advertisements
(Detected Using EasyList)
Chain must have at least 3 elements

81. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 24
Extracting Tuples For Compliance Analysis
Advertisements
(Detected Using EasyList)
Chain must have at least 3 elements
Associated
Tuples

82. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 24
Extracting Tuples For Compliance Analysis
Advertisements
(Detected Using EasyList)
Chain must have at least 3 elements
Associated
Tuples

83. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 24
Extracting Tuples For Compliance Analysis
Advertisements
(Detected Using EasyList)
Chain must have at least 3 elements
Associated
Tuples

84. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 24
Extracting Tuples For Compliance Analysis
Advertisements
(Detected Using EasyList)
Chain must have at least 3 elements
Associated
Tuples

85. 25
Compliance Per Tuple
ads.txt ads.txt

86. 25
Compliance Per Tuple
ads.txt ads.txt
Percentage of non-compliant
Exchange-DSP tuples per publisher

87. 25
Compliance Per Tuple
ads.txt ads.txt
Percentage of non-compliant
Exchange-DSP tuples per publisher

88. 25
Compliance Per Tuple
ads.txt ads.txt
Percentage of non-compliant
Exchange-DSP tuples per publisher
75% publishers have 100%
compliant inventory

89. 25
Compliance Per Tuple
ads.txt ads.txt
Percentage of non-compliant
Exchange-DSP tuples per publisher
75% publishers have 100%
compliant inventory
Compliance over time

90. 25
Compliance Per Tuple
ads.txt ads.txt
Percentage of non-compliant
Exchange-DSP tuples per publisher
75% publishers have 100%
compliant inventory
Compliance over time
Compliance has increased
over the year

91. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 26
Top Non-Compliant Tuples
Exchange /
Seller
DSP /
Buyer
% Across
all pubs
gumgum domdex 20.4
gumgum appnexus 20.5
taboola weborama 52.7
taboola rubiconproject 11.6
gumgum pubmatic 27.3
justpremium openx 100
rubiconproject yahoo 2.63
springserve pubmatic 49.3
nativeroll weborama 100
nativeroll seedr 100

92. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 26
Top Non-Compliant Tuples
Exchange /
Seller
DSP /
Buyer
% Across
all pubs
gumgum domdex 20.4
gumgum appnexus 20.5
taboola weborama 52.7
taboola rubiconproject 11.6
gumgum pubmatic 27.3
justpremium openx 100
rubiconproject yahoo 2.63
springserve pubmatic 49.3
nativeroll weborama 100
nativeroll seedr 100
Sellers
• Some sellers are systematically
non-compliant (e.g. NativeRoll, JustPremium).
• Top exchanges do not appear as non-compliant.

93. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 26
Top Non-Compliant Tuples
Exchange /
Seller
DSP /
Buyer
% Across
all pubs
gumgum domdex 20.4
gumgum appnexus 20.5
taboola weborama 52.7
taboola rubiconproject 11.6
gumgum pubmatic 27.3
justpremium openx 100
rubiconproject yahoo 2.63
springserve pubmatic 49.3
nativeroll weborama 100
nativeroll seedr 100
Sellers
• Some sellers are systematically
non-compliant (e.g. NativeRoll, JustPremium).
• Top exchanges do not appear as non-compliant.

94. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 26
Top Non-Compliant Tuples
Exchange /
Seller
DSP /
Buyer
% Across
all pubs
gumgum domdex 20.4
gumgum appnexus 20.5
taboola weborama 52.7
taboola rubiconproject 11.6
gumgum pubmatic 27.3
justpremium openx 100
rubiconproject yahoo 2.63
springserve pubmatic 49.3
nativeroll weborama 100
nativeroll seedr 100
Sellers
• Some sellers are systematically
non-compliant (e.g. NativeRoll, JustPremium).
• Top exchanges do not appear as non-compliant.
Buyers
• Most are actually top exchanges (sellers).
• 8/10 top sellers are buying non-compliant
inventory.

95. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 27
Compliance of ads.txt Standard

96. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 27
Top exchanges and DSPs appear to be honoring the standard
Compliance of ads.txt Standard
More than 75% of publishers that run RTB have 100%
compliance
Major ad exchanges purchasing non-compliant inventory

97. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 28
Talk Overview
1. RTB background & motivation for the ads.txt standard
2. Data collection
3. Adoption trends
4. Compliance of the ads.txt standard
5. Means of providing transparency

98. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 29
Identification of A&A Domains
Efforts to identify Advertising and Analytics related domains.
• Filter lists (e.g. EasyList) are manually curated
• Machine learning based approaches

99. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 29
Identification of A&A Domains
Efforts to identify Advertising and Analytics related domains.
• Filter lists (e.g. EasyList) are manually curated
• Machine learning based approaches
google.com, pub−7439281311086140, DIRECT
rubiconproject.com, 11078, DIRECT
c.amazon−adsystem.com, 3159, DIRECT
openx.com, 183753,RESELLER RESELLER
https://cnn.com/ads.txt
indexexchange.com, 197545, DIRECT
adtech.com, 232547, DIRECT
openx.com, 203799,RESELLER
https://washingtonpost.com/ads.txt
1035 valid domains
Top Exchanges like Google,
openX have adopted

100. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 30
New Standards Coming As Well
Have the potential to make the ecosystem even more transparent
• App-ads.txt (For mobile apps)
• Sellers.json (To reveal relationships of sellers in the ecosystem)

101. IMC 2019: A Longitudinal Analysis of the ads.txt Standard 31
Conclusions / Takeaways
• A 15-month study to study the ads.txt standard
• We observed heathy adoption (64% publishers have adopted)
• Major exchanges & DSPs are part of it, and honoring the standard
• A step towards making the ecosystem transparent
• Data needs to be cleaned
Questions?
ahmad@ccs.neu.edu
Dataset
https://personalization.ccs.neu.edu/Projects/Adstxt