Presented by Kiril Tsemekhman, SVP & Chief Data Officer, Integral Ad Science at AdExchanger's Omni.Digital conference on September 10, 2015.

1. KNOW WHAT YOU MEASURE.
AND WHY.
KIRIL TSEMEKHMAN
CHIEF DATA SCIENTIST,
INTEGRAL AD SCIENCE

2. 1

3. WHAT MATTERS TO YOU?
• Measuring performance:
incrementality
• Attribution: rewarding rightly
• Driving performance: the importance
of environment
• Taking it in: influencing the user
• Revealing what matters
2

4. ATTRIBUTION: REWARDING RIGHTLY
• Establishing the baseline
• Measuring and attributing only incremental actions
• Giving credit where credit is really due
3

5. 4

6. 5

7. 6

8. LESSONS LEARNED
• The treasure chest had “baseline coins”
• Pirates had to find a way to separate baseline coins from their
“incremental loot”
• Knowing each pirate’s effort leads to rewarding rightly
7

9. DRIVING PERFORMANCE:
THE IMPORTANCE OF ENVIRONMENT
• Is viewability all that matters?
• Other dimensions that make a difference
– How many times driver sees the billboard?
– How often?
– Is there enough time to read it?
8

10. 9

11. 10

12. LESSONS LEARNED
• Picking the right place is not that simple
• Consider what’s happening at any given moment
• How big is the opportunity to be noticed and remembered?
11

13. TAKING IT IN: INFLUENCING THE USER
• Opportunity does not equal viewability
• Viewability does not equal engagement
• Engagement does not equal action…yet
12

14. 13

15. 14

16. 15

17. LESSONS LEARNED
• You are not cured by getting a prescription
• You are also not cured by filling a prescription
• You have a better chance to be cured by taking the
medication as directed
– but then again, different dosages may yield better results
16

18. CAUSAL IMPACT OF USER ENGAGEMENT
WITH DIGITAL ADS
• “Natural” A/B test
• Unviewable ads instead of PSAs
• Causal impact of different treatment levels
– Number of ads shown to user
– Total exposure time
17

19. AND NOW…REVEALING WHAT MATTERS
Measuring incremental (causal) actions:
• Does the number of exposures matter?
• Does the frequency of exposures matter?
• Does the total exposure duration matter?
• Are two long ads better than ten short ads?
18

20. 600K
200K
1 2 3 4 5 6 7 8 9 10 11
Impressions per User
NumberofUsersOverOneWeek
7% of users were served over 5 impressions
2.5% of users were served over 10 impressions
AD FREQUENCY: MOST USERS SAW 1-2 ADS
19

21. 700K
120K
1-5 5-15 15-30 30-45 45-60 60-75 75-90 90-120 120-150 >150
Total User Exposure Time (seconds)
NumberofUsersOverOneWeek 57% of users have no viewable impressions
USER EXPOSURE TIME: MOST EXPOSED <5 SEC
20

22. 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 50 100 150
Gaming Telcom
Exposure to Advertising (seconds)
AdditiveLiftinConversions
EFFECT OF TOTAL EXPOSURE TIME ON
INCREMENTAL CONVERSIONS
21

23. 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 2 4 6 8 10
Gaming Telecom
Number of Impressions
AdditiveLiftinConversions
EFFECT OF REPEATED EXPOSURES ON
INCREMENTAL CONVERSIONS
22

24. 0
0.05
0.1
0.15
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
Number of Impressions
AdditiveLiftinConversionsCOMBINED EFFECT OF EXPOSURE
FREQUENCY & DURATION
23

25. LESSONS LEARNED
• Number and frequency of exposures matter
• Total exposure duration matters
• One long exposure does not quite reach the goal
24

26. ULTIMATE LESSONS LEARNED
• Remember what matters and ask the
right questions. Then…
• Measure the right things and measure
them right.
25

27. THANK YOU
KIRIL TSEMEKHMAN
kiril@integralads.com