Presented at SMX Advanced (Seattle, June 2018), Crealytics founder Andreas Reiffen discusses whether retargeting always offers the right approach for marketers. References the limitations and challenges of measuring for incrementality via Google Analytics; and offers a more effective testing approach.
2. #SMX #31c @AndreasReiffen
About…
• Data-driven online advertising strategist
• E-Commerce retail expert
• Entrepreneur
• SaaS product and agency services for
Google Shopping & Search
• €3 billion in client revenues this year
• 170 true experts in their field
• Offices in NYC, London and Berlin
… me … crealytics
3. #SMX #31c @AndreasReiffen
Agenda
Why common practice of retargeting is not necessarily the right approach.
How to measure incrementality using Google Analytics.
Limitations and challenges when applying this methodoligy.
Introducing an advanced and more effective testing approach.
Implications on the skill set of modern digital advertising teams.
5. #SMX #31c @AndreasReiffen
Finance: „Double ROAS within 8 months“
Campaign manager: „Easy“
ROAS target achieved
MoM, Set by Finance Key Insight
RLSA boosted traffic
By Campaign Manager
+102%
3 52 64 71 8 4
43%
37%
86 7
42%
5
50%
24%
19%
23%
19%
1 2 3
Retargeting website visitors
helps to show great
performance.
It is a – well known and used
– secret weapon we use to
effortlessly reach targets.
Does it help the overall
business?
6. #SMX #31c @AndreasReiffen
ROAS looks great but new customer rate halved.
Is that what Finance has asked for?
531
18%
15%
14%
12%
18%
2 4 87
11%
6
12%
10%
-48%
Sharp drop in new
customer rateNew customer rate, MoM
With increased RLSA spend,
new customer rate declines.
Decreased new customer
rate leads to missed out
future revenue from repeat
purchases.
7. #SMX #31c @AndreasReiffen
Pre-RLSA Boost
RLSA – bottom-line incremental impact or just an artifact
as a result of wrong measurement?
17
2
Unpaid
Paid RLSA
20
Paid Non-RLSA
1
Margins
Post-RLSA Boost
14
17
4
42
2
23
Margins
20
Margins
100% RLSA incrementality0% RLSA incrementality
What is incremental?
How many people would
have bought without
seeing the retargeting ad?
Which portion of my paid
performance was non-
paid before?
How to apply data instead
of gut feeling?
M£
VS?
8. #SMX #31c @AndreasReiffen
The better the numbers you see, the less incremental they
might be! Test for yourself.
Display prospecting
Brand
Designer PPC
Generic PPC
RLSA & remarketing
Conversion rate
Incrementality
(+)
(-)
(-)
(+)
With our clients, we have
observed a negative correlation of
conversion rate and
incrementality.
Conversion rate
9. #SMX #31c @AndreasReiffen
Today, it’s common practice to bid according to buying
propensity and improve the overall ROAS
Today’s unquestioned
common practice
Common sense tells us that with increasing
recency and engagement, buying propensity
increases.
The higher the buying propensity the more
likely people are to buy and the better the
ROAS will look.
The better the ROAS looks, the less
incremental sales will be.
Engagement
Recency
10. #SMX #31c @AndreasReiffen
A well-known retargeting company took this to an extreme
- boosting bids in Google Shopping campaigns
Engagement
Recency
RLSA
Bid Modifier
+413%
CPC
$5.8
CR
64.9%
ROAS
28.0
Source:AdWords.Timerange: Q2 / 2017. Industry: Fashion
Are the numbers just looking good or
are they actually good?
11. #SMX #31c @AndreasReiffen
A well-known retargeting company took this to an extreme
- boosting bids in Google Shopping campaigns
Engagement
Recency
RLSA
Bid Modifier
+301%
CPC
$4.9
CR
45.9%
ROAS
24.0
Source:AdWords.Timerange: Q2 / 2017. Industry: Fashion
Are the numbers just looking good or
are they actually good?
12. #SMX #31c @AndreasReiffen
A well-known retargeting company took this to an extreme
- boosting bids in Google Shopping campaigns
Engagement
Recency
RLSA
Bid Modifier
+184%
CPC
$4.1
CR
26.4%
ROAS
17.0
Source:AdWords.Timerange: Q2 / 2017. Industry: Fashion
Are the numbers just looking good or
are they actually good?
13. #SMX #31c @AndreasReiffen
A well-known retargeting company took this to an extreme
- boosting bids in Google Shopping campaigns
Engagement
Recency
RLSA
Bid Modifier
+99%
CPC
$2.8
CR
13.5%
ROAS
12.0
Source:AdWords.Timerange: Q2 / 2017. Industry: Fashion
Are the numbers just looking good or
are they actually good?
14. #SMX #31c @AndreasReiffen
Retargeting is highly addictive.
Is there a way back?
My boss will kill me
if he finds out I
wasted our budget.
Director
Performance
Marketing
Agency
Less RLSA means
we will lose a lot
of money!
Let‘s both keep
quiet.
Better-looking
results mean
higher PPC
Budgets.
Google
$ $$ $
15. #SMX #31c @AndreasReiffen
Bid higher with high buying
propensity?
Bid high where incrementality
is likely to be high?
Engagement
Recency
How can it be that even experts couldn‘t tell which of two
opposite bidding strategies is superior!
Engagement
Recency
$
$
16. #SMX #31c @AndreasReiffen
Key take aways for target KPIs
1. Be skeptical about too good to be true results
2. Especially on simple metrics like ROAS while not
taking into account new customer acquisition
3. The better your numbers look the less
incremental they might be
18. #SMX #31c @AndreasReiffen
Run an A/B test by following two groups of website
visitors.
Split your visitors in
two groups Comment
Show retargeting ads
only to Test group
Show ad
Don‘t show ad
A: Test group
B: Control group
Split your visitors evenly into
two cohorts.
Follow the cohorts over time
to see if showing ads to the
Test group increases
conversions and revenue in
comparison to the control
group.
19. #SMX #31c @AndreasReiffen
Example Setup for testing RLSA Incrementality uses
Google Analytics and Google AdWords
Set up
audiences
Export
and collect
users
Activate in
campaigns
Measure
results
1
2
3
4
20. #SMX #31c @AndreasReiffen
Randomising website visitors as test and control group
A
B
For two groups, assign 50 buckets eachIn GA, use condition User Bucket
1
21. #SMX #31c @AndreasReiffen
From Analytics to AdWords
1
2
3
4
Start collecting users in AdWordsIn GA, finalize A/B audiences and export
IncrementalityTest – Control group
2
22. #SMX #31c @AndreasReiffen
Combine buckets with existing audiences for more
granular tests
ATest bucket B Control bucket
Basket Abandoners
A
Basket Abandoners
B
Basket Abandoners
Category PageVisitors
Product PageVisitors
Category PageVis. -
A
Category PageVis. -
B
Product PageVis.
A
Product PageVis.
B
Exported
lists
Existing
audiences
Bid Modifiers
+100-90
+100-90
+100-90
+100-90
24. #SMX #31c @AndreasReiffen
We tested high funnel and low-funnel retargeting.
At first glance, results were inconclusive. Natural fluctuations higher.
AdWords revenue
and audience boosts
Analytics bucket
revenue by case
Low funnel
Exclude Allow high-funnel Boost low funnel
High funnel
+138%
+21%
Boost low
funnel
Exclude Allow
high-
funnel
+8% -2%
Bid Modifiers
-90
-90
-90
+-0 +-0
+90
-90 +-0 +90
Revenue
challenge
Uplift in AdWords
audience revenue is
only 1.4% of total
Analytics revenue
Solutions:
• test all visitors
• stronger boosts
• longer test time
frame
25. #SMX #31c @AndreasReiffen
User number challenge: too many users who never had
the chance to see AdWords retargeting
Who was eligible to see
retargeting?
Use Analytics sequence
to specifiy users
Irrelevent users
succesfully reduced
All new users
during test
Possibly
eligible
414,000
108,000
By excluding users not
eligible to retargeting, the
effect is less clouded,
revealing results quicker.
• Has seen the basket.
• Not purchased in first
session.Not eligible
for
retargeting
Possibly
eligible
A vs B comparison now on
reduced user set
All users
26. #SMX #31c @AndreasReiffen
After filtering, we see: Buying propensity and
incrementality may indeed be antagonistic.
0
10
20
30
40
50
60
70
80
90
100
1.00.0 0.5 1.5 2.0 2.5
Conversion Rate
High funnel
Incrementality
All retargeting
Range of prediction still wide.
Incrementality for all retargeting
ranges between 9% and 34%.
High funnel retargeting tends to
have a higher incrementality and
lower conversion rates.
How can we get more precise?
27. #SMX #31c @AndreasReiffen
Alternative: improved audience creation by event
triggering, leading to smaller A/B test with clearer result
VS
Identify users visiting
certain page, e.g. basket
JavaScript randomiser
Trigger Analytics event
to build and follow audience
Compare audiences and
nothing else after having seen
or not seen an ad
Follow events in Analytics, i.e.
A vs B is small and sharp.
Better randomization of basket
viewers into A vs B.
By comparison, Analytics
buckets may be unfair on
smaller, specific sub-lists.
Advantages
30. #SMX #31c @AndreasReiffen
There is nothing quite so useless as doing with great
efficiency, something that should not be done at all –
– Peter F. Drucker
Team of mechanics
• Optimising the numbers without questioning them
• Tweaking keywords, ads, landing pages
Mechanics supported by data scientists/analysts
• Focus on measurability and data accuracy
• Challenge target KPIs and budget allocation
Run fast, drive efficiency Run into the right direction, be effective
Data Scientists/Analysts
31. #SMX #31c @AndreasReiffen
Share these #SMXInsights on your social channels!
#SMXInsights
Be skeptical about
too good to be true results
Constantly test incrementality and sense-
check your attribution
Run fast, but also into the right direction
Set up random audiences using GA’s user bucket dimension
Export GA audiences to AdWords and collect data for search eligibility
Set up campaigns to reflect test design and activate
Measure results with GA custom report
User Buckets are a random number from 1-100 assigned to any visitor of your website
User Bucket information is stored in the cookie, i.e. is persistent across sessions
In the GA Audience Builder, go to Conditions
Look for filter dimension *bucket*
Set to desired group size – for two groups of the same size, choose: A: <=50, B: >50
‘Users over last 7 days’ will give you a first idea how large the list might become (1)
Assign a name to the audience (2) and finalize the Audience Builder work by choosing a destination for the audience – in this case: AdWords (3)
Once in AdWords, you will have to wait for the list to collect enough users to become eligible for search retargeting (4).
To make sure test is administered while the control group is treated as usual
Create combined lists of existing RLSA lists with each random list
Control audiences inherit the existing bid modifiers
Test Audiences’ bid modifiers are set to 0%
Accept lower ROAS numbers
Fewer conversions eligible for counting will mean lower ROAS figures.
Remember it is just numbers. Nothing in your company’s overall performance has changed. Yet.
Consider better KPIs
To escape the vicious cycle of ROAS targets, consider more meaningful KPIs.
More meaningful as in: ‘directly linked to business objectives’, like profit or CLV.