Catalyst’s Cara deBeer attended SMX West 2019 to join the insightful paid search panel “Ad Testing In A Multi-Format World.” As part of the panel, Cara focused on how to analyze and understand RSA performance by examining important KPIs. In her presentation, Cara discusses how to implement SA360. Here is her step-by-step guide for implementing SA360 and her steps for separating RSA results from ETA results.

1. #SMX @CatalystSEM
Ad Testing In A Multi-Format World
Understanding RSA
Performance by Looking at
KPIs That Matter
Cara deBeer, Partner, Paid Search Director, Catalyst

2. #SMX @CatalystSEM
• Organic Search
• Paid Search
• Social Media
• eCommerce Marketing, including
Amazon Advertising
WHAT WE DO
CatalystDigital.com
Info@CatalystDigital.com

3. #SMX @CatalystSEM
▪ The promise of RSA
– More real estate
– Machine-learning optimization
for best ad, instead of
cumbersome manual ad copy
tests
RSA

4. #SMX @CatalystSEM
▪ The challenges of RSA
– Supported reporting is high-level
and the only concrete metric is
impression lift
– Lack of KPI insight means “top
ads” are meaningless
– Lack of ability to aggregate top
ads to understand performance
trends
RSA

5. #SMX @CatalystSEM
– Basic set-up: RSA and ETAs side by
side
– SA 360 Workaround
– Marin Workaround
Reporting Workaround

6. #SMX @CatalystSEM
Results Overview
-41%
While ETAs perform better on CTR and CPCs, the superior performance from
RSAs on CVR and thus CPA makes them the clear choice
RSA vs.
ETA
CTR % change
-31%
+11%
CVR % change CTA % change CPC % change
+60%

7. #SMX @CatalystSEM
Desktop
Results Segmentation by Device
Greater CVR
benefit in
desktop and
tablet; mobile
benefits less
dramatically
Mobile
Tablet
CTR %
change
CVR %
change
CTA %
change
CPC %
change
-34% -44% +2%+83%
-42% -1%
+28% +27%
-42% -33%
+59% +7%
RSA vs. ETA
RSA vs. ETA
RSA vs. ETA

8. #SMX @CatalystSEM
Results Segmentation By Match Type
BMM
BMM benefitted more Exact, although both improved. Match type variations
in performance may be in part because Exact is more likely to show a longer
ad in any case
-40%
CTR % change
-2%-56%
+40%
CVR % change CTA % change CPC % change
-10% -21%
+10%
+125%
Exact
RSA vs. ETA
RSA vs. ETA

9. #SMX @CatalystSEM
Results Segmentation by Keyword Type
Brand
Looking at Brand vs Generic, CVR is worse in RSA Generic
-35%RSA vs. ETA
CTR % change
-34%
-32%
+10%
CVR % change CPA % change CPC % change
+61%
-18%
+33% +22%
Generic
RSA vs. ETA

10. #SMX @CatalystSEM
Some variation between products, but CVR is stable or improved
and CTA improves on all products except #1
Results Segmentation by Product
CTR % change CVR % change CPA % change CPC % change
-49% -2% +16% +14%
-49% +49% -23% +15%
-17% -2% -5% -7%
-35% +67% -34% +10%
-20% +7% -14% -8%
Product 1
Product 2
Product 3
Product 4
Product 5

11. #SMX @CatalystSEM
More typical RSA results from this Fortune 500 company show a
higher CTR on RSA but lower CVR when looked at in aggregate
Results Overview – Fortune 500
ETA vs.
RSA
CTR % change
-15% -13%
CVR % change CPA % change CPC % change
+3%+22%

12. #SMX @CatalystSEM
Results Segmentation by Match Type
BMM
Again, CVR does much better for BMM than Exact match keywords,
which underperform here with RSA
CTR % change
-11%-22%
+26%
CVR % change CTA % change CPC % change
+13%
Exact
RSA vs. ETA
RSA vs. ETA +25%
-39% -23%
+24%

13. #SMX @CatalystSEM
Results Segmentation by Keyword Type
CTR % change
-24%
+151%
CVR % change CTA % change CPC % change
+33%
-61%
+34%
+12%
-3%
Brand
RSA vs. ETA
Generic
RSA vs. ETA
-43%
On this account, it’s Generic keywords that benefit from RSA

14. #SMX @CatalystSEM
Results Segmentation by Geo Market
+13% -92% +1009% -9%
+32% +497% -83% +3%
+19% +53% -38% -5%
+25% -28% +30% -6%
-4% -81% +389% -9%
+23% -14% +1% -13%
CTR % change CVR % change CPA % change CPC % change
Chicago
Houston
LA
NYC
San Fran
Rest of US
Houston and LA benefit from RSA, but other markets do not

15. #SMX @CatalystSEM
Why so much variation?
It didn’t work well for our Fortune 500 client.
It worked well for our global publishing
client.
Why?

16. #SMX @CatalystSEM
1A century of brand equity
They have decades of brand
equity, and while there are plenty
of other magazines in the world,
they have a unique value
proposition which isn’t met by any
one competitor, and much of their
success is brand-driven.
These factors combine to create a situation where increased ad real estate and
(hopefully) relevance is an enormous asset, allowing RSA to out-perform ETAs
2Competition
They also have a lot of
competitors capitalizing on their
brand equity to sell their own
products or to resell our global
publishing clients’ products
(looking at you, Amazon).
3Multiple product offerings
They also have multiple internal
product offerings beyond the
magazine, such as theTV
offerings and their travel
program.

17. #SMX @CatalystSEM
What we know
What we don’t know
Wild speculation
What’s really going on

18. #SMX @CatalystSEM
Share these #SMXInsights on your social channels!
#SMXInsights
Everyone Should Test
RSAs, But Not Everyone
Should Test Them All The
Time

19. #SMX @CatalystSEM
Share these #SMXInsights on your social channels!
#SMXInsights
▪ Do Use If:
– You want to take advantage of the
increased real estate
– You are able to differentiate RSA
performance from regular ETAs
– You’ve got a scaleable handful of top
adgroups that will allow you to use existing
reporting on top combinations via manual
data pulls

20. #SMX @CatalystSEM
Share these #SMXInsights on your social channels!
#SMXInsights
▪ Don’t Use If:
– You are in a highly regulated industry
– You need to drive performance, but you have
no way of looking at results beyond Google’s
current reporting
– You want to do specific message testing with
KPIs beyond impression/click lift

21. #SMX @CatalystSEM
Share these #SMXInsights on your social channels!
#SMXInsights
RSA performance can
yield much more when
you dig deep into the
details

22. #SMX @CatalystSEM
Get My Step-by-Step Guide
For implementing SA360 and
separating RSA & ETA results
bit.ly/2sGBWa3
Available on the Catalyst blog:

23. #SMX @CatalystSEM
Thank you!
CatalystDigital.com