Andrew Miller's presentation from April 2017 at HeroConf in Los Angeles covers the intersection of Pay Per Click advertising and Conversion Rate Optimization.
2. @AndrewCMiller
#HeroConf
How to budget for PPC to support
CRO tests.
How to factor PPC strategies into CRO
test design.
How to layer PPC audiences to avoid
skewing your CRO test results.
A 6-Step CRO process that begins
with research and ends with analysis.
8. SMALL BUSINESSES
Constrained
by budgets
Small teams of
generalists
Time
LARGE BUSINESSES
Multiple teams
Different
stakeholders
Competing
objectivesANYBODY
Wrong metrics
HiPPO or
group think
Tactics before
strategy
9. @AndrewCMiller
#HeroConf
A Q U O T E O F T E N I N C O R R E C T L Y
A T T R I B U T E D T O I S A A C N E W T O N
We build
too many
walls and
not
enough
bridges.
18. @AndrewCMiller
#HeroConf
6 Results Rollout
& Repetition
What did we learn?
What can be applied
to future tests?
Document and share
test results to inform
future campaigns.
STAGE
QUESTION ACTIONS
21. @AndrewCMiller
#HeroConf
Use identical campaign settings
Compare similar markets & devices
Don’t mix audiences & keywords
Controlled VariablesMinimize the impact of incomplete or inconclusive data
22. @AndrewCMiller
#HeroConf
Try not to test
over holidays or
major news events
Consider other
marketing
and promotions
Analyze the same
number of days and
days of the week
Tests Don’t Occur in a VacuumDon’t let seasonal or other temporary variables skew test results
26. @AndrewCMiller
#HeroConf
ALL WEB USERS
PREVIOUS VISITORS &
CUSTOMER LISTS
ABANDONED
FUNNEL
Include & Exclude The Right AudiencesAim for clean, consistent segments to avoid misreading results
CONVERTED
Extremely
vague
Brand aware
Ready to activate
Primed for loyalty
27. @AndrewCMiller
#HeroConf
PPC Targeting Impacts CRO Test ResultsAim for clean, consistent segments to avoid misreading results
Do users on different
devices have different
experiences?Does geographic proximity
impact conversion rates?
Are your offers appealing to
specific gender?
Are your most valuable
prospects older or younger?
30. Determine a proper sample size
Aim for > 95% confidence level
Ensure the validity of your test results
Statistical Significance
Reduce probability of an invalid result
31. VWO’s A/B Test Duration Calculator:
vwo.com/ab-split-test-duration/
KissMetrics A/B Significance Test
getdatadriven.com/ab-significance-test
Optimizely’s A/B Test Sample Size
Calculator
www.optimizely.com/resources/sample-
size-calculator/
Helpful Budgeting ToolsEnsure the validity of your test results
32. @AndrewCMiller
#HeroConf
How to budget for PPC to support
CRO tests.
How to factor PPC strategies into CRO
test design.
How to layer PPC audiences to avoid
skewing your CRO test results.
A 6-Step CRO process that begins
with research and ends with analysis.
Building a bridge over a treacherous river
Failure is not an option
At worst: imminent demise
At best: a decent shot at $10 grand on AFV
Does this make anybody nervous? Probably piranhas and all sorts of previously undiscovered bacteria.
This is how I felt the first time I set up a CRO test with PPC traffic.
This is what I see nearly every day when marketers ignore the links between PPC and CRO
Missed opportunities to improve CPA.
Flawed test results and bad data
Bad business decisions
PPC is capture
CRO is convert
PPC and CRO are both iterative processes
Testing and learning
CPA can be lead, sale, or any conversion
CPC is cost to acquire one visitor
Conv Rate is percentage of visitors that convert
CPA = CPC ÷ Conv Rate
To lower CPA, one must lower CPC or raise CR
Adjust CPA target, CPC, and Conversion Rate
Find scenarios that meet or exceed your target CPA
Make a copy!
This is not just a problem for small businesses or enterprise. This affects everybody.
10 years, only a handful get it right.
It’s just hard.
But small business can beat larger competitors
by out-converting
Reality is that too many companies build walls between PPC and CRO.
Too many people assume this is the only way.
So to those that just accept the constraints faced by small or large businesses, I say
We can do better.
We can break down the walls between PPC and CRO.
But first, we have to understand CRO and how to design a proper test.
Why have a process?
Repeatable actions lead to consistency.
Iterative – can always be improved.
Most people jump straight to step 4 – Design & Copywriting.
Starts with Research & Analysis.
Goal is to use data to identify the problem.
Use the data to inform your hypothesis.
"I believe we can improve metric X by changing variable Y"
Possibly the most crucial step. Getting it wrong will skew your data or lead to invalid results.
Outline all test components as if you were going to have somebody else replicate it.
This is how real scientists validate each other's findings.
The fun part.
What most people associate with CRO or LPO.
Impossible to justify the test variables or treatments without a solid test plan.
A great time to share results with other marketing channels.
How PPC impacts your CRO and LPO tests.
Linking PPC and CRO is like building a bridge:
Beautiful when complete, dangerous when under construction.
Apples to apples comparisons
Analyze results by comparing similar segments
Clean data allows for better decisions.
Set up in PPC campaigns
gclid, mslkid
UTM parameters, valuetrack parameters
Test variant, offer, control vs. experiment group, previous customer data
Make sure your CRO platform, analytics, and CRM are configured to capture this info
These are primarily PPC audiences.
Consider all potential combinations of website users and previous customers.
Exclude or include based on your strategy & experiment design.
Factor in device, geography, demographics.
Some CRO platforms allow for segmenting test subjects to run clean tests.
If yours doesn’t, use your PPC settings or analytics segments to derive insights.
Too little traffic leads to long tests or inconclusive results.
Building a bridge over a treacherous river
Failure is not an option
At worst: imminent demise
At best: a decent shot at $10 grand on AFV