2. test1
/test/ verb
take measures to check the quality, performance, or
reliability of (something), especially before putting it into
widespread use or practice; a means of trial.
3. How many of you would say
you are testing strategically?
4. How many of you actually
understand what I mean by
“strategically” testing?
5. Testing Philosophies…
I test whatever I can in a
structured way that directly
shows impact.
I'm going to test this logical
approach (Ad A), then test that
logical approach against a
similar variation with a lil' funky
twist (Ad B).
Going outside of the box
definitely helps.
We should focus on one
change at a time until
statistical significance is
reached.
When I test something, it has
to be something we have
never done before and that I
strongly believe it will make a
positive impact.
Not just test to test.
I try to consider what I think
will be an interesting test that
will show differences in
performance (hopefully for the
better).
I usually try to isolate one
aspect of ad copy, landing
page, etc to limit the variables.
Then will track for 30 days or
however long it takes to find
a statistically significant
winner.
6. Strategic testing
/strəˈtējik/ test-ing/
A test strategy is an outline that describes the testing approach of the software
development cycle. It is created to inform project managers, testers, and developers
about some key issues of the testing process.
This includes the testing objective, methods of testing new functions, total time
and resources required for the project, and the testing environment.
7. Components of:
Test Plan Document
Components of:
Testing Strategy Document
• Introduction
• Test items
• Features to be tested
• Features not to be tested
• Test techniques
• Testing tasks
• Features pass or fail criteria
• Scope and Objectives
• Roles and responsibilities
• Communication and status
reporting
• Test deliverables
• Industry standards to follow
• Test automation and tools
• Testing measurements and
metrics
8. Components of:
Test Plan Document
Components of:
Testing Strategy Document
• Introduction
• Test items
• Features to be tested
• Features not to be tested
• Test techniques
• Testing tasks
• Features pass or fail criteria
• Scope and Objectives
• Roles and responsibilities
• Communication and status
reporting
• Test deliverables
• Industry standards to follow
• Test automation and tools
• Testing measurements
and metrics
9. This presentation will reshape the way you
think about, and approach testing to
provide the most impactful results for your
campaigns.
Regardless of your level of “testing expertise”, I will
have you thinking strategically about the process.
29. Testing Philosophies…
Identify the purpose
for creating a test.
Then set a
benchmark/control and
create one or more
variables with one clear
distinction (each) from
the control.
Testing should be
done to address
an issue or test a
hypothesis.
30. We need to start asking the
before theWHY WHAT
31. Don’t think about what you should
test; start by thinking about why
your current test isn’t working.
@audreybloemer
@seerinteractive
44. State your Hypothesis.
Our Hypothesis is that by increasing awareness of
extra features available, we will increase paid
subscription offers.
Identifying the “Why”
Part 3
49. Think of the SOLUTION as the strategy.
…“how” you plan to fix the problem.
50. Increase Paid
Subscriptions
Users are unaware of
the benefits of paid
service
Use Remarketing to
Promote Benefits of Paid
Service to Free Users
Possible Solutions
Goal
Problem
Solution
51. Increase Paid
Subscriptions
Users are unaware of
the benefits of paid
service
Use Remarketing to
Promote Benefits of Paid
Service to Free Users
Possible Solutions
Goal
Problem
Solution
Increase Paid
Subscriptions
Users are unaware of
the benefits of paid
service
Change Landing Page
layout to emphasis Paid
Service features
Goal
Problem
Solution
52. Increase Paid
Subscriptions
Users are unaware of
the benefits of paid
service
Use Remarketing to
Promote Benefits of Paid
Service to Free Users
Possible Solutions
Goal
Problem
Solution
Increase Paid
Subscriptions
Users are unaware of
the benefits of paid
service
Change Landing Page
layout to emphasis Paid
Service features
Goal
Problem
Solution
54. Increase Paid Subscriptions
Users are unaware of the benefits of paid
service
Use Remarketing to Promote Benefits of
Paid Service to Free Users
Promote Free
Trial in Ads
Goal
Problem
Solution
Tactics
55. Increase Paid Subscriptions
Users are unaware of the benefits of paid
service
Use Remarketing to Promote Benefits of
Paid Service to Free Users
Promote Free
Trial in Ads
Highlight Top
Features in Ads
Goal
Problem
Solution
Tactics
56. Increase Paid Subscriptions
Users are unaware of the benefits of paid
service
Use Remarketing to Promote Benefits of
Paid Service to Free Users
Promote Free
Trial in Ads
Highlight Top
Features in Ads
Run Remarketing
Video Ads
Goal
Problem
Solution
Tactics
57. Increase Paid Subscriptions
Users are unaware of the benefits of paid
service
Use Remarketing to Promote Benefits of
Paid Service to Free Users
Promote Free
Trial in Ads
Highlight Top
Features in Ads
Run Remarketing
Video Ads
Goal
Problem
Solution
Tactics
62. Increase Return on Ad Spend
Users researching products without intent to
purchase immediately are driving up costs
Optimize Bids based on users intent to
purchase
Utilize AdWords API to change bids based
on the weather
Outerwear Client Example
Goal
Problem
Solution
Tactics
68. Best College Ever
Award-Winning Education Program.
Complete Your Online Application.
www.college.com
Importance of a Defined Goal
Best College Ever
Complete Your Online Application.
Award-Winning Education Program.
www.college.com
69. Best College Ever
Award-Winning Education Program.
Complete Your Online Application.
www.college.com
Importance of a Defined Goal
Best College Ever
Complete Your Online Application.
Award-Winning Education Program.
www.college.com
Flipping Description
lines is not a test.
There are no deep
learnings or actionable
next steps as a result
of this data.
70. Ask yourself, “are there any actionable
takeaways I can learn from this test?”
@audreybloemer
@seerinteractive
74. Set benchmark data before you
launch so you can identify success.
Benchmarks should align with your goals; such
as year-over-year data, before vs. after data, etc.
75.
76. The Importance of understanding the
problem preventing you from achieving
your goal.
81. Increase Conversion Rate
Mobile Conv. Rates are low because we do
not have a mobile optimized page
Create a new landing page that is optimized
for mobile
Utilize a shortened form with large, easy to
use form fields
Education Client Example
Goal
Problem
Solution
Tactics
84. Identifying the problem allows you to create
an effective solution to achieve your goal.
@audreybloemer
@seerinteractive
85. Identifying the Problem
Device Reports
Time of Day
Reports
Auction Insights
Google Trends
Talk to other
internal teams
Device Reports
Bounce Rates
Location Reports
Second Page
Reports
Impression Share
86. Identifying the Problem
Device Reports
Time of Day
Reports
Auction Insights
Google Trends
Talk to other
internal teams
Device Reports
Bounce Rates
Location Reports
Second Page
Reports
Impression Share
90. Increase CTR
Ad copy doesn’t resonate with audience;
need to speak to emotional triggers
Create content focused on user’s emotions
and needs
Write new ad copy that empathizes with
target audience
Hospice Client Example
Goal
Problem
Solution
Tactics
