In this talk, delivered at BrightonSEO in April 2023, Will Critchlow, founder and CEO of SearchPilot covers a method for generating practically-unlimited SEO A/B test ideas. Going on a journey from what SEO has been to what it needs to become, Will covers the mindset and strategy shifts needed, as well as the tactical implementation details. Download resources including detailed guides to SEO testing, and the free tool he uses to generate the ideas (plus explainer video).

1. How to generate 8
million SEO test ideas
slideshare.net/willcritchlow/
@willcritchlow
Will Critchlow
SearchPilot

2. You know how SEO
has always been…

3. Audits

4. Keyword research

5. Recommendations

6. It’s led to too much
failure

7. Our industry has been trying and failing to gain
influence
28%
23%
17%
What do you think is the
biggest risk to technical SEO
success?
Aira: The State of Technical SEO Report 2023

8. 1. Organizational
lack of resources

9. 2. Getting things
done

10. 3. Proving the
results

11. Let’s find a new way

12. Many of you know Dharmesh

13. Dharmesh is the king of inbound

14. So you might assume I’m going to talk INBOUND

15. But actually this talk is about ZIGGING when others
ZAG
Here’s the secret to
what makes Brian
special. He pushes
himself and HubSpot to
zig where others zag.

16. Sidenote: the host, Jason Lemkin is well worth a follow
It’s spooky 👻. Every
time I hit a business
problem, Jason has
just written an article
on that very subject.

17. Take a moment to THINK ABOUT YOUR
COMPETITION

18. Some are spending BIG $$$ on AGENCIES

19. Some pockets are EVEN DEEPER

20. So you’re up against BIG TEAMS

21. to ZIG?
Are
you
ready

22. SEO testing
It all starts with some new abilities:

23. SEO testing means you can tie individual on-site changes to organic search
performance (reference).

24. SEO testing means you can tie individual on-site changes to organic search
performance (reference).

25. SEO testing means you can tie individual on-site changes to organic search
performance (reference).
+14%
organic
traffic

26. & winning tests
new pages
Modern on-
site SEO
should cut
out untested
changes and
focus on:

27. Get the
recording
and slides

28. Get the
recording
and slides
http://bit.ly/brightonseo-moneyball

29. …but in fact we have
some new ones!
We might think testing solves all our problems

30. Most items in most audits are a waste of time for SEO
Alt attributes: a good idea, but no organic traffic improvement:

31. Most items in most audits are a waste of time for SEO
Fixing internal redirects: probably better than not, but tiny benefit:

32. Most items in most audits are a waste of time for SEO
Trivial canonicalization: probably better than not, but tiny benefit:

33. Many
that
could be
good are
too
vague

34. Many
that
could be
good are
too
vague
“Add the keywords
identified in our
research to the titles
of these pages.”

35. Many
that
could be
good are
too
vague

36. Up to 80%+ of tests are negative or inconclusive

37. Up to 80%+ of tests are negative or inconclusive
So we need to do something different:

38. Business doesn’t believe the uplift vs. counterfactuals

39. Business doesn’t believe the uplift vs. counterfactuals
If you don’t see a
MoM uplift, is it real?

40. There are
some
concerns
that turn
out not to
be a
problem

41. There are
some
concerns
that turn
out not to
be a
problem
“Do we need to
worry about
running out of
test ideas?”

42. There are
some
concerns
that turn
out not to
be a
problem

43. There are
some
concerns
that turn
out not to
be a
problem
SearchPilot blog: Will we run out of test ideas?

44. How long do
results last?
There are
some
concerns
that turn
out not to
be a
problem

45. How long do
results last?
We don’t know exactly
without re-testing, but
most people deprioritize
that.
There are
some
concerns
that turn
out not to
be a
problem

46. Can we revert
tests
successfully?
There are
some
concerns
that turn
out not to
be a
problem

47. There are
some
concerns
that turn
out not to
be a
problem
Can we revert
tests
successfully?
Mainly yes. Only redirects
and canonicalization are
really hard. Links can be
slow.

48. … we get to change
our mindset
As we overcome all of these…

49. The
old
way
Test things you
would have done
anyway

50. 1. Quantify the
impact
This does
have
benefits!

51. 1. Quantify the
impact
2. Avoid unexpected
losses
This does
have
benefits!

52. This does
have
benefits!
1. Quantify the
impact
2. Avoid unexpected
losses
3. Save engineering
time

54. In other words:
This is a conference
presentation! It is about the
boundaries of what’s possible. If
you haven’t started testing
yet, start there!

55. An
even
better
way
Build a testing
mentality

56. 1. Zig while others
zag
Even
bigger
benefits!

57. 1. Zig while others
zag
2. Sift for
unexpected
winners
Even
bigger
benefits!

58. Even
bigger
benefits!
1. Zig while others
zag
2. Sift for
unexpected
winners
3. Take radical risks

59. Going from auditor to
explorer
Let’s talk about the details

60. Abundance
mindset

61. Abundance
mindset
Not “what should
we do next”, but
“what are all the
things we could
do?”

62. Embrace
uncertainty

63. Embrace
uncertainty
Not “what’s best”
but “what’s
possible?”

64. Seek the
radical

65. Seek the
radical
What will our
competition not
be brave enough
to try?

66. [ Image here ]
Example: radical removal of text
Many e-commerce websites optimize their
categories by adding a big chunk of text under
the product listings.
… that’s something…which I would try to
avoid.
– John Mueller, Google

67. Example: keyword experimentation

68. Example: machine-learning-generated text (blog post)

69. Sounds hard?

70. I have good news!
Sounds hard?

71. SEO test idea
generation
tool for title
tags

72. http://bit.ly/brighton-8-million

73. The demo runs on
the keyword
[hotels in London]

74. We can see how common each feature is in the top 100

75. …or just in the top 20

76. …or we can compare the top 20 and bottom 20

77. It all runs off a ScreamingFrog crawl of the top 100
results

78. We do a load of analysis of the features:

79. The combinations here give 8 million title ideas
Count: 23
223 = 8,388,608

80. One of the things we want to look up is “contains
brand”

81. [ Image here ]
We need a mapping from domain name to brand

82. [ Image here ]
We need a mapping from domain name to brand
Some are easy

83. [ Image here ]
We need a mapping from domain name to brand
Some are tricky

84. [ Image here ]
We need a mapping from domain name to brand
Some are hard!

85. [ Image here ]
This feels like the kind of thing we have to do manually

86. [ Image here ]
“young watchmaker
working intensely wearing
an eyeglass loupe, highly
detailed, realistic
illustration” – midjourney
This feels like the kind of thing we have to do manually

87. Or this could be an elusive real use of ChatGPT

88. @benedictevans said it well:
“Talking about ML does tend to
be a hunt for metaphors, but I
prefer the metaphor that this
gives you infinite interns”

89. For each of these domains, can
you tell me the brand name of
the company?
The key is crafting the prompt. What we want to ask:

90. For example:
● parkgrandlondon.com would be "park grand
london"
● noxhotels.co.uk would be "nox hotels"
● hotels.com would be "hotels.com"
● guide.michelin.com would be "michelin"
I found it was improved by giving it examples

91. [ Image here ]
Putting this all together looks like this:

92. [ Image here ]
Putting this all together looks like this:

93. I recorded a short video explainer

94. http://bit.ly/brighton-8-million

95. There is an irony in
the way the process
to zig when others
zag builds on
LOOKING AT WHAT
EVERYONE ELSE IS
DOING

96. Let’s bring it back to Dharmesh…
… via a little detour
into high finance

97. The performance
of an investment
compared to the
market
𝛼

98. 👍
“We’re
up 25%”

99. 👎
“The
market is
up 35%”

100. Alpha is
NEGATIVE
𝛼

101. Generating
sustained
alpha is
difficult

102. Generating
sustained
alpha is
difficult
Everyone
copies
you!

103. Build the
“alpha
making”
machine

104. It’s not the titles you
find (your competitors
can copy them).
It’s your process for
continuing to improve.

105. Remember Dharmesh:
Zig where
others zag

106. Podcast
SEO test result library
Bonus material

107. 1.

108. 1.
http://bit.ly/business-class-lounge

109. 2.
www.searchpilot.com/newsletter

110. 2.
www.searchpilot.com/newsletter

111. Thanks
!
Please do ask me any questions you have on
Twitter
@willcritchlow