With the increase in the render capabilities of search bots, the influence of Javascript on SEO is not only real, but growing in a way that demands attention from anyone serious about search optimization. But most traditional web crawlers can’t see content and links rendered in JavaScript, like comment and recommendation widgets, despite the fact that search engines index this content routinely. That's where Botify comes in!

1. Botify Webinar
December 13th, 2017 (EN)
December 14th, 2017 (FR)
Understand the Impact of
JavaScript on SEO

2. 1. Why Is JavaScript Important for SEO?
2. JS Does Different Things on Different Websites
3. Enterprise-Grade JS Crawler Needed
4. What Can You Analyze, What Types of
Takeaways?
Webinar Agenda

3. Part #1
Why Is JavaScript
Important for SEO?

4. ● It allows to build user friendly, highly
interactive websites
● More and more websites use JS
frameworks (like Angular JS, React, etc.)
● Unavoidable with the move towards
Progressive Web Apps and Google's
"mobile first" index
JS Has Become Ubiquitous
What JS does on the website varies significantly from site to site.

5. C/S communication: Standard HTML
Browser asks Server what it should do
Server responds with HTML (“display this”)
Browser displays HTML
1
2
3
How Does JavaScript Work?

6. C/S communication:
Standard HTML + JS
Browser asks Server what it should do
Server responds with HTML +
JavaScript (“display this, and do that”)
Browser reads and displays HTML
Browser executes the JavaScript and
fetches many more resources to
execute
1
2
3
4
How Does JavaScript Work?

7. So What Does JS Crawling Mean ?
Robot asks Server what it should do
Server responds with HTML +
JavaScript
Robots reads HTML
Robot executes the JavaScript and
fetches many more resources to
execute
Robot stores resulting page
1
2
3
4
5

8. Robot asks Server what it should do
Server responds with HTML +
JavaScript
Robots reads HTML
Robot executes the JavaScript and
fetches many more resources to
execute
Robot stores resulting page
So What Does JS Crawling Mean ?
1
2
3
4
5

9. ● HTML-only crawl is not enough for most
websites
● BUT crawling JS is extremely
resource-intensive
This mean:
➔ Not all JS is rendered - even by Google.
➔ Analyzing a handful of pages is not
enough to draw any sort of conclusion.
JS Impacts Search Bots

10. What about Pre-rendering?
Robot asks Server what it should do
Server gets HTML and executes JS
Server responds with resulting HTML
Robots reads HTML
Robot stores page
1
2
3
4
5
For search engines bots, no JS
execution needed

11. Asking The Right Questions
● Analyze differences
between HTML-only and
JS-rendered pages
● Analyze impact on
website structure
(linking, content
quality…)
● Analyze JS performance
Partial JS
JS-generated elements
matter for SEO
• Analyze impact of JS on
performance
• Pre-rendering: check
server-side vs client-side
rendering
Full JS
• HTML-only analysis is all
that is needed for
structural and content
analysis of the website.
• BUT worth wondering
whether these JS
elements may negatively
impact Google's crawl
Partial JS
JS-generated elements
don't matter for SEO

12. Part #2
Javascript does different
things on different websites

13. Example #1: Products Rendered with JS

14. Example #1: Product Links Rendered with JS

15. Example #2: Faceted navigation with JS

16. Example #2: Faceted navigation with JS

17. Example #3: Content in product page inserted with JS

18. Example #3: Content in product page inserted with JS

19. Example #4: Share buttons and other user actions

20. Example #4: Share buttons and other user actions

21. Example #5: Comments inserted with JS

22. Example #5: Comments inserted with JS

23. JavaScript disabled
JavaScript enabled
Example #6: Full JS

24. Part #3
Enterprise-grade JS crawler
needed

25. Careful setup needed:
● Crawl speed: Number of requests
multiplied by 10-100, risk of overloading site(s)
● Crawl scope: Some JS function should
not be executed (analytics, cart management
on e-commerce websites, 3rd party resources,
mouse tracking…)
JavaScript Crawling Requires Caution

26. What does the JS crawl enable?
Botify:
✓ Can execute ALL JS code (internal & external)
✓ Allows selection of resources to execute or to ignore
✓ Enables caching of external resources
✓ Complies with Robots.txt’s instructions (they can however be
replaced with other specific instructions)
As a result Botify will:
✓ Crawl links (<a> tags) inserted using JavaScript
✓ Discover tags inserted using JavaScript
✓ Analyze page Content inserted using JavaScript
✓ While reporting in details on JS resources and performance
Botify’s JavaScript Crawl

27. As with standard
crawls, crawled
HTML code is
available
Botify’s JavaScript Crawl
See page code resulting from JS rendering

28. Botify’s JavaScript Crawl
What is the recommended process to run a first JS crawl?
The Botify team
configures (and
launches) a standard
(non-JS) crawl of the
site on a small number
of URLs
The Botify team
configures (and
starts) a JS crawl
without any
restrictions on the
same number of
URLs
The Botify team
works in
collaboration with
you and your tech
team to fine-tune the
JS configuration
You can start the
larger JS crawl using
the custom JS
configuration
(and any subsequent
crawl with the same
config.)
1 2 3 4
Without crawling any
tracking tags by
default!

29. Javascript performance and resources
Botify’s JavaScript Crawl: Key JS Metrics
New JS charts in the performance section!

30. Botify’s JavaScript Crawl: Key JS Metrics
Number of JS resources
executed, by segment
JS rendering time, by segment
JS resources blocked
by robots.txt, by
segment

31. Botify’s JavaScript Crawl: Key JS Metrics
More new metrics in the URL explorer… also available to use as filters

32. Part #4
What can you analyze,
what what types of
takeaways?

33. Without JS
With JS
Compare without JS / with JS:
● Impact on pages inventory and linking

34. Compare without JS / with JS
Zooming in on pages which have more links with JS (using a filter based on number of
links in previous analysis - without JS - and current analysis - with JS)

35. Details in the URL Explorer
Compare without JS / with JS

36. Avg. number of words,
full content
Without JS
With JS
Avg. number of words,
excluding template
Compare without JS / with JS:
● Impact on content

37. Compare without JS / with JS
Zooming in on pages which have more content (excl. template) with JS

38. Full-JS website
All aspects of load time (initial HTML + JS rendering)

39. Understanding JS load time: number of executed resources
Full-JS website

40. Put this JS crawl in perspective: number of resources executed vs all resources
associated to pages
Full-JS website

41. Same approach as standard crawl analysis, taking also into consideration JS-related
metrics, as JS represents a major hurdle for search engines:
Assess what robots can see on the website, and what may add difficulty.
- Accessibility (JS rendering time and resources, non compliant pages, depth,
internal linking, presence in sitemaps, ...)
- Content quality (HTML tags, content size excluding template, content
uniqueness…)
- Correlate to visits (when web analytics data connected to Botify), correlate to
Google's crawl behavior (when logs analysis).
Full-JS website

42. ● Javascript assessment on websites has become
mandatory - there is no ignoring it.
● It is new territory for SEO managers.
● This type of analysis requires working closely with the
website's technical team.
In Conclusion

43. Any
questions?
Get in touch!
marketing@botify.com