Benjamin Bussière - Javascript and seo misconceptions, misunderstandings, and reality

1. Make search work
JS and SEO:
Misconceptions,
Misunderstandings,
and Reality

2. Quick introduction
Benjamin Bussière
Director Professional Services
We're giving SEO stakeholders the data, tools, and
diagnostics necessary to create and maintain high
quality digital assets for search engines and voice
assistants.

3. Why do Search Engines need to
embrace JavaScript?

4. JavaScript is here to stay
Why you can’t turn you back on JavaScript
How does JavaScript affect Search Engines?
What does Google say about JavaScript?
What is the real impact of JavaScript on
SEO?
A few examples of how JavaScript can impact your SEO
How can you optimize your JavaScript for
Search Engines?
What can you do at the moment with free tools, with
Botify?
1.
2.
3.
4.

5. 1 JavaScript is here to stay
Why you can’t turn you back on JavaScript

6. ● Tells the browser what to display
● Tree of nodes, described by tags
● Nodes can contain:
○ Text
○ Metadata
○ Links
○ (Assets) - not relevant
○ (Style) - not relevant
● Uses HTML to tell the browser what to display
● Allows a developer to dictate (‘Script’) a browser’s
behaviour after receiving a Web Page
● Can make extra requests
● Can react to user interaction (clicks, mouse movement,
keyboard, etc…)
● Can store information on the user’s computer
● Can modify the HTML
HTML is the browser’s display language JavaScript is the browser’s behaviour language
What is JavaScript?

7. Usage Top 1 million sites by traffic
Detections of JavaScript Libraries and Functions.
Source : https://trends.builtwith.com/javascript
6 millions JS
My favourite framework is Vanilla JS ;)

8. How did JavaScript become so prevalent?
JavaScript plays an
important role on Mobile
Websites are more
interactive, feature rich,
customizable
Popular JavaScript
frameworks are endorsed
by developers1 2 3
4
JavaScript allows to easily externalize features, for instance:
Internal Search Engine
Personalized Recommendations’
Engines
Comment plug-in for engaged
communities
Currency converter Location-related content Tag Containers

9. 1) Browser asks Server what it should do
2) Server responds with HTML + JavaScript
(“display this” and “then do that”)
3) Browser reads and displays HTML
4) Browser executes the JavaScript
• Multiple browser/server communication
• Browser modifies what it displays through JavaScript
• User-interaction can be handled by the browser, without
server communication
JavaScript as seen by Browsers
How do browsers handle JavaScript?

10. 1) Crawler asks Server what it should do (pretending it’s a browser)
2) Server responds with HTML + JavaScript
(“display this” and “then do that”)
3) Crawler launches Browser, feeds HTML and JavaScript
4) Browser executes the JavaScript
5) JavaScript modifies the HTML
6) Browser sends modified HTML back to Crawler
7) Crawler reads and analyzes HTML
8) Crawler finds links
9) Crawler goes back to (1) for every link found
In step (5) and (6), the JavaScript can make requests, setup user
interaction triggers, and modify the page.
This results in modified HTML that has been ‘augmented’ by the execution
of the page’s JavaScript. The modified HTML is what is fed back to the
crawler for analysis.
JavaScript as seen by Crawlers
How do crawlers handle JavaScript?
1
2
3
4 5
6
7
8
9

11. JavaScript’s role for SEOs has changed over time:
JavaScript for SEO
User tracking Dynamic (uncrawled) content Linking All Content
Last Decade Last Decade Last Years Today

12. Common JavaScript SEO Misconceptions
● “For good SEO, you can’t use any JavaScript on your website”
● “It’s magical - no need to worry about it”
● “There are no huge gains to be head in JavaScript optimisations”
● “If you have JavaScript on your website you must prerender”

13. 2 How does JavaScript affect Search Engines?
What does Google say about JavaScript?

14. We should not duck out, but rather embrace JS

15. Google IO/2018 : Deliver search-friendly JavaScript-powered websites
➔ https://www.youtube.com/watch?v=PFwUbgvpdaQ
As explained at Google IO/2018

16. How it should ideally work:
JavaScript indexing is not an instant process

17. Initial indexing:
1. Crawl page
2. Fetch server side
rendered content
First wave of Indexing
Server side rendering & initial indexing

18. Googlebot waits for
available resources to
render JavaScript by itself.
JavaScript rendering resources
Deferring JavaScript indexation
Rendering is actually deferred until Google has the resources ready to render the client
side content.
Once resources are available, Google index the content further.

19. As Googlebot might index a
page before rendering is
complete, the final render
can arrive several days later.
Second wave of Indexing
Client side rendering, final render & side effects
Once the final render done, Google performs another way of indexing on that client side
rendered content. … if your site is using a heavy amount of client side JavaScript for
rendering, you could be tripped up at times … due to … this two-phase indexing.

20. A new step in the Search Process
CRAWL
INDEX
(HTML)
RENDER
INDEX
(JS+HTML)
UPDATED SAME NEW NEW

21. 3 What is the real impact of JavaScript on SEO?
A few examples of how JavaScript can impact your SEO

22. Example #1: Products Rendered with JavaScript

23. Example #1: Product Links Rendered with JavaScript

24. Example #2: Faceted navigation with JavaScript

25. Example #2: Faceted navigation with JavaScript

26. Example #3: Content in product page with JavaScript

27. Example #3: Content in product page inserted with JS

28. Example #4: Full JavaScript
JavaScript disabled
JavaScript enabled

29. Asking the Right Questions
● Analyze differences between
HTML-only and JS-rendered
pages
● Analyze impact on website
structure (linking, content
quality…)
● Analyze JS performance
Partial JS
JS-generated elements matter
for SEO
• Analyze impact of JS on
performance
• Pre-rendering: check server-
side vs client-side rendering
Full JS
• HTML-only analysis is all that
is needed for structural and
content analysis of the
website.
• BUT worth wondering whether
these JS elements may
negatively impact Google's
crawl
Partial JS
JS-generated elements don't
matter for SEO

30. 4 How can you optimize your JavaScript for Search
Engines?
What can you do at the moment with free tools, with Botify?

31. What can you already check
and optimize?
A few tricks already at you fingertips

32. ● Missing (or Replaced) Metadata
● Incorrect HTTP Codes
● Lost (or Replaced) Canonical Tag
● Missing (or Replaced) text content
● Missing (or Replaced) inlink content
● Rendered DOM + rendering errors
Some basic checks

33. REQUIREMENTS
■ Install a stand alone Chromium 41 to fit with the Google Web Rendering Service
■ Install the Chrome Web Developer plugin & JavaScript switch
■ Open and keep a google.com page open, we’ll need it for the next steps
■ Open and keep a google Mobile-Friendly Test page open, we’ll need it for the next steps
A few requirements before running checks

34. Find a content rendered via JavaScript
Use Chrome Web Developer &
the JavaScript Switch

35. Find a URL rendered
with JavaScript =>
use Chrome Web
Developer & the
JavaScript switch
Identify visual differences on the screen

36. Identify which headings are missing

37. Identify how many links are missed

38. Fetch and Render via Google Search Console
Does rendering look correct or broken ?

39. (1st pass / no JS)
Which content indexing is being deferred?

40. Test a URL & content that should be Index
Search in Google:
site:<your_url> <unique phrase>

41. NOW we know what important LINKS and CONTENT are rendered client-side
■ Can we find an important piece of information in Google's Index ?
→ Is it relevant to assess JavaScript rendering for this website ?
■ Test: site:https://reverb.com/marketplace/acoustic-guitars "Martin D-18 Dreadnought Acoustic Guitar"
■ Results:
○ Googlebot fetched the JavaScript
○ The rendering was DONE
○ The 2-pass indexing is OK
Are important expressions in Google’s Index?

42. Find an expression that is always there in the
HTML-only version: "used from"
Go further: What proportion of the page is rendered?

43. Find an expression that is always there in the
rendered version (but not in HTML-only): "Make an
Offer"
Go further: What proportion of the page is rendered?

44. Use site: in Google with the first expression, then the second, to get the number of indexed results and
calculate the rendering ratio.
13,700 / 17,400 = 79% 2-pass indexed (rendered)
Go further: What proportion of the page is rendered?
+

45. Rendering Budget is the new
KPI
The real deal is to ease Search
Engines work

46. ● How information is fetched: several steps are needed to crawl a page
○ Crawl HTML
○ Crawl resources
○ Render (execute) is device dependant
● Expensive "cost" of rendering (CPU-intensive)
○ Crawling JavaScript content is 50 to 100 times more "expensive" for Googlebot
○ If JavaScript is also intensive for your servers (this is ALSO a caveat for websites crawlers)
● Some JS scripts are not only not relevant for SEO, but counter productive, if not penalizing
○ Even dangerous (= slow while generating no SEO value)
○ Expensive for the website server (ex: cart management, product comparison...)
○ Use robots.txt if possible
JavaScript greatly affects Search Engines

47. This graphic shows all calls use by a browser
So to be efficient on SEO we need to exclude :
● Analytics
● Cart management on e-commerce
websites
● 3rd party resources
● mouse tracking…
JavaScript crawl complexity
Simple Example

48. First Paint (FP) First meaningful Paint (FMP) Time to interactive (TTI)
Make sure your meaningful JS is loaded within 10 sec
https://developers.google.com/web/fundamentals/performance/user-centric-performance-metrics
https://gtmetrix.com/blog/first-contentful-paint-explained/
Timeline
First Contentful Paint (FCP) DOM content loaded (DCL)
73ms 3315ms 5000ms
Main Metrics

49. Understanding JS load time: number of executed resources
Identify which page
template is the most
impacted by
JavaScript.

50. Measuring Googlebot’s 2-wave process
This is a key consideration for websites where content value is related to freshness.
In your log files, you can measure the number of HTML crawls to a specific page, and the
number of API calls that populate that page via JavaScript.
Googlecrawls
Googlecrawls

51. Key Takeaways
JavaScript assessment on websites has
become mandatory - there is no ignoring it.
After Crawl Budget comes Rendering Budget
JavaScript is challenging
for Search Engines
Cost of rendering
New metrics, same method!
Time to render
Complexity of JS structure
JavaScript is new territory for
SEO managers.
Specific checks on individual
templates (by type of device)
JavaScript crawl (fine JS
settings, respect robots.txt…)
Logs analysis to analyze rendering at scale
1
2
3
4

52. Thank you
for your attention
Want to know more about
how Botify can help you
with your JavaScript?
Come talk to us at
our booth!