SEO for Progressive Web Apps (PWA) and JS Frameworks

SEO for PWA
What PWA are and how to SEO-optimize them
By Christian Oliveira & Natzir Turrado

What we are going to learn
1. What is a PWA and how it works
2. How search engines deal with Javascript (JS)
3. Rendering approaches for JS sites
4. Good practices for PWA and JS sites
5. Experiment: PWA without prerender (CSR)
6. Conclusion

1. What is a PWA and how it works

Native Apps
$ Longer funnel
Fewer and more
expensive developers
Support for different
versions Device support
Investment in another
channel
More time invested in
development
Native And PWA: Choices, Not Challengers! - https://goo.gl/fq6X2E

Native Apps
How Much Does it Cost to Develop and Build an App - https://goo.gl/uQmoQm
SIMPLE APP BBDD/API APP
MULTI-FEATURE
/ ENTERPRISE
GAME
$10K $10K-50K $50K-150K $5K-250K
2-4 weeks 2-3
months
3-6
months
Varies a
lot

Native Apps
How Much Does it Cost to Develop and Build an App - https://goo.gl/uQmoQm
USER PROFILES IN-APP
PURCHASES GEO-LOCATION
SYNC DATA
ACROSS DEVICES
$7,5K-9,5K $5,5K-15K $7,5K-17K $5,5K-7,5K
EMAIL LOGIN SOCIAL LOGIN SOCIAL
INTEGRATION
RATING SYSTEM
$5,5K-7,5K $2K-3K $2K-15K $5,5K-20K

Native Apps
“In a consumer
mobile app, every
step you make a user
perform before they
get value out of
your app will cost
you 20% of users.”
Gabor Cselle, PM at @Google
Every Step Costs You 20% of Users - https://goo.gl/QyaHUn

Mobile Web APPs
$ Browser support
Not indexable on App
Stories
Difficult RWD
Device support
No off-line navigation
Web Development Cost / Rate Comparison - https://goo.gl/tQdXuZ

Progressive Web APPs (PWA)
“App Experience”
Faster transitions and
purchases
Push notifications
Offline navigation
Icon on home screen
No installation
Easily linkable and
shareable
Indexable on search
engines
Unified experience
Indexable on App stories
Progressive Web APPs - https://goo.gl/jrSk5u
Progressive and
Responsive

PWA use Javascript (JS) libraries and frameworks
Huge community
Safer
Cheap / open-sourceEasier development
Better
performance

Top 5 frameworks, libraries and
other technologies
PWA use Javascript (JS) libraries and frameworks
Stack Overflow Developer Survey '17 - https://goo.gl/P8S1X7
Top 5 programming languages

How a PWA works
{i}
JSON
S/MPA Service
Worker

How a PWA works
Service
Worker
Beyond single-page apps: alternative architectures for your PWA (Google I/O '18) - https://youtu.be/X6yof_vIQnk
Cache

How a PWA works
Service
Worker
req
Cache
req

How a PWA works
Service
Worker
Cache
req
req

How a PWA works
Service
Worker
Cache
req
res

How a PWA works
{i}
JSON
Web App Manifest - https://goo.gl/SB2ur3

PWAs: building bridges to mobile, desktop, and native (Google I/O '18) - https://youtu.be/NITk4kXMQDw
Browsers support for PWA features
✔ ✔ ✔ ✔ ✔ ✔
✔ ✔ ✔ ✔ ✔ ✔ ✔
✔ ✔ ✔ ✔ ✔ ✔ ✔
✔ ✔
✔ ✔ ✔ ✔
✔ ✔
Add to home screen
Service Workers
Push notifications
Credential
Management API
Payment
Request API
Meta Theme Color
Supported Beta Development✔ ✔

PWA and speed
Better first interaction
performance and even
better after first
interaction
The Definitive Guide to Ecommerce Performance - https://goo.gl/YVUngH

PWA and speed
35% faster purchases
49% search

PWA+AMP and speed
Almost instant load from
Search Engines

PWA+AMP and speed
Service Worker loads on background
<amp-install-serviceworker>

PWA+AMP and speed
User enters the PWA

Google/SOASTA Research '17 - https://goo.gl/o1MWEK
The importance of performance
1s - 3s the probability of bounce increases 32%
As page load time goes from:

Speed Matters Vol.3 - https://goo.gl/QM8cM4 & Ericsson ConsumerLab. Neurons Inc. 2015 - https://goo.gl/NyQd4Z
Page load speedResponsive
How attractive
the site looks
Findability
The importance of performance
What is more important for users?
Simplicity
24 % 58 % 61 % 66 % 75 %
Slow websites cause
more stress than
horror movies

2. How search engines deal with Javascript (JS)

Only Google and ASK render JS, Baidu tries
Baidu Adds New Spider with Rendering Job - https://goo.gl/Jd79qW

Crawling =! Rendering =! Indexing =! Ranking
49
indexed pages
28indexed pages
17
indexed pages
30indexed pages

JS rendering costs A LOT
The Cost of JavaScript - https://goo.gl/6jwY3U
22s35s 14s
10% sites 25% sites 50% sites
Time To Interaction (TTI)
on mobile devices
170 KB of JS
!==
170 KB of JPEG

JS rendering costs A LOT
“To execute JS for every page at
our scale [would] require
10,000–15,000 servers, and we
believe our customers are not
ready to pay for that yet.”
Dmitry Gerasimenko, CEO Ahrefs
Ahrefs crawlers are now rendering web pages and executing JavaScript - https://goo.gl/DQuero
~173 urls per second per server
~0,86 urls per second per server
Ahrefs every 24 hours:
● Crawl 6 billion pages
● Render 30 million JS pages
● Which result in 250 million JS links
● With 400 servers

Google won’t wait indefinitely to render a page
5’
Time Google waits according to our own experiments,
confirmed with other SEOs
There is no official time - https://goo.gl/MhNXdd
’

Google won’t wait indefinitely to render a page
5’
Time Google waits according to our own experiments,
confirmed with other SEOs
There is no official time - https://goo.gl/MhNXdd
’
Use the snapshot of
the Mobile Friendly
Tool as a guide

Google renders JS in 2 phases
CRAWL
INDEX
RENDER
Instant, 1st
wave of indexing
2nd wave of
indexing
New links to
be crawled
As rendering
resources become
available
Deliver search-friendly JavaScript-powered websites (Google I/O '18) - https://youtu.be/PFwUbgvpdaQ

CRAWL
INDEX
RENDER
Instant, 1st
wave of indexing
2nd wave of
indexing
New links to
be crawled
As rendering
resources become
available
Google defers
rendering because
of lack of
resources

CRAWL
INDEX
RENDER
Instant, 1st
wave of indexing
2nd wave of
indexing
New links to
be crawled
As rendering
resources become
available
Reads canonical, amphtml and
https status only on initial
fetch

CRAWL
INDEX
RENDER
Instant, 1st
wave of indexing
2nd wave of
indexing
New links to
be crawled
As rendering
resources become
available
If Noindex, Hreflang and
rel=next/prev are not on the
html, will read them on the
rendered page

CRAWL
INDEX
RENDER
Instant, 1st
wave of indexing
2nd wave of
indexing
New links to
be crawled
As rendering
resources become
available
Google uses Chrome 41 for
rendering. It is done by
caffeine with a WRS subsystem

Types of events and links Google follows
Google crawls, index and pass PageRank to plain html links (<a href=...)
● <a href="/this-link"> will be crawled </a>
● <a href="/this-link" onclick="changePage('this-link')"> will be crawled </a>
● <a onclick="changePage('this-link')"> Not crawled </a>
● Not crawled

https://goo.gl/rRPWcC & https://goo.gl/aZaECU & https://goo.gl/aZaECU & https://goo.gl/j1ccxF
BUT: Google crawls and index anything that looks like a link, in order to
discover, although it won’t pass PageRank:
● <a onclick="changePage('/this-link')"> will be crawled </a>
● will be crawled 
● /this-link will be crawled

https://goo.gl/rRPWcC & https://goo.gl/aZaECU & https://goo.gl/aZaECU & https://goo.gl/j1ccxF
BUT: Google crawls and index anything that looks like a link, in order to
discover, although it won’t pass PageRank:
● <a onclick="changePage('/this-link')"> will be crawled </a>
● will be crawled 
● /this-link will be crawled
Onscroll events are followed sometimes, onmouseover events are not followed. If
you have pagination which works with onscroll it may get indexed (only the first
pages)
Knowing what kind of events and links Google follows is important because all
indexed URLs are taken into account to evaluate the site’s quality

3. Rendering approaches for JS sites

Client Side Rendering (CSR)
req
res: JS
JS+JSON=HTML req
res: JSON

Server Side Rendering (SSR)
JS+JSON
= HTML
req
res: HTML

Hybrid Rendering
Building a hybrid-rendered PWA - https://goo.gl/ZNfzr1
SSR
CSR

Dynamic Rendering (DR)
WEB
SERVER
Users
Googlebot
and others
CSR or Hybrid
rendering
SSR

WEB
SERVER
Users
Googlebot
and others
CSR or Hybrid
rendering
SSR
User-agent or
Reverse DNS

Googlebot
Fetch & Render As Any Bot - https://goo.gl/1augcc

Fetch & Render As Any Bot - https://goo.gl/1augcc
UsersGooglebot

What is best for users?
Netflix and SSR - https://goo.gl/ym6fWN

Walmart and SSR - https://goo.gl/EwmPyD
SSR CSR
FCP >

Walmart and SSR - https://goo.gl/EwmPyD
SSR CSR
TTFB <

What is best for IT?
The Benefits of Server Side Rendering Over Client Side Rendering - https://goo.gl/EwmPyD
CSR

What Google recommends?
JS features
Type of website > Chrome 41 < Chrome 41
Small and static SSR / DR CSR
Small and dynamic SSR / DR CSR / SSR / DR
Big and static SSR / DR SSR / DR
Big and dynamic SSR / DR SSR / DR

What Google recommends?
Barry Adams’ tweet - https://goo.gl/umSS1j

4. Good practices for PWA and JS sites

PWA and performance audit
https://developers.google.com/web/tools/lighthouse/ & https://sonarwhal.com

Mobile-Friendly Test / Screenshot
Mobile-Friendly Test - https://goo.gl/FjHMk5

Mobile-Friendly Test / Rendered HTML

Mobile-Friendly Test / Page loading issues

Fetch and Render Fetch & Render is less
tolerant waiting to
render than the real WRS
subsystem

Fetch and Render on any domain
Fetch and Render of any domain - https://github.com/screamingfrog/fetch-and-render
Add this meta to
the original code!

Fetch and Render on any domain
https://www.analistaseo.es/fetch-and-render/
?url=https://www.ejemplo.com&type=mobile
Fetch and Render of any domain - https://github.com/screamingfrog/fetch-and-render

Fetch and Render on staging
1. Noindex staging
2. Reverse DNS

Crawl and compare staging with production
- https://www.fandangoseo.com/

Crawl emulating mobile and JS rendering
- https://www.fandangoseo.com/

Check meta tags with other user agents
Meta SEO inspector + User-Agent Switcher - https://goo.gl/tNo3ug & https://goo.gl/7r7rcn
Google: User-Agent String: Mozilla/5.0 (compatible;
Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus
5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible;
Googlebot/2.1; +http://www.google.com/bot.html)
Facebook: facebookexternalhit/1.1
(+http://www.facebook.com/externalhit_uatext.php)

Check meta tags with other user agents
Meta SEO inspector + User-Agent Switcher - https://goo.gl/tNo3ug & https://goo.gl/7r7rcn

Browse without JS with other user agents
Web Developer - https://goo.gl/WqjjLa

UsuariosGooglebot

Check console errors and links on Chrome 41
Descarga Chrome 41 - https://goo.gl/3tpxpH

☑
☑
☑
☑
☑
☑
☑
☑
☑
SEO+PWA Checklist for CSR
1. Canonicals + metadata do not depend on CSR
2. Page renders in less than 5 seconds
3. Page loads and is functional with Chrome 41
4. Rendered links are <a> elements with their real href
5. HTML+design is the expected on Mobile Friendly Tool
6. HTML+design is the expected on Fetch and Render with Search
Console
7. Pages appear on Google after searching for specific text strings
8. Content is not replicated in different pages
9. Fragments (!#)are not used and every page has its own URL

☑
☑
☑
☑
☑
☑
☑
☑
SEO+PWA Checklist for SSR / Dynamic Rendering
1. SSR is served to Googlebot and other specific bots (FB, TW…)
2. Content Googlebot receives is the same a normal user sees
3. Http response codes are the same in both cases
4. Meta tags are the same in both cases
5. Server is able to manage the rendering under high pressure
6. SSR version includes all the content and links in a correct way
7. All pages have the expected behaviour in both cases
8. Fragments (!#)are not used and every page has its own URL

☑
☑
☑
☑
☑
☑
☑
☑
☑
☑
☑
☑
☑
PWA Checklist UX & WPO
Progressive Web App Checklist - https://goo.gl/jGj4D2
1. Site is served over HTTPS
2. Pages are responsive on tablets & mobile devices
3. All app URLs load while offline
4. Metadata provided for Add to Home screen
5. First load very fast even on 3G
6. Page transitions don't feel like they block on the network
7. Site uses cache-first networking
8. Site appropriately informs the user when they're offline
9. Content doesn't jump as the page loads
10. Pressing back from a detail page retains scroll position on the
previous list page
11. When tapped, inputs aren't obscured by the on screen keyboard
12. Content is easily shareable from standalone or full screen mode
13. Any app install prompts are not used excessively

5. Experiment: PWA without prerender (CSR)

Experiment: PWA without prerender (CSR)
Worona - https://www.worona.org/
Before: classic
wordpress
After: PWA with
Worona (CSR)

Experiment: PWA without prerender (CSR)
CSR = No content
if JS is not
executed

Test 1: rendering
Page renders on
Mobile Friendly
Tool
Page renders on
Search Console
Google renders and
index the page

Test 1: rendering
Page renders on
Mobile Friendly
Tool
Page renders on
Search Console
Google renders and
index the page
15’ after
“Request
indexing” on SC

Test 1: rendering
Observations:
● Google has been able to render and index the content
● The process has been relatively fast (15’)

Test 2: rankings
Observations:
● Rankings haven’t been lost after migrating to a PWA
without SSR

Test 3: new content indexing
We publish a new
post
We force the blog
homepage to be
reindexed

4 days after it
has not been
indexed

Observations:
● If we don’t force the indexing, Google takes some time to
index new content

Test 4: the two waves of indexing
We add a made up
word only on the html
version of the page

We use “Fetch” (not
rendering) in SC + Request
Indexing

Minutes later… Google
reindex the page and
associate the word to it

On the other side, Google’s
cache for this page was not
updated

And the page keeps ranking for
text strings from the previous
version

Days after, Google updated the
cache with the rendered page

And, from that moment, the
page no longer ranks for the
made up word

Despite the fact that if we
check the “Only text” version
of the cache, the word is there

Observations:
● Before rendering the page Google has indexed the content
on the page without JS (first wave)
● This new content has been associated to the previous
version of the web (the one without PWA)
● Once rendered, the page only ranks for the content
present on the rendered version (second wave indexing)

● Up to date very few websites have migrated directly
to a CSR PWA (no pre-rendering) for all
user-agents. Those who did and lost traffic, had
also other technical problems (e.g. hulu.com)
● New websites which are created directly as CSR PWA
can rank, but they have problems with indexing
● Up to date, and regarding Google, Dynamic Rendering
is the solution that works without any problems
Conclusions

SEO for PWA
THANKS!
Natzir Turrado
analistaseo.es
hola@natzir.com
@natzir9
Christian Oliveira
christianoliveira.com
christian@usal.es
@christian_wilde

SEO for Progressive Web Apps (PWA) and JS Frameworks

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Kürzlich hochgeladen

Kürzlich hochgeladen (9)

Empfohlen

Empfohlen (20)

SEO for Progressive Web Apps (PWA) and JS Frameworks