Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Combatting Crawl Bloat & Pruning Your Content Effectively

Wird geladen in …3

Hier ansehen

1 von 39 Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (19)

Ähnlich wie Combatting Crawl Bloat & Pruning Your Content Effectively (20)


Aktuellste (20)

Combatting Crawl Bloat & Pruning Your Content Effectively

  1. 1. Charlie Whitworth SEO Director @WhitworthSEO http://www.slideshare.net/whitworthseo Banc
  2. 2. what will we cover? bancmedia.com Crawl Bloat & Pruning @WhitworthSEO
  3. 3. beat the bloat Crawl Bloat Making search engines work too hard and not taking full advantage of your crawl budget @WhitworthSEO bancmedia.com
  4. 4. crawl definitions = Crawl Budget bancmedia.com@WhitworthSEO Crawl Rate & Crawl Demand
  5. 5. crawl rate (limit) : maximum fetching rate Search Console Limit s bancmedia.com@WhitworthSEO Website/Server Health
  6. 6. crawl demand Popularity (Traffic) Staleness @WhitworthSEO bancmedia.com How much does Google want to see your URLs?
  7. 7. Crawl budget – determined by rate & demand What Google can & wants to crawl @WhitworthSEO bancmedia.com • Empty pages • No valued added • Dupe • Soft 404 • SPAM • Hacked content
  8. 8. crawl budget “Prioritizing what to crawl, when, and how much resource the server hosting the site can allocate to crawling is more important for bigger sites, or those that auto-generate pages based on URL parameters” Gary Ilyes bancmedia.com@WhitworthSEO
  9. 9. how to identify crawl bloat bancmedia.com@WhitworthSEO
  10. 10. how to identify crawl bloat bancmedia.com@WhitworthSEO
  11. 11. why is this a problem? bancmedia.com@WhitworthSEO Don’t work them too hard Search engine bots hate crawling rubbish
  12. 12. why is this a problem? Engagement metrics are key especially now we’re “mobile first” @WhitworthSEO bancmedia.com
  13. 13. examples of rubbish: search result URLs /search/ bancmedia.com@WhitworthSEO /sessionkey= /search-results?q=
  14. 14. examples of rubbish: paginated URLs bancmedia.com rel=next/prevnoindex @WhitworthSEO
  15. 15. examples of rubbish: faceted navigations json-ld disallow parameters bancmedia.com@WhitworthSEO noindex
  16. 16. who’s seen this?
  17. 17. cut the crap Get Rid Of It @WhitworthSEO bancmedia.com
  18. 18. plan
  19. 19. noindex <meta name=“robots" content=“noindex"> Won’t deal with crawl bloat in the short term Still an effective tactic bancmedia.com@WhitworthSEO
  20. 20. url parameters @WhitworthSEO Powerful tool when used properly No need for developer support bancmedia.com
  21. 21. xml sitemaps Redirected URLs Noindexed URLs Poor quality pages @WhitworthSEO bancmedia.com
  22. 22. robots.txt bancmedia.com@WhitworthSEO Exercise Caution! Powerful crawl bloat tool when used effectively
  23. 23. robots.txt Blocking Crawl Paths – Stems Authority Flow bancmedia.com@WhitworthSEO Google needs to see tags to honour them!
  24. 24. google’s hint Google gives you all you need to monitor your results All these features are your friend s @WhitworthSEO bancmedia.com
  25. 25. crawl stats Great way to gauge your efforts @WhitworthSEO bancmedia.com
  26. 26. bancmedia.com@WhitworthSEO Pruning the prune
  27. 27. pruning – the fun bit SEO Housekeepin g Now you have your crawl under control, show off your best bits bancmedia.com@WhitworthSEO
  28. 28. pruning – why? • Spinning • Excessive blogging • Sparse articles @WhitworthSEO bancmedia.com • Over optimisation • Saturation
  29. 29. stage one – content audit/content is king Content auditing still essential nonetheless bancmedia.com@WhitworthSEO SEO’s most annoying cliché
  30. 30. stage two: the prune 410 – Gone @WhitworthSEO bancmedia.com For pages you want gone forever
  31. 31. stage two: the prune bancmedia.com@WhitworthSEO Won’t help crawl bloat short term Will disappear eventually*
  32. 32. stage two: the prune Crawl Bloat Authority Indexation @WhitworthSEO bancmedia.com
  33. 33. stage two: the prune NoIndex Will be crawled less over time ensures a high quality SER P For duplicate content, cannibalisation and API style pages @WhitworthSEO bancmedia.com
  34. 34. stage two: canonical For duplicate content, cannibalisation and API style pages @WhitworthSEO bancmedia.com canonical
  35. 35. stage three: content repurposing/enhancement @WhitworthSEO bancmedia.com Evergreen Seasonal Long form Research
  36. 36. search quality guidelines @WhitworthSEO bancmedia.com Search Quality Evaluator Guidelines
  37. 37. stage four: the finishing touch Link Profile Analysis Links pointing at excluded pages Usual disavow request @WhitworthSEO bancmedia.com Don’t ruin all your hard work with a crap link profile
  38. 38. crawl bloat and pruning checklist Does your site have crawl bloat? Choose the appropriate fixes for your site Prune your remaining content Continually improve and enhance @WhitworthSEO bancmedia.com
  39. 39. final tips: thanks for coming @WhitworthSEO bancmedia.com Thanks for Listening! 2 Tips