Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
The	Search	Universe	Saga	-
Links,	Log	Files,	GSC	and	everything	
in	between.
Michal	Magdziarz
CEO	&	Founder
DeepCrawl
Ista...
@DeepCrawl @Magdziarz	#digitalzone
179,769,000,000,000,000,000,000,000,000,000,000,000,0
00,000,000,000,000,000,000,000,00...
The	Search	Universe	limits
[scheme]://[domain-name].[TLD]/[path-name]
4-5	chars
max	253	chars
(37	allowed	chars)
max	2083	...
The	Search	Universe	formula
Sum[(37^k)*	1002	*		Sum[(74^j),	{j,	0,	2071-k}],	{k,	2,	248}]
@DeepCrawl @Magdziarz	#digitalzo...
179,769,000,000,000,000,000,000,000,000,000,000,000,0
00,000,000,000,000,000,000,000,000,000,000,000,000,00
0,000,000,000,...
The	Search	Universe	formula
Sum[(37^k)*	1002	*		Sum[(74^j),	{j,	0,	2071-k}],	{k,	2,	248}]
@DeepCrawl @Magdziarz	#digitalzo...
Total	potential	number	of	URLs
@DeepCrawl @Magdziarz	#digitalzone@DeepCrawl @Magdziarz	#digitalzone
78473766246985183923814815742482105696053525424342762703234779816549659018650988449707938454508725013166558473232302902096...
@DeepCrawl @magdziarz
10,000,000,000,000,000,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000,000,000,...
@DeepCrawl @magdziarz
1
The	Googol	number
@DeepCrawl @Magdziarz	#digitalzone
I want to tell you a story
Starring:
Mr Deepson as The URLSimple Story about an URL in Search Universe
Starring:
Me as The URL
Also	Starring…
as	the	Backlink	Source
as	the	Traffic	Source
as	the	Log	Summary	Source
as	the	Consumer	Source
@DeepCrawl @M...
Where	do	backlinks	land	on	your	site?
404 myurl.com/my-page 1000s	of	Backlinks!
Backlinks	to	broken	pages
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#digitalzone
Backlinks	to	broken	pages
@DeepCrawl @Magdziarz	#digitalzone
How	is	backlink	authority	distributed?
myurl.com/my-page 1000s	of	Backlinks!
Backlinks	to	a	nofollow page
@DeepCrawl @magdziarz
myurl.com/page-1
myurl.com/page-4...
@DeepCrawl @magdziarz
Backlinks	to	low	DeepRank/high	level	pages?
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#digitalzone
myurl.com/my-page 1000s	of	Backlinks!
@DeepCrawl @magdziarz
Where	do	backlinks	land	on	your	site?
Just	1	link	out
@DeepCra...
Disallowed myurl.com/my-page 1000s	of	Backlinks!
Non-indexable	pages	with	backlinks
@DeepCrawl @magdziarz@DeepCrawl @Magdz...
Backlinks	to	duplicate	pages
@DeepCrawl @magdziarz
myurl.com/my-page 876	Backlinks
Backlinks	to	duplicate	pages
myurl.com/my-page.html 456	Backlinks
Duplicates Canonical
456...
@DeepCrawl @magdziarz
What	can	Google	Search	Console	add?
URL	story	with	Google	Search	Console
myurl.com/my-page TrafficDisallowed
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#digit...
Find	poor	experience	pages	with	traffic
myurl.com/my-page
5,320	Clicks
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#digital...
Understand	how	bots	crawl	with	different	user	agents
Understand	how	bots	crawl	with	different	user	agents
@DeepCrawl @magdziarz
Responsive Dynamic Separate	Mobile
Would	expect...
Impact	of	Mobile-first
@DeepCrawl @magdziarz
-CRITICAL- - Desktop	only	(Heavily	Affected)
-HIGH	PRIORITY- - Desktop	+	Dedi...
Order	of	Mobile-first	Indexing
@DeepCrawl @magdziarz
Responsive
Page
AMP
Page
Mobile
Page
Rel=canonical
Rel=alternate amp
...
Order	of	Mobile-first	Indexing
@DeepCrawl @magdziarz
Dynamic
Mobile Page
Dynamically served
INDEXABLE
Responsive
Page
AMP
...
Architecture:	Internal	Site	Architecture
Desktop	Site	Architecture	
www.domain.com
Mobile	Site	Architecture
m.domain.com
O...
Pages	with	image	search	traffic,	with	broken	images
myurl.com/my-page
5,320	Clicks
@DeepCrawl @magdziarz@DeepCrawl @Magdzi...
Traffic	to	orphaned	pages
@DeepCrawl @magdziarz
Maintaining	Value	Through	Migrations
myurl.com/page.aspx
myurl.com/c/page
301 redirect
myurl.com/page
404
301 redirect
1 R...
How	is	crawl	budget	distributed/utilized?
@DeepCrawl @magdziarz
Identify	where	crawl	budget	is	getting	WASTED
@DeepCrawl @magdziarz
Indexable Pages	
without
Googlebot
requests
Non-indexa...
myurl.com/non-indexable 100	requests
Identify	crawl	budget	wastage
myurl.com/broken 330	requests
myurl.com/page?sort=price...
myurl.com/sitemap_index.xml 2,300	requests
Determine	most	crawled	site	sections
myurl.com/categories 330	requests
myurl.co...
Now	it	gets	fun!	
myurl.com/dress
myurl.com/ dress?colour=red
2,768 Visits
1000s of Backlinks
50 Googlebots
2,423 Clicks
@...
@DeepCrawl @magdziarz
All	well	&	good	knowing	what	Google	sees…
@DeepCrawl @magdziarz
But	don’t	forget	about	consumer	power
@DeepCrawl @magdziarz
Google	drives	awareness	but	people	drive	ROI!
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#digitalzone
What	do	visitors	see	and	convert	on?
@DeepCrawl @magdziarz
A	simple	URL	story	with	visits
Soft-200 myurl.com/my-page 1000s	of	Visits!
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#dig...
A	simple	URL	story	with	visits
Blog	post myurl.com/my-page 1000s	of	Visits!
@DeepCrawl @magdziarz
No	conversion	
points
@D...
An	advanced	story	with	visits	&	backlinks
Disallowed
myurl.com/my-page
2768	Visits!
301	redirection
myurl.com/redirect-pag...
A	Simple	Story	about	a	URL
. That’s	me
A	Simple	Story	about	a	few	URL
..
.
.
.
.
.
A	Story	about	a	few	more	URLs
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
A	Story	about	even	more	URLs
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
....
…hidden	among	the	others
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
...
78473766246985183923814815742482105696053525424342762703234779816549659018650988449707938454508725013166558473232302902096...
Merging	Excel	spreadsheets
@DeepCrawl @Magdziarz	#digitalzone
Automation	needed!
@DeepCrawl @magdziarz
Scaling	up
1000	URLs
100k+	URLs
1M+	URLs
@DeepCrawl @magdziarz@DeepCrawl @Magdziarz	#digitalzone
Your	search	universe!
@DeepCrawl @magdziarz
The	Ultimate	Search	Universe
@DeepCrawl @magdziarz
Thank	you	Istanbul!
@DeepCrawl @magdziarz
One	more	thing	…
• Providing	clarity	with	data
• Educating	the	market
• You	at	the	forefront	of	this	
development
Taking	part	in	this	journ...
Ilyas Teker
DeepCrawl CAB	Member
Michal Magdziarz @Digitalzone - The Search Universe - Links, Log Files, GSC and Everything in Between
Michal Magdziarz @Digitalzone - The Search Universe - Links, Log Files, GSC and Everything in Between
Michal Magdziarz @Digitalzone - The Search Universe - Links, Log Files, GSC and Everything in Between
Michal Magdziarz @Digitalzone - The Search Universe - Links, Log Files, GSC and Everything in Between
Nächste SlideShare
Wird geladen in …5
×

Michal Magdziarz @Digitalzone - The Search Universe - Links, Log Files, GSC and Everything in Between

8.835 Aufrufe

Veröffentlicht am

Michal Magdziarz was at Digitalzone stage on October 20, 2017 and made a presentation about the search universe!

Veröffentlicht in: Präsentationen & Vorträge
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Michal Magdziarz @Digitalzone - The Search Universe - Links, Log Files, GSC and Everything in Between

  1. 1. The Search Universe Saga - Links, Log Files, GSC and everything in between. Michal Magdziarz CEO & Founder DeepCrawl Istanbul – October 2017
  2. 2. @DeepCrawl @Magdziarz #digitalzone 179,769,000,000,000,000,000,000,000,000,000,000,000,0 00,000,000,000,000,000,000,000,000,000,000,000,000,00 0,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000,0 00,000,000,000,000,000,000,000,000,000,000,000,000,00 0,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000,0 00,000,000,000,000,000,000,000,000,000 The Search Universe number @DeepCrawl @Magdziarz #digitalzone
  3. 3. The Search Universe limits [scheme]://[domain-name].[TLD]/[path-name] 4-5 chars max 253 chars (37 allowed chars) max 2083 chars (74 safe chars) 1002 delegated TLDs @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  4. 4. The Search Universe formula Sum[(37^k)* 1002 * Sum[(74^j), {j, 0, 2071-k}], {k, 2, 248}] @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  5. 5. 179,769,000,000,000,000,000,000,000,000,000,000,000,0 00,000,000,000,000,000,000,000,000,000,000,000,000,00 0,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000,0 00,000,000,000,000,000,000,000,000,000,000,000,000,00 0,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000,0 00,000,000,000,000,000,000,000,000,000 The Search Universe number @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  6. 6. The Search Universe formula Sum[(37^k)* 1002 * Sum[(74^j), {j, 0, 2071-k}], {k, 2, 248}] @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  7. 7. Total potential number of URLs @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  8. 8. 78473766246985183923814815742482105696053525424342762703234779816549659018650988449707938454508725013166558473232302902096811504404934076854307 77688870625168655782985431595150681477406513213395328771032817979822070623067880021557858582442427033738131665029996349133129288380874567410864 14034991347610789451621292215819421206230373165721570143065828756021435250420465590663912250173143392605384240521094509022523540697440824778264 42325264535976865048908831206200250295415225957848511039478407125212449608219704733094898718061393236285114781093048478937100163386870742309944 80382819814211559774426079984237004971410153332165398745553347463752768231651773514730654797031848298991140733872292366951092618400047899836044 53324991340941655648287702893018982019830434132160648353081081005848080709448555461376282508130708715287823006991789011101013014065967376808332 96973097311126033510919532364799966469108619264414565566429780915777075549446895450854999125034884132489494209990933896344115147084948730931376 42745334512172794166215863679650206883098883226412412533869285362147086071202117736490211788264888681452274934359153841488546263138401896337683 58538905027008087006841813556572033860966221049238231833733946565380386244124984109688966961648437100420381824077956889191290810166325049291081 35695965659485481190832742158152630735369805475905872022568836964293750528524111465179017254098375891444809395423322623381562374364064921804515 37611090180112564430616802952992301714416864531622716518682056952812530620637026909406083100490077402623699951457778075196454447227399298046478 37095500046983063332876102471983151557113494680741150705304549419006358172575934724486579166905034524117854803988871731643331832172215688107182 45423534289787269897692584333049692487978120133186645481716634035397650655531305055142287963190066195517600937052405614826457032618343115494036 44979918401507977556406613935843475679482599662222428774537045879460603574407941335915394815613528418189105789957261820512794038777966033755777 04655607957739037822676397174110366865834093778720767859576404947254163985066830821471838933850390655177324561337184584023980676711820580024727 86972907521894417652488622748959256739821148835038690304583144731721405546764947564898188409956097030768277988336831491610645233375082689663608 01478282219295476114638361981356777566591420546662090432041262893349177973606719017086614159819654371133395545627161949589824931165383477955452 78511627142351880721678464572361476663390962861071847525999474193850600711465076544469329807817220619380631187767945863516056151401186321613983 58383396729985267774251335092327970967184993329770237244313978111628580746451883790317371602234293618658518504247532513866537589207156379867202 39086772979960209708861948184281890071182723819202470880426982866185067084325025580477788198921379069735555849106745009300190834002649919903577 91769325024233390912498433688283756884535816171834060933902179741409280466261635754274996172731978825221837596709530618949259329507953138896244 72786356814088366055177878307043317299892732643083235819969046555286901425714428492632296658100838971525219414156350815328246795161595692081670 11293103975331302623420352543498604231833236255337260488073591957033032737060848219299552982268131634972997775653590402337186319344193126688466 66541453172142434828508980807887921061235992632892651036902166658963526289963465032742747060724259120518645721843238959291710203690370798848425 08041313605830741852392486897104194949423244475931177116494727605862918047498560295178911026354218664928746914791090454022683841437711316466551 21662329835914851717789536069184550924058466980065597086830187256141634826947112736078964945243398768914579306351485433434435725789132374523602 17337958300907091238202126481680666289211018573484312786272606967133344066848518380084707242221409750271173333084049965367203904578 Total potential number of URLs 7.95636796670821… x 103848 @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  9. 9. @DeepCrawl @magdziarz 10,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000 ,000,000,000,000,000,000,000,000 The Googol number 10^100 @DeepCrawl @Magdziarz #digitalzone
  10. 10. @DeepCrawl @magdziarz 1 The Googol number @DeepCrawl @Magdziarz #digitalzone
  11. 11. I want to tell you a story
  12. 12. Starring: Mr Deepson as The URLSimple Story about an URL in Search Universe
  13. 13. Starring: Me as The URL
  14. 14. Also Starring… as the Backlink Source as the Traffic Source as the Log Summary Source as the Consumer Source @DeepCrawl @Magdziarz #digitalzone
  15. 15. Where do backlinks land on your site?
  16. 16. 404 myurl.com/my-page 1000s of Backlinks! Backlinks to broken pages @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  17. 17. Backlinks to broken pages @DeepCrawl @Magdziarz #digitalzone
  18. 18. How is backlink authority distributed?
  19. 19. myurl.com/my-page 1000s of Backlinks! Backlinks to a nofollow page @DeepCrawl @magdziarz myurl.com/page-1 myurl.com/page-4 Meta Nofollow @DeepCrawl @Magdziarz #digitalzone
  20. 20. @DeepCrawl @magdziarz Backlinks to low DeepRank/high level pages? @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  21. 21. myurl.com/my-page 1000s of Backlinks! @DeepCrawl @magdziarz Where do backlinks land on your site? Just 1 link out @DeepCrawl @Magdziarz #digitalzone
  22. 22. Disallowed myurl.com/my-page 1000s of Backlinks! Non-indexable pages with backlinks @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  23. 23. Backlinks to duplicate pages @DeepCrawl @magdziarz
  24. 24. myurl.com/my-page 876 Backlinks Backlinks to duplicate pages myurl.com/my-page.html 456 Backlinks Duplicates Canonical 456 Backlinks! 1332 Backlinks! @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  25. 25. @DeepCrawl @magdziarz
  26. 26. What can Google Search Console add?
  27. 27. URL story with Google Search Console myurl.com/my-page TrafficDisallowed @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  28. 28. Find poor experience pages with traffic myurl.com/my-page 5,320 Clicks @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  29. 29. Understand how bots crawl with different user agents
  30. 30. Understand how bots crawl with different user agents @DeepCrawl @magdziarz Responsive Dynamic Separate Mobile Would expect Google to use both user- agents on the same URL to validate that the same content is returned. Google needs to crawl with both user agents to validate the mobile version. Hint: Use the Vary HTTP header! Google needs to crawl the dedicated mobile URLs with a mobile user agent to validate the pages and confirm the content matches the desktop pages. @DeepCrawl @Magdziarz #digitalzone
  31. 31. Impact of Mobile-first @DeepCrawl @magdziarz -CRITICAL- - Desktop only (Heavily Affected) -HIGH PRIORITY- - Desktop + Dedicated Mobile (Affected) -HIGH PRIORITY- - Dynamic (Affected e.g. Content Issues) -MEDIUM PRIORITY- - Responsive (Not affected) Impact of Mobile-first on different Mobile Configurations:
  32. 32. Order of Mobile-first Indexing @DeepCrawl @magdziarz Responsive Page AMP Page Mobile Page Rel=canonical Rel=alternate amp NON-INDEXABLE NON-INDEXABLE Rel=canonical Rel=alternate mobile Responsive Page Mobile Page INDEXABLE INDEXABLENON-INDEXABLE GOOGLE SERP @DeepCrawl @Magdziarz #digitalzone
  33. 33. Order of Mobile-first Indexing @DeepCrawl @magdziarz Dynamic Mobile Page Dynamically served INDEXABLE Responsive Page AMP Page Rel=canonical Rel=alternate amp NON-INDEXABLE Responsive Page INDEXABLE GOOGLE SERP @DeepCrawl @Magdziarz #digitalzone NON-INDEXABLE
  34. 34. Architecture: Internal Site Architecture Desktop Site Architecture www.domain.com Mobile Site Architecture m.domain.com Orphaned Mobile Pages 404
  35. 35. Pages with image search traffic, with broken images myurl.com/my-page 5,320 Clicks @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  36. 36. Traffic to orphaned pages @DeepCrawl @magdziarz
  37. 37. Maintaining Value Through Migrations myurl.com/page.aspx myurl.com/c/page 301 redirect myurl.com/page 404 301 redirect 1 Reuqest 1 Reuqest 1 Reuqest 2768 Visits 2768 Visits 2768 Visits 500 Links @DeepCrawl @Magdziarz #digitalzone
  38. 38. How is crawl budget distributed/utilized? @DeepCrawl @magdziarz
  39. 39. Identify where crawl budget is getting WASTED @DeepCrawl @magdziarz Indexable Pages without Googlebot requests Non-indexable Pages with Googlebot requests Indexable Pages with Googlebot requests @DeepCrawl @Magdziarz #digitalzone
  40. 40. myurl.com/non-indexable 100 requests Identify crawl budget wastage myurl.com/broken 330 requests myurl.com/page?sort=price 56 requests @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  41. 41. myurl.com/sitemap_index.xml 2,300 requests Determine most crawled site sections myurl.com/categories 330 requests myurl.com/products 232 requests @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  42. 42. Now it gets fun! myurl.com/dress myurl.com/ dress?colour=red 2,768 Visits 1000s of Backlinks 50 Googlebots 2,423 Clicks @DeepCrawl @magdziarz myurl.com/ dress?colour=blue myurl.com/ dress?colour=green myurl.com/ dress?colour=green&price=1000+ Meta Canonical myurl.com/ dress?colour=…&price=… (…) @DeepCrawl @Magdziarz #digitalzone
  43. 43. @DeepCrawl @magdziarz
  44. 44. All well & good knowing what Google sees… @DeepCrawl @magdziarz
  45. 45. But don’t forget about consumer power @DeepCrawl @magdziarz
  46. 46. Google drives awareness but people drive ROI! @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  47. 47. What do visitors see and convert on? @DeepCrawl @magdziarz
  48. 48. A simple URL story with visits Soft-200 myurl.com/my-page 1000s of Visits! @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  49. 49. A simple URL story with visits Blog post myurl.com/my-page 1000s of Visits! @DeepCrawl @magdziarz No conversion points @DeepCrawl @Magdziarz #digitalzone
  50. 50. An advanced story with visits & backlinks Disallowed myurl.com/my-page 2768 Visits! 301 redirection myurl.com/redirect-page 301 redirection myurl.com/other-page No Indexed! No Backlinks! 1000s of Backlinks! @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  51. 51. A Simple Story about a URL . That’s me
  52. 52. A Simple Story about a few URL .. . . . . .
  53. 53. A Story about a few more URLs .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .
  54. 54. A Story about even more URLs .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .
  55. 55. …hidden among the others .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . .. . . .. . . . . . . . .. . . . . . .. . . . . .. . . .. . . .. . . . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . . .. . . . . .. . . .. . . .. . . . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . .. . . .. . . . . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . .. . . .. . . . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. .. . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . ... . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . .. . . . . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . ... . . . . .. . . . . .. . . . . . .. . . . . .. . . . .. . . . . . . . . .. . . . . .. . . .. . . . .. .. . . . .. . . . . .. . . . . .. . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .
  56. 56. 78473766246985183923814815742482105696053525424342762703234779816549659018650988449707938454508725013166558473232302902096811504404934076854307 77688870625168655782985431595150681477406513213395328771032817979822070623067880021557858582442427033738131665029996349133129288380874567410864 14034991347610789451621292215819421206230373165721570143065828756021435250420465590663912250173143392605384240521094509022523540697440824778264 42325264535976865048908831206200250295415225957848511039478407125212449608219704733094898718061393236285114781093048478937100163386870742309944 80382819814211559774426079984237004971410153332165398745553347463752768231651773514730654797031848298991140733872292366951092618400047899836044 53324991340941655648287702893018982019830434132160648353081081005848080709448555461376282508130708715287823006991789011101013014065967376808332 96973097311126033510919532364799966469108619264414565566429780915777075549446895450854999125034884132489494209990933896344115147084948730931376 42745334512172794166215863679650206883098883226412412533869285362147086071202117736490211788264888681452274934359153841488546263138401896337683 58538905027008087006841813556572033860966221049238231833733946565380386244124984109688966961648437100420381824077956889191290810166325049291081 35695965659485481190832742158152630735369805475905872022568836964293750528524111465179017254098375891444809395423322623381562374364064921804515 37611090180112564430616802952992301714416864531622716518682056952812530620637026909406083100490077402623699951457778075196454447227399298046478 37095500046983063332876102471983151557113494680741150705304549419006358172575934724486579166905034524117854803988871731643331832172215688107182 45423534289787269897692584333049692487978120133186645481716634035397650655531305055142287963190066195517600937052405614826457032618343115494036 44979918401507977556406613935843475679482599662222428774537045879460603574407941335915394815613528418189105789957261820512794038777966033755777 04655607957739037822676397174110366865834093778720767859576404947254163985066830821471838933850390655177324561337184584023980676711820580024727 86972907521894417652488622748959256739821148835038690304583144731721405546764947564898188409956097030768277988336831491610645233375082689663608 01478282219295476114638361981356777566591420546662090432041262893349177973606719017086614159819654371133395545627161949589824931165383477955452 78511627142351880721678464572361476663390962861071847525999474193850600711465076544469329807817220619380631187767945863516056151401186321613983 58383396729985267774251335092327970967184993329770237244313978111628580746451883790317371602234293618658518504247532513866537589207156379867202 39086772979960209708861948184281890071182723819202470880426982866185067084325025580477788198921379069735555849106745009300190834002649919903577 91769325024233390912498433688283756884535816171834060933902179741409280466261635754274996172731978825221837596709530618949259329507953138896244 72786356814088366055177878307043317299892732643083235819969046555286901425714428492632296658100838971525219414156350815328246795161595692081670 11293103975331302623420352543498604231833236255337260488073591957033032737060848219299552982268131634972997775653590402337186319344193126688466 66541453172142434828508980807887921061235992632892651036902166658963526289963465032742747060724259120518645721843238959291710203690370798848425 08041313605830741852392486897104194949423244475931177116494727605862918047498560295178911026354218664928746914791090454022683841437711316466551 21662329835914851717789536069184550924058466980065597086830187256141634826947112736078964945243398768914579306351485433434435725789132374523602 17337958300907091238202126481680666289211018573484312786272606967133344066848518380084707242221409750271173333084049965367203904578 Total potential number of URLs 7.95636796670821… x 103848 @DeepCrawl @Magdziarz #digitalzone@DeepCrawl @Magdziarz #digitalzone
  57. 57. Merging Excel spreadsheets @DeepCrawl @Magdziarz #digitalzone
  58. 58. Automation needed! @DeepCrawl @magdziarz
  59. 59. Scaling up 1000 URLs 100k+ URLs 1M+ URLs @DeepCrawl @magdziarz@DeepCrawl @Magdziarz #digitalzone
  60. 60. Your search universe! @DeepCrawl @magdziarz
  61. 61. The Ultimate Search Universe @DeepCrawl @magdziarz
  62. 62. Thank you Istanbul! @DeepCrawl @magdziarz
  63. 63. One more thing …
  64. 64. • Providing clarity with data • Educating the market • You at the forefront of this development Taking part in this journey:
  65. 65. Ilyas Teker DeepCrawl CAB Member

×