White paper on how to manage transitions for URLs through a website redesign, in order to retain built-up SEO values. Includes information of the importance of maintaining URL structures, URL rewriting, redirection best practices, and importance considerations in the redesign process, so you don't mess up your hard earned search engine equity.
2. 1
Site Redesign: The Importance of URL
Strategy
Marketing Sherpa reported in January 2006 that site redesign and updating has
become one of the highest priorities for online marketers, and is now more
popular than any other interactive marketing initiative. Considering that any
number of factors in a redesign can have a positive or negative impact on a
brand’s search visibility, knowledge of best practices in search-friendly web
design has become imperative. One major element of redesign that is the
treatment of URL equity, which is the sum of several important values tied into
URL structure. Changing domains or renaming existing deep URL structures can
wipe out years of gains in links, traffic, sales and conversions, and prior natural
search investments.
Lack of strategy can have serious consequences
for search visibility
One of the greatest threats to a search engine presence is a lack of URL strategy
in a site redesign. Being indifferent about URLs means that a single variable
implemented in the development or planning process has the potential to erase
years of positive gains. The impact of ignoring URL structures is felt in years of
lost links, lost search engine traffic, dramatic loss of indexed pages, lost sales
and conversions, and lost search investments.
This is not to suggest that sites should never be redesigned, or that URLs should
not be re-named. There are many legitimate reasons to alter URLs, including re-
vamping of the user path and experience, changing technical platforms, or
removing content. A redesign should be treated as an opportunity to bridge
current equity and to maintain a sustainable URL structure that will position a site
for a long-term positive natural search presence. Understanding the inherent
value of URLs will not only help bridge accrued natural search gains, but also
bridge a site’s search engine reputation for years to come.
The elements of URL equity
Uniform Resource Locators (also known as Uniform Resource Identifiers or
URIs) are the axis point for all traffic that runs throughout a site. A site’s URL
equity is composed of the following elements:
Link equity: This is the positive buildup of links over time, both to the homepage
and the internal URL structure of a site. Links may come from many places such
as blogs, newspaper sites, authority sites, directories or internal site pages. It is
not uncommon for many major brands to have hundreds of thousands, or even
millions of quality links pointing to a domain and deep internal site structures.
Links and other “off-the-page” factors are now the most crucial element in how
search engines retrieve their results. Preservation of existing links alone is a
strong enough reason to maintain URL equity and natural search engine
presence.
3. Where there is link equity, there can also be link liability. If a domain or URL has
been banned in an engine due to bad linking practices, then another URL should
2 be considered.
Positive search engine equity: Positive search engine equity is gained from
the history of search engine crawling, indexing and retrieving over time. URLs
are like a serial number for a document, and search engines use domains and
internal URLs as the reference point for these processes. For sites that do not
spam and have a positive history with the search engines, URLs become more
trusted over time.
Using new URLs or domains in a redesign means that engines must start the
crawling, indexing, and retrieval processes over again. Search optimizers share
the common knowledge that legacy domains and trusted URLs are like gold; if a
domain or trusted URL structure is several years old or older, there is a distinct
advantage over sites built on newer domains and structures. As mentioned
above, a current or previous ban in the engines indicates a liability. In that case,
renaming may still be a positive step for a redesign.
Bookmark equity: One additional (and often overlooked) factor in assessing
URL equity is bookmark traffic. A bookmark represents a site conversion for
visitors who have found useful content and will return later. Failure to bridge the
gap for converted visitors is a poor usability experience because these visitors
are left hanging with their intentions unfulfilled.
Search investment: Costs are incurred whether natural search is performed in-
house, by an agency or by a consultant. When URLs change without a plan,
there is a risk of completely wiping out search investments. This could include
previous link development campaigns, general site promotion designed to drive
traffic to deep site URLs, or implementation of general search engine-based Web
design practices.
The consequences of redesigning without a URL
strategy
Redesigning a site without paying attention to these values can have a
tremendous negative impact on search visibility. This negative impact may create
the following issues for search engine visibility:
● Indexed pages drop out of the search engines
● New site pages can not be found by crawlers
● Backlink history is lost
● Navigation becomes difficult, and visitors can not find what they are
looking for
● Bookmarks are rendered useless, leading to “not found” error pages
● Server bandwidth is wasted
● Valuable site traffic is lost
● Lost conversions and sales
4. 3 With the value of URL structure established, here are three key questions
every marketer or web redesigner should ask before planning and
redesigning a web site:
1. How is URL architecture included into business and technical
requirements?
Simply put, getting URL structure and SEO established in the business
and technical requirement prior to redesign goes 90% of the way to
maintaining positive search equity. Design and development teams are
experts at solving problems, and knowing the pitfalls will help them solve
these problems. Trying to tackle complex search issues in the middle or
end of redesign is a losing battle that nobody wins.
2. How much URL equity is established in the current site
structure?
URL equity should be assessed based on a number of factors, including
number of backlinks, quality of backlinks, prior investment in search, the
age of the domain and URL structure, and positive search engine equity.
Conversely, URL liabilities may be a cause to change domains and URL
structures entirely.
3. How can the existing URL structure be preserved?
It is a best practice for search to maintain a sustainable structure from
redesign to redesign in order to preserve positive equity. While changes
in URL structure are not always avoidable due to user experience
changes and technical issues, making every attempt to maintain
consistent structure will provide greater long term benefits for search.
Sir Tim Berners-Lee is a strong proponent of sustainable URL structures.
Visit this link to read his arguments for maintaining URLs in a redesign:
http://www.w3.org/Provider/Style/URI.html
Assessing URL Equity
Once website stakeholders understand the value in preserving URL structures in
a redesign, there are many considerations for assessing the quality of existing
link structures. To assess the value of URL’s prior to redesign, consider the
following:
Quality of inbound home page and deep site links – Does the site contain
links from authority sites? Are there any .gov or .edu links (engines give
higher weight to these links)? Are there any Yahoo! Or DMOZ directory links
pointing into the domain or internal site pages? Time should be spent
reviewing hundreds, or even thousands of links via manual backlink check in
a major search engine. Identify and prioritize these links.
Age and history of domain and URLs – The age of a domain and internal
URL structure can also have a positive impact on a site’s search visibility.
Asking the following questions will help assess this value:
• When was the domain registered?
• How long has it been online?
• Has the web host always been reliable?
5. • How long has the existing URL structure been in place, and does
the current structure perform well in the engines?
4
URL liabilities – URL’s can also have a negative history that could reduce
the search performance of a website. If a site has ever engaged in tactics
looked down upon by the search engines, or has been banned at any time, a
change of domain may be considered.
Log file history – Reading log files can also provide a good indicator of
which internal site pages are performing well, back on incoming search
engine referrals and link traffic. If any pages stand above the rest, take extra
care with the transition of these pages and URL in a site redesign, and ask
the following questions:
• Are internal pages pulling high volume search traffic?
• Why are these pages performing well? (link traffic, search engine
traffic, etc)
• Does the sum of all non-homepage pages refer substantial
traffic?
Deep Link Ratio – Calculating the Deep Link Ratio to determine the
aggregate percentage of internal site links compared against the
homepage. This number is a great indicator into how search engines
value internal site content. This ratio can be calculated by dividing the
total number of internal links (minus the homepage backlinks), divided by
the total number of inbound site links.
Transitioning URL equity
No matter how much preparation is made for a smooth transition, inevitably some
URL’s will have to change, and some documents will be removed. In this event,
proper redirection techniques are essential in preserving positive search engine
visibility. In most cases, the 301 permanent redirect is the best solution for using
multiple domains, and for pages that have moved to a new location.
301 redirects vs. 302’s: One common mistake made by server
administrators is the use of 302 status redirects for moved pages. When
a page is permanently moved to a new location, a 301 status will tell the
search engine to remove the previous page from the index, and start
crawling the new location from that point forward. Additional POV’s are
available for proper redirection processes for different web servers.
Pointing multiple domains: Pointing multiple domains is okay with the
search engines as long as 301 redirects are used. When 302 or 200
status redirects are utilized, duplicate content and issues will arise.
Duplicate content issues can result in the total ban of a site or pages,
and will often decrease the overall search engine performance of a
website (though most duplicate content is removed without any site
penalty).
URL rewriting: If URLs must change, one solution is to rewrite the new
URLs to the format to the old structure, or create a new search-friendly
structure entirely. URL rewriting allows more freedom to change
platforms and file names, while maintaining a consistent naming
convention. A separate POV on URL rewriting is available, and
addresses rewriting on different server platforms.
6. 5
Redirection planning: Once valuable URLs are identified, a plan for
mapping old URLs to new URLs should be created.
Pages that are removed entirely: These URLs should be 301
redirected to either the home page or the site map.
Pages that move to a new location: The old page URLs should
be mapped via 301 redirect to the new location where the new
file is found (old “Contact Us” page should be 301’ed to the new
“Contact Us” URL location.