Next: Validation
Previous: Traffic Analysis

Uniform Resource Locators (URLs)

If you were around during the Internet gold rush or dot.com boom or whatever they called it you will remember all those IPO reports that talked about brand creation. Apparently the suits were going to corner the Internet by squatting on great, catchy domain names and then pour bizillions of venture capital and IPO dollars into building those brands. Names like Boo.com, Kozmo.com and Webvan.com. Those environmentally friendly management consultants were going to recycle the same ideas they'd applied to bricks-and-mortar. Having blown huge amounts of other people's money, the three examples cited are now little more than dot.compost. Sent to the great domain name registry in the sky. It appears that the suits just didn't get the Web.

Unless you are already a well known brand, building awareness of your name takes a large marketing budget. Most websites are like a shop in main-street. You need some publicity otherwise you will get little more than passing trade from link partners. That publicity is supplied by search engines. Two components that improve rankings in many search engines are relevant anchor text and inbound-links.

Domains

When people create inbound-links to your site they often use just the domain name. For example:

asite.com has some interesting information on maine coons

or even worse

for information on Maine Coons click here

That's great with search engines that treat inbound-links as a vote for the site. Google and also Yahoo! and the new MSN Search put some store on inbound-links but we are missing out on good anchor text. We can go around asking web masters to use specific anchor text. Some will do this but many are too busy to go back over existing links.

Now let's assume our site is all about Maine Coon cats, in fact we are a Maine Coon cat breeder. A better choice of domain name might be:

mainecoonbreader.com

maine-coon-breeder.com

Whether to use hyphens or not is a trade off. Websites will often insert the spaces when writing your domain name, the advantage with hyphens is that all the major search engines already treat these as spaces. The disadvantage with hyphens is that it makes it harder to give the domain name over the telephone or radio and it is more difficult for people to remember the exact breakdown of the words if they see the domain in print or street advertising. One member of the forum alt.internet.search-engines has coined the term 'oral sharing' to describe this form of transmission. If branding is important then use shorter domains. Excessive hyphening, for example

bargain-international-phone-calls.com

may even get penalized by search engines.

So now we automatically get our main target keywords mentioned in every reference to our site. We can still trade under a different name and register that domain with a redirect to our main site just in case. This also helps us use our main keywords in our page content. Some examples:-

welcome to Maine-Coon-Breeder.com...
Maine-Coon-Breeder.com supplies pedigree Maine Coon kittens...
Contact Maine-Coon-Breeder.com...

Okay, you get the idea. Don't overdo the number of references in a page or it or it will look like the worst kind of spam email. The only problem with this idea is if your target keywords have already been registered. There are plenty of alternative top level domains (TLDs) such as .org, .net, .info or even .co.uk, .fr for a geographically based site. Search engines don't really care whether you are a dot.com or not. Most domain registrars automatically list alternatives in response to a search.

The majority of sites are not based around just a couple of keywords. The use of keywords in the domain name and URL helps with search engine optimization but being an on-page factor is not that significant. However the knowledge does give us some clues about how to structure our website. It is good practise to group pages belonging to the same theme in a subdirectory and to use keywords in the resource name. An example would be a website covering sport teams. This may, amongst other sports, cover baseball:

teams/baseball/new-york-yankees.html
teams/baseball/boston-red-sox.html
teams/baseball/atlanta-braves.html

Plan the site structure with a view to future expansion and make sure spelling is correct. Fixing problems down the line can be extremely disruptive as the ranking of a page is associated with a particular URL and it may prove impossible to update all the inbound-links. We have used hyphens in the above examples. For domain names the only legal characters are 'a' to 'z', numbers and hyphen. Both Yahoo! and Microsoft's new algorithmic search engine treat hyphens and underscores as spaces however Google only treats hyphens as spaces and for this reason underscores should be avoided.

Multiple Domain Names

Domain names are pretty cheap these days. Some SEOers register multiple domain names and there are some good reasons to do this.

to cover different spellings or misspelt words
to stop competitors registering the domain
to protect various copyrights and trademarks from cyber-squatting
to secure the domain in various geographical locations
to track traffic from specific SEO efforts... (shadow domains).
to track advertising campaigns
to target different keywords

Having a site accessible through multiple domain names can also dilute SEO efforts as each domain will be treated differently by search engines. They may even trigger excessive cross-linking or duplicate page filters. Some black-hatters treat domain names as a disposable resource to be used during an intensive marketing campaign and then abandoned should the search engines penalize their methods. For businesses that have invested heavily in their domain and branding this is not an option.

Shadow Domains

Some SEO companies use shadow domains to market your website. This is an alternative domain name frequently under the direct control of the SEO outfit. They may explain that this is a good idea to protect your main domain from a possible site ban because they are using dubious or aggressive tactics. However when the relationship ends they may point their shadow domain at another site, stealing any transferred benefits from PageRank and anchor text. Worse, they could sell it to one of your competitors.

Domain Redirects

If you do have to make changes you can redirect the search engine to the new pages. If you are using the Apache web server and have full access to your site you can do this with the '.htaccess' file (note the dot in front of the file name, this is a hidden file on Unix although Apache works well on Windows as well). The htaccess file is found at the root of the HTML directory and is a text file. Be careful editing this file as small changes can have a big effect. You should always make a backup before making changes.

Redirects can be added at the end of the file. The format is the location of the redirect file relative to the htaccess file (html docroot) followed by the new, full URL.

redirect 301 /index.htm http://www.mydomain.com/

Note the figure '301', this is an HTTP (HyperText Transfer Protocol) error code. It tells search engines and web browsers that something is wrong with the requested resource. In this case 301 corresponds to 'moved permanently'. Error code 302 'not found' should not be used as this implies a temporary redirect and the search engine will continue to index the old page. Search engines should adjust their index to use this new URL.

It is also possible to redirect a search engine by using the Meta tag redirect in the old document:

This meta tag should be placed in the Head part of the HTML file and any old content removed. In this case the search engine and any users visiting the site will first download the old file. They will then immediately load the resource specified in the URL tag. PageRank will not transferred to the new page so this technique is not recommended.

There are some fairly heinous examples of the meta tag out on the World Wide Web, at least in terms of search engine optimisation. Here is one I found. The international ski resort of La Plagne in the French Alps has a website at: www.laplagne.com. Most people who link to the website use this URL and consequently they have a reasonable PR5. However, at the time of writing, if you go to the site you are immediately redirected to another page: http://www.la-plagne.com/index_hiver.html. What they want to do is send people to different parts of the site depending on which season it is. Their webmaster uses a Meta Refresh tag located in the Body element of the home page. The consequence is that the page with all keywords and links to the rest of the site has a miserable PR2. Time to shoot the webmaster, I think.

Domain Rewriting

The htaccess solution shows its limits when we have a large number of URLs to change. If these are made to a pattern we can use a URL rewriting module. We already discussed the case of database driven content where we want to disguise dynamic URLs as search engine friendly static URLs. Under Apache the module is called mod_rewrite and on Internet Information Server, ISAPI Rewrite. .Net programmers can also use the HttpModule package to create a bespoke rewrite engine.

A typical use of rewriting is to combine page rank. Most websites have an address of the form:

http://www.mydomain.com

In terms of the Internet name system (Domain Name Service) their is nothing significant in the prefix 'www', it is simply a computer somewhere within the 'mydomain.com' domain. DNS simply translates it to an Internet (IP) Address which is how data is really moved about the Internet. Your web server could just as easily be called 'snafu'. A common feature of commercial web-space is that the URL: http://mydomain.com and http://www.mydomain.com map to the same Internet Address and so to the same website. However search engines think in terms of URLs so these are actually two different sites as far as they are concerned. Where inbound-links target the different domains we end up by splitting the value of the links. The following lines in the htaccess file will actually create a moved '301' redirect for all pages addressed as: http://mydomain.com:

RewriteEngine On
RewriteCond %{HTTP_HOST} ^mydomain.com
RewriteRule (.*) http://www.mydomain.com/$1 [R=301,L]

Pretty cool huh? You can even create separate .htaccess files in subdirectories if you wish to modify a branch of the directory tree.

Sometimes rewrite rules can cause problems where there are other redirect rules in the htaccess file or you may be in an environment that doesn't support rewrite rules. In this case It is possible to implement redirects programmatically in Perl, PHP or even ASP files. The following example is for PHP, an extremely popular programming language for creating dynamic Web content.

<?
    $request = $_SERVER['REQUEST_URI'];
    $server = $_SERVER['HTTP_HOST'];

    $prefix = strtolower(substr($server,0,4));
    if ($prefix != "www.") {
        header("HTTP/1.1 301 Moved Permanently");
        header("Location: http://www.$server$request");
        exit;
    }
?>
<HTML><HEAD><TITLE>redirector</TITLE></HEAD>
<BODY>
You are at
<?
print("<h1>$server$request;</h1>");
?>
</BODY></HTML>

The PHP code is between the special <? and ?> tags. It gets the server name and request. It then checks the first four letters of the server name having first converted them to lowercase. Remember that Internet names are case insensitive. If the name doesn't begin with www. it redirects the search engine to the correct name.

Domain Registering

Make sure you secure your domain name and variants before making plans about your site public. In 1995 Digital Equipment launched the AltaVista search engine without first checking that the domain was available. In truth the benefit of a unique domain for their project probably didn't occur to them at the time. It took five years, a court case and $3.3 million before they finally secured AltaVista.com.

Even more flagrant was British Telecom's project to re-brand its prestigious UK research labs at Martlesham Heath. The site was to be renamed Adastral Park, an allusion to the fact the labs were built on a former Royal Air Force base and that the 4000 geeks housed there should themselves be reaching for the stars (the RAF's motto). Unfortunately the high powered management team had not registered the domain. A quick thinking BT contractor called Ric Hayman registered the domain www.adastralpark.com, he later offered it back to BT for £2 million.