Next: Geographic Information from Log Files
Previous: Traffic Analysis and Requests

Error Pages

If you've ever restructured a site, something to be avoided but that can happen through poor initial design choices, then you will see a lot of error pages. These are entries in the log file where the server returns a 404 code to the user. They can be down to inbound-links pointed to non-existent files or to search engines that have not yet updated their indexes. In the latter case this can take several months as search engines do not automatically drop a page they can't find. The first thing to do is to redirect errors to your site map so that users can attempt to find the missing resource. Depending on the web server there are different ways of doing this. On Apache this is done through the .htaccess file:

ErrorDocument 404 /sitemap.htm

If the file has moved you may want to redirect requests to the new resource. If there are errors on script pages (in particular: HTTP 206 - partial content delivered) it may be that they have bugs or problems accessing resources such as databases and are timing out. You should periodically check the internal link structure and the outbound-links. A tool such as the freeware Xenu Link Sleuth <http://home.snafu.de/tilman/xenulink.html> will automate this process. Be aware that if you pay for your website's bandwidth this can be quite expensive on resources. It is a good idea to set up a staging, mirror site to to perform link and other tests before migrating the changes to the live site.

Watch out for any search engine robots that get strange error results. Your site will not get indexed if they can't see your content. Some webmasters have reported Googlebot seeing 406 - no acceptable content errors. The Googlebot will only accept HTML and text content types but the server wants to return a different type of content.

Error Pages

See Also