Next: Hilltop Algorithm
Previous: Google

Hidden Pages

Be careful about the almost divine ability of search engines to ferret out pages on your website. This is especially important if you are hosting the site on an inhouse server.

There are a number of mechanisms whereby search engines can find hidden pages. If someone looks at your hidden pages then surfs to another site the URL of the hidden page will be in the site's log file. If this is accessible to a search engine robot it will eventually follow this referrer link to your hidden pages.

Some toolbars and browsers like Opera communicate the addresses of pages that they visit to search engines. The engine may then dispatch a robot to index those pages.

The sort of information that search engines find can range from new versions of the website, to private intranets to security information on the host server such as website administration scripts and even the server's password files.

Hidden Pages

See Also