Next: What is Search Engine Optimization?
Previous: Introduction to Search Engine Optimization

What is a Search Engine?

From a user standpoint a search engine is a web page that allows keywords or phrases to be entered and returns results based on the relevance of the documents it holds in its index. Relevance is determined by a set of automated routines referred to as algorithms. Search engines index many billions of pages of internet content ranging from web pages, images, Adobe PDF and Microsoft Word documents through to music and video clips. It would be impossible for a search engine to explore all the data it indexes in real time. Instead it runs a number of automated programs called robots (also called bots, crawlers or spiders). These are programs that explore the content held in the search engine's index. Starting from the home page of a website and by following the links it finds, the robot can eventually explore all the linked pages in the site and even discover new websites through outbound links. This process is called spidering. Each document that the robot discovers is ranked based on the search engine's algorithms. This provides a concise picture of the content of the page and it is this information that is used when a search is performed.

The search engine builds up its index either by having webmasters submit their sites directly via free or paid submission pages or through discovering new sites during the spidering process described above. Obviously if a search engine doesn't know about a page it won't be found by its users. Yahoo! claim that they index around 80% of the Web although it is difficult to know how accurate this statement is. In order to keep the index fresh robots must periodically revisit a site. The amount of time between visits depends on the resources available to the search engine and to how up-to-date the search engine wants to keep a particular page. Some of these periodic visits, such as those prior to the GoogleDance, generate much excitement from search engine experts.

Examples of general purpose Internet search engines are Google, Yahoo! and MSN Search. Most search engines offer advanced search options that allow users to narrow down their search criteria, for example to find recently updated web pages or pages in a certain language. Some popular search engines use indexes from other engines, at the time of writing AOL search is powered by Google’s technology. There are also a number of specialty search engines, an example being Google News which indexes news stories from around the world. Google is currently the most popular internet search engine. Users perform over 200 million searches on Google per day and it is estimated that it handles between a half and three quarters of all searches.

What is a Search Engine?

See Also