Next: Spam
Previous: Site Maps

Site Rank

It would be nice to know how our website ranks in a search engine for a particular search term. It is quite feasible to do this by typing the keywords into a search engine and trawling through the results. However this is longwinded especially if you have a number of keywords to check. You could always write a program that accessed the search engine and sifted through the results for you. However parsing web pages is not very convenient and your program will use search engine resources that would be better used answering genuine queries. Google and other search engines specifically discourage this kind of query.

There is an alternative. Google has published a Web Service API <http://www.google.com/apis/>. This enables people to programmatically access the Google search engine. The results are delivered in a format that is easy to process. The good news is that Google allows you to use this API to create tools to check site rank etc. To use the web service you have to download the developer’s kit, create a Google API account and write programs using your Google License Key. A single key enables you to make 1000 queries per day.

This still requires you to install a development environment on your computer and know a bit about programming (more later). If this is not your cup of tea there are a number of tools on the Internet that use the Google API. You'll need your own Google API key to use some of them so applying for an account is a good idea anyway.

Site Rank Tool

The following piece of code is written in the Perl computer language. Perl is a popular interpreted scripting language oriented towards the processing of text. The program accesses the Google API through the Simple Object Access Protocol (SOAP) Lite module that is part of the Perl language. If you are using Microsoft Windows you will need to download ActiveState's <http://www.activestate.com/> version of the Perl language called ActivePerl. You will also need the Google developer’s kit and a Google API license key. Install ActivePerl and check that it works by opening a command line interpreter (DOS Window) and typing Perl. Save the program below to a file called: rankme.pl. You will need to add your license key to the file and copy another file called GoogleSearch.wsdl to the same directory as rankme.pl. This last file tells the SOAP interface how to access the Google Web Service API.

#!c:/usr/perl/bin/perl
# Perl client to check google site rank
#
# (c) Copyright 18 August 2004 David George
#
use SOAP::Lite;
use strict;

# uncomment this line if you access the Internet via a proxy server
#$ENV{HTTP_proxy} = "http://localhost:80/";;

# add your own google API key here
my $key="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";

my $query = $ARGV[0] || "-h"; # either type on the command line or it defaults to help
my $domain ="http://$ARGV[1]/";
my $max=$ARGV[2] || 100; # defaults to 100 search results

if ($query eq "-h") {
    print qq|usage:
    keyword(s) domain [max]\n
    keyword(s) - keyword to search, surround multiple keywords with quotes
    domain - The domain to be ranked, e.g. www.google.com
    max entries - max number of results to check, higher values take longer, defaults     to 100
    \n
    examples:
    widgets www.mywidget.com
    "widget wranglers" www.mywidget.com/wranglers 1000
    \n|;
    exit;
}

# this file must be in the same directory
my $service = SOAP::Lite
    ->service('file:GoogleSearch.wsdl');

print "rank domain $domain for keywords $query, checking $max results\n";

my $found=0;
my $loop=0;
my $time=0;
my $total;
while ($loop < $max) {
    # $result is hash of the return structure. Each result is an element in
    # the array keyed by 'resultElements'. See GoogleSearch.WSDL for details.

    # results are delivered in blocks of ten
    my $result = $service->doGoogleSearch($key, xml_clean($query), $loop, 10, "false", "", "false", "", "latin1", "latin1");
    if ($loop==0) {
        $total=$result->{estimatedTotalResultsCount};
    }

    if(defined($result->{resultElements})) {
        my $start = 0;
        $time+=$result->{searchTime};

        while ($start < 10) {
            my $url=$result->{resultElements}->[$start]->{URL};
            $start++;
            if ($url =~ /$domain/) { # match start of url
                $found++;
                print "rank: $start $url\n";
            }
        }# while
    }
    $loop+=10;
}# while

if ($found == 0) {
    print "No entry for $domain in first $max found\n";
}
    print "Search Took: $time seconds, found: $found in a total number of responses: $total\n";

sub xml_clean {
    my $text = shift;

    $text =~ s/\&/\&/g;
    $text =~ s/</\</g;
    $text =~ s/>/\>/g;
    $text =~ s/\"/\"/g;
    $text =~ s/\'/\'/g;

    return $text;
}

Results

The following examples are for demonstration purposes only and don't represent real searches. They show how the rankme tool can be used to evaluate how well a page is optimized for a range of criteria.

The keywords: search engine optimization are used for all the examples. Google will find pages which contain these keywords in any order, although it will have a preference for the order given. To search for the exact phrase enclose it in single quotes.

Keyword Search

Check first 35 results for keyword phrase "search engine optimisation":

rankme.pl "search engine optimization" www.abcseo.com 35

rank domain http://www.abcseo.com/ for keywords search engine optimization, checking 35 results
rank: 29 http://www.abcseo.com/seo-book/
rank: 30 http://www.abcseo.com/comments/A133_0_1_0_C/
Search Took: 0.651425 seconds, found: 2 in a total number of responses: 2970000

The results tell us that the there are 2,970,000 results for the keywords. A competitive phase. Our site lies in 29th and 30th place.

Anchor Text Search

The same search is repeated but looking for "search engine optimization" inthe anchor text only. Google ranks all the sites that have at least one in-bound link containing this anchor text:

rankme.pl "allinanchor:search engine optimization" www.abcseo.com 35

rank domain http://www.abcseo.com/ for keywords allinanchor:search engine optimization, checking 35 results
rank: 21 http://www.abcseo.com/articles/seo/
Search Took: 0.962904 seconds, found: 1 in a total number of responses: 90,100

There are 90,100 sites with this in-bound link, abcseo.com site is number 21.

Body Text Search

The search can be restricted to just the page contents using the "intext" operator. This search ignores URLs, anchor text and document title:

rankme.pl "allintext:search engine optimizations" www.abcseo.co.uk 35
rank domain http://www.abcseo.co.uk/ for keywords allintext:search engine optimizations, checking 35 results
rank: 30 http://www.abcseo.co.uk/seo/
Search Took: 1.365586 seconds, found: 1 in a total number of responses: 3380000

Title Element Search

The "title" operator restricts the search to documents which contain the query occurs in the title element:

rankme.pl "allintitle:search engine optimization" www.abcseo.com 35
rank domain http://www.abcseo.com/ for keywords allintitle:search engine optimization, checking 35 results
rank: 21 http://www.abcseo.com/seo-book/
Search Took: 0.966249 seconds, found: 1 in a total number of responses: 276,000

URL Search

The "inurl" operator restricts the search to the documents where the query occurs in the Uniform Resource Locator (URL)

rankme.pl "allinurl:search engine optimization" www.abcseo.co.uk 35
rank domain http://www.abcseo.co.uk/ for keywords allinurl:search engine optimization, checking 35 results
No entry for http://www.abcseo.co.uk/ in first 35 found
Search Took: 0.828275 seconds, found: 0 in a total number of responses: 16800

In this case no results were found, which is what we would expect as the term does not occur in any of the domain's web addresses.

Some Useful Google Query Operators

Google supports the following query operators

intitle: Putting "intitle:<query>" in front of a query term will restrict the results to documents containing that word in the title. Put the query in quotes to search for the exact query in the title.

Note: Putting "intitle:" in front of every word in your query is equivalent to putting "allintitle:" at the front of your query.

allintitle: Starting a query with the term "allintitle:" restricts the results to those documents with all of the query words in the title.

inurl: Putting "inurl:" in front of the query term restricts the results to those documents containing that word in the result URL.

Note: "inurl:" only works on words and ignores punctuation, so inurl:apple.co.uk, will search for urls containing the terms "apple" "co" "uk" in any order

allinurl: Starting a query with the term "allinurl:" restricts the results to those documents with all of the query words in the result URL.

allintext: Starting a query with the term "allintext:" restricts the results to those documents with all of the query words in the body text and ignores link, url and title matches.

allinlinks: This operator does no longer appears to work, see allinanchor.

allinanchor: Starting a query with the term "allinanchor:" restricts the results to those documents that have all of the query words in their inbound-links.