“All search engines are biased”

BANGALORE: Web search is a basic requirement for Internet navigation, yet the number of web search engines is decreasing. Targeting Google, Nutch, in an effort to implement an open-source web search engine says, "Today's oligopoly could soon be a monopoly, with a single company controlling nearly all web search for its commercial gain. That would not be good for users of the Internet."

Nutch claims that it provides a transparent alternative to commercial web search engines. They say that only open source search results can be fully trusted to be unbiased, or at least that the bias is public. Nutch claims that all existing, major search engines, have proprietary ranking formulas, and will not explain why a given page ranks as it does.

"Additionally, some search engines determine which sites to index based on payments, rather than on the merits of the sites themselves. Nutch, on the other hand, has nothing to hide and no motive to bias its results or its crawler in any way other than to try to give each user the best results possible," states the organization.

Although, none of these charges have been proved, Google’s critics have accused the world’s leading search engine of "bias" and of promoting advertising, discreetly. Google’s (patent pending) PageRank technology, relies on the ‘democratic’ nature of the web by using its vast link structure, as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google says that it looks at more than the sheer volume of votes, or links a page receives and also analyzes the page that casts the vote. Votes cast by pages that are themselves "important", weigh more heavily and help to make other pages "important," are Google’s claims.

Critics say that Google’s ranking system of the importance of pages is flawed. Avid blogger, Steven Johnson finds some holes or "blind spots" in Google. He points out that, while searching for some information on flowers with the keyword "flower", Google throws up mostly online florists.

Another charge is that bloggers have a stronghold, on Google search results. Try the name "steven" and the search results show Steven Johnson’s blog, as either the No.1 or No.2 result. He says, " So, Google is guessing on average that more people searching for Steven are looking for me, than for Spielberg, Seagal, Soderbergh, etc., which is most unlikely."

For all its "geeky" image, Google’s page rank technology is proprietary. That’s where the fault lies, claim organizations like Nutch. Open source supporters claim that, only an open source search engine offers complete transparency because anybody can examine the code and the technology that throws up the search results.

However, no open source engine has actually taken off, till date. It required a ‘Yahoo’ to nurture Google, from a students’ initiative to a world class search engine. Organizations such as Nutch are working on procuring funds (from Google competitors like Overture), to develop an open source search.