Overlap Among Major Web Search Engines

Our paper examined the overlap among results retrieved by three major Web search engines for a large set of more than 10,316 queries. Previous smaller studies have discussed the lack of overlap in results returned by Web search engines for the same queries. Our large-scale study measured the overlap of first page results (both non-sponsored and sponsored) across three major Web search engines - Google, Ask Jeeves and Yahoo - using a large number of randomly selected Dogpile.com queries submitted within a 24 hour - 36 hour period. We then compared the results to results retrieved for the same queries from the meta-search engine Dogpile.com. The percent of total results unique to only one of the three major Web search engines was 85%. 12% of total search results were found on two of the three Web search engines, and 3% of results were found across all three Web search engines. This small level of overlap reflects major differences in Web search engines retrieval and ranking results. Results point to the value of meta-search engines in Web retrieval to overcome the biases of individual search engines