Raising reliability of web search tool research through replication and chaos theory

Because the World Wide Web is a dynamic collection of information, the Web search tools (or “search engines”) that index the Web are dynamic. Traditional information retrieval evaluation techniques may not provide reliable results when applied to the Web search tools. This study is the result of ten replications of the classic 1996 Ding and Marchionini Web search tool research. It explores the effects that replication can have on transforming unreliable results from one iteration into replicable and therefore reliable results following multiple iterations.