Automated functional testing of search engine

This paper describes the construction of an automated test framework for search engine and reports our current practice in establishing a process for test automation. The paper presents the technical solutions that overcome the difficulties in search engine testing, which involve large volume of data, complicated ranking rules and randomization in ordering search results. One reason that many test automation efforts failed in industry is because automated scripts are not developed and structured for future maintenance and evolution. The test framework enables effective and efficient reuse and maintenance of test scripts. The paper also discusses issues related to the process of test automation. They are equally as important as overcoming technical barriers. All of these discussed above have been applied to the automated functional testing of a commercial search engine.