Search Engine Design Based on Web Service and Lucene

Web service is selected as the key technology to design a search engine based on B/S architecture. Several spider clients are deployed on many computers to finish crawling Web pages in this B/S architecture after all Web pages have been analyzed and purified. And some useful Web pages are stored into database for index design. These spider clients are controlled by a spider server. Finally some indexes are created according to these Web pages by Lucene so that the search engine could offer search services for any Web user. Furthermore, the spider clients communicate with the server by Microsoft message queue and lots of important data-operated functions are designed as many Web methods in Web service. Final experiments show that distributed spider system of the search engine is more efficient than only one spider system.