Title :
An application-level network performance measure method for distributed information retrieval
Author :
Zhang, Weizhe ; Di, Wenchen ; Wei, Yifan
Author_Institution :
Sch. of Comput. Sci. & Technol., Harbin Inst. of Technol., Harbin, China
Abstract :
To balance the load for the crawlers in distributed information retrieval system, a new network performance measure method is put forward. Adopting the application-level interaction procedure and theory of the distributed crawling, pipeline number and page size are incorporated into the network performance metric. Experimental results prove that the new method has a high stability under the statistics, and can be a standard of the network distance to build the network coordinates.
Keywords :
Internet; computer network performance evaluation; information retrieval; paged storage; pipeline processing; search engines; application-level interaction; application-level network performance measure method; distributed crawling theory; distributed information retrieval system; network performance metric; page size; pipeline number; statistics; Crawlers; Data mining; IP networks; Linux; Pipeline processing; Support vector machines; distributed information retrieval; network distance; network performance measure;
Conference_Titel :
Computer Science and Service System (CSSS), 2011 International Conference on
Conference_Location :
Nanjing
Print_ISBN :
978-1-4244-9762-1
DOI :
10.1109/CSSS.2011.5974806
