SiteRank-Based Crawling Ordering Strategy for Search Engines

Search engines are playing a more and more important role in discovering information nowadays. Due to limitations of time-consuming, network bandwidth and hardwares, we cannot obtain the whole information on the Web and have to download important information first. In this paper we propose a novel crawling ordering strategy which is based on SiteRank. Experimental results running on over 15 million pages indicate that it can work efficiently in discovering important pages under the PageRank evaluation of page quality. Furthermore, it exhibits the ability of anti-spamming.