Title :
Efficient web crawling with proposed URL ordering
Author :
Sandhya ; Rafiq, M.Q. ; Farooq, Omar
Author_Institution :
Aligarh Muslim Univ., Aligarh, India
Abstract :
In this paper an efficient and modified ordering algorithm is proposed. Which shows the better utilization of storage space. This paper shows the results of the modified proposed URL (Uniform Resource Locator) ordering algorithm. Experimental result shows that it can work efficiently in finding the important pages comparatively to the traditional PageRank. We have performed experiment on data set using web logs from university and we achieve statistical significant improvement in ordering by getting two more categories on an average also more URLs in the higher order rank. And it shows 6.95% time saving in downloading and 46.6% better in Quality than traditional PageRank.
Keywords :
Internet; URL ordering algorithm; Web crawling; Web logs; higher order rank; storage space; uniform resource locator ordering algorithm; Algorithm design and analysis; Crawlers; Servers; Signal processing; Signal processing algorithms; Time frequency analysis; Web pages; Downloads Quality; URL ordering; Web Pages; Web crawler;
Conference_Titel :
Multimedia, Signal Processing and Communication Technologies (IMPACT), 2011 International Conference on
Conference_Location :
Aligarh
Print_ISBN :
978-1-4577-1105-3
DOI :
10.1109/MSPCT.2011.6150516