Title :
An evaluating method of spider detection techniques by trap
Author :
Chunlong, Fan ; Zhouhua, Yu ; Lei, Xu
Author_Institution :
Dept. of Comput. Sci., Shenyang Inst. of Aeronaut. Eng., Shenyang, China
Abstract :
Spider is a program for obtaining internet resources. For monitoring spider visits to your website, Decision Tree, Bayesian Network and other Spider Detection Techniques (SDT) are proposed. At present, the evaluation of these detection techniques mainly relies on manual analysis of web log data to calculate the recall rate and precision rate. In order to avoid subjectivity caused by manual analysis, an Evaluation Method based on Trap detection technique of spider (EMT) is proposed in this paper which can evaluate the detecting capability of SDT. The traps layout information on the website and the process information of users accessing website resources are used to calculate relevant parameters, indicators and error range of EMT according to the binomial distribution theory. The Experiment results indicate that EMT and the artificial analysis method have consistent conclusion.
Keywords :
Bayes methods; Web sites; decision trees; online front-ends; search engines; Bayesian network; EMT; Internet resources; SDT; Spider detection technique; Web site; binomial distribution theory; decision tree; trap detection technique; Aerospace engineering; Bayesian methods; Computer science; Decision trees; Humans; Internet; Manuals; Robots; Search engines; Uniform resource locators; accuracy rate; evaluate; recall rate; spider detection; trap;
Conference_Titel :
Future Computer and Communication (ICFCC), 2010 2nd International Conference on
Conference_Location :
Wuhan
Print_ISBN :
978-1-4244-5821-9
DOI :
10.1109/ICFCC.2010.5497315
