Feature Selection algorithms based on HTML tags importance

Traditionally in Web crawling, the required features are extracted from the whole contents of HTML pages. However, the position which a word is located inside the HTML tags indicates its importance in the web page. This research proposes two ideas concerning the Feature Selection stage in HTML web pages. The first idea reduces the features by simply extracting them from the important tags in an HTML page in order to achieve faster classification. The second idea gives weights for each of the important tags. Two algorithms are presented in this paper based on these ideas: i) Important HTML tags only algorithm, ii) Weighted Important HTML tags only algorithm. The selected features are classified using two famous classifiers in the literature: Support Vector Machine (SVM) and Naïve Bayes classifier (NBC). The accuracy of each algorithm is computed. Comparison between the accuracies of traditional feature selection method, which uses the whole contents of HTML page, and the proposed algorithms is performed. Complete evaluation is performed which indicates the effectiveness of using our technique. The experimental results show improved precision and recall with the proposed algorithms with respect to keyword-based search. The algorithms are implemented in JAVA and its extended packages.