Comparing classification methods for link context based focused crawlers

Focused crawlers aim to fetch pages only related to a specific subject area from millions of web pages on the Internet. The essential task in a focused crawler is to predict whether a page is related to the target subject area or not without actually fetching the page content itself. Link context based focused crawlers focus on the surrounding text around each link to classify the page pointed by the URL. In this paper, we aim to compare three different classification methods (naïve bayes, decision tree, and support vector machines) for the task of link context based focused crawling.