Improving navigation page detection by using DOM-based block text identification

Internet changes very fast, it is necessary to classify the web pages for different usages. According to user purpose, web pages can be classified into navigation pages and content pages. To detect navigation pages is useful for web crawling, topical detection, etc. In this paper, we use DOM-Based block text identification method to improve navigation pages detection. Experimental results suggest that, compared to prior methods, our method is more effective.