A system for the identification of multi-page Web documents

When a World Wide Web (WWW) document spans multiple Web pages, it is often inconvenient to print or download the entire document using available tools. A two-phase iterative approach has been developed for the automated identification of pages residing within the same document boundary, given a starting uniform resource locator (URL) as input. This system was applied to a test suite of 98 Web documents, and the results were compared to a ground truth document boundary in each case. Using a set intersection metric, an overall success rate of 73 % was achieved. This is a significant improvement over existing tools, which are not fully automated, and can achieve a success rate of only 61 % with user assistance.