CELB: Content extraction based on line-block

In this paper, we propose a simple, fast and accurate content extraction method: CELB. Compared with traditional methods, this approach does not parse the DOM trees and uses only information from lines of original HTML documents. We propose a concept called line-block, to extract contents more effectively and a new feature distance-text number (DTN) for distinctions between contents and non-contents. First, we preprocess original HTML documents, and then combine lines into line-blocks. Next, we calculate values of content features for each line-block, and use thresholds to determine whether a lineblock is part of the main content or not. Experiments show satisfied results, especially for the running time.