Web Object Mining Based on Entropy Pruning

Currently, a large number of Web information on the Internet is presented in structured objects. Mining object information from Web is of great importance for Web data management. MDR algorithm is a fully automated data records mining method. The algorithm can successfully detect data records in pages, but the effect is not very ideal because the noise existing in the page can not be eliminated. The paper introduced information entropy theory based on MDR algorithm and presents a new Web object mining method. Firstly, a Web page is divided to blocks and we construct a semantic DOM tree for each block. Then we compute the entropy value of the Web page and find topic regions using entropy pruning algorithm. Lastly, data records are mined from topic regions. Experiment proved the effectiveness and practicality of the method.