A Generalized Hidden Markov Model Approach for Web Information Extraction

A generalized hidden Markov model (GHMM) which extends traditional HMMs by making use of Web-specific information for Web information extraction is presented in this paper. Web content blocks are used instead of content terms as basic extraction unit in our approach. Besides, instead of using the traditional sequential state transition order, the state transition orders of GHMMs are detected based on layout structures of the corresponding Web pages. Furthermore, multiple emission features are applied instead of single emission feature. In this way GHMMs can better accommodate Web information extraction. Experiments show promising results of GHMMs