Hybrid Chinese Text Chunking

Author

Liao, Panpan ; Liu, Ying ; Chen, Lin

Author_Institution

Dept. of Chinese Language & Literature, Tsinghua Univ., Beijing

fYear

2006

fDate

16-18 Sept. 2006

Firstpage

561

Lastpage

566

Abstract

Text chunking is an effective method to decrease the difficulty of natural language parsing. In this paper, a statistical method based on hidden Markov model (HMM) is used for Chinese text chunking. Moreover, a transformation based error-driven learning approach is adopted to improve the performance. The definition of transformation rule templates is the key problem of this machine learning approach. All the templates are learned from the corpus automatically in this paper. The precision using HMM is 88.19% and the precision is 92.67% combining HMM and transformation based error-driven learning

Keywords

grammars; hidden Markov models; learning (artificial intelligence); natural language processing; text analysis; text editing; hidden Markov model; hybrid Chinese text chunking; machine learning; natural language parsing; statistical method; transformation based error-driven learning; transformation rule template; Asia; Data mining; Entropy; Hidden Markov models; Machine learning; Natural languages; Statistical analysis; Tagging; Text categorization; Text recognition;

fLanguage

English

Publisher

ieee

Conference_Titel

Information Reuse and Integration, 2006 IEEE International Conference on

Conference_Location

Waikoloa Village, HI

Print_ISBN

0-7803-9788-6

Type

conf

DOI

10.1109/IRI.2006.252475

Filename

4018552