Chinese sentence compression based on statistics probability and dependency analysis

Author

Zhao, Qing ; Li, Lei

Author_Institution

Intell. Sci. & Technol. Res. center, Beijing Univ. of Posts & Telecommun., Beijing, China

fYear

2011

fDate

27-29 Nov. 2011

Firstpage

68

Lastpage

72

Abstract

In this paper we describe a Chinese sentence compression tool, which makes use of various technologies. We first describe how to prepare our own Chinese training corpus and from what we learn the removal and non-removal probabilities. Then we bring in a knowledge base to preprocess the input sentences. Following that, how an input sentence gets analyzed is described, using Chinese word segmentation, a pos-tagging removal program and a shallow parser which outputs the collapsed dependencies. Next, we use the removal probabilities and compression rules to finish the task. Experimental results have shown that this method is feasible.

Keywords

grammars; knowledge based systems; natural language processing; probability; statistics; Chinese sentence compression tool; Chinese training corpus; Chinese word segmentation; compression rules; dependency analysis; knowledge base; nonremoval probabilities; parser; pos-tagging removal program; statistics probability; IEL; Speech; Tagging; collapsed dependency; compression rule; knowledge base; removal probablity; word segmentation;

fLanguage

English

Publisher

ieee

Conference_Titel

Natural Language Processing andKnowledge Engineering (NLP-KE), 2011 7th International Conference on

Conference_Location

Tokushima

Print_ISBN

978-1-61284-729-0

Type

conf

DOI

10.1109/NLPKE.2011.6138171

Filename

6138171