Title :
Chinese sentence compression based on statistics probability and dependency analysis
Author :
Zhao, Qing ; Li, Lei
Author_Institution :
Intell. Sci. & Technol. Res. center, Beijing Univ. of Posts & Telecommun., Beijing, China
Abstract :
In this paper we describe a Chinese sentence compression tool, which makes use of various technologies. We first describe how to prepare our own Chinese training corpus and from what we learn the removal and non-removal probabilities. Then we bring in a knowledge base to preprocess the input sentences. Following that, how an input sentence gets analyzed is described, using Chinese word segmentation, a pos-tagging removal program and a shallow parser which outputs the collapsed dependencies. Next, we use the removal probabilities and compression rules to finish the task. Experimental results have shown that this method is feasible.
Keywords :
grammars; knowledge based systems; natural language processing; probability; statistics; Chinese sentence compression tool; Chinese training corpus; Chinese word segmentation; compression rules; dependency analysis; knowledge base; nonremoval probabilities; parser; pos-tagging removal program; statistics probability; IEL; Speech; Tagging; collapsed dependency; compression rule; knowledge base; removal probablity; word segmentation;
Conference_Titel :
Natural Language Processing andKnowledge Engineering (NLP-KE), 2011 7th International Conference on
Conference_Location :
Tokushima
Print_ISBN :
978-1-61284-729-0
DOI :
10.1109/NLPKE.2011.6138171
