A New Machine Learning Method for Chinese Overlapping Disambiguity--Conditional Random Fields

Conditional random fields (CRFs) are employed in this paper for resolving Chinese overlapping ambiguity in Chinese word segmentation. Instead of the traditional methods which treated the Chinese overlapping ambiguity as classification problem, the proposed approach regards this task as a sequence labeling problem. The best benefit of this method is that it can deal with overlapping ambiguous strings with any lengths no matter the ambiguous strings are pseudo ambiguity or true ambiguity. Several methods are tested on the same training and test corpora. The experimental results show that the CRF models achieve state-of-the-art performance. In comparison with the maximum entropy classifier and the traditional word bigram model, the accuracy has increased 3.98 % and 9.27 % respectively.