Integrating N-gram model information for Chinese word segmentation based on conditional random fields

This paper presents a Chinese word segmentation system based on conditional random fields, which integrates the result information of N-gram model as features of conditional random fields. Since dictionary-based N-gram model can deal with in-vocabulary words very well, while conditional random fields have the advantage of recognizing out-of-vocabulary words. This approach is evaluated using the PKU data from Sighan Bakeoff 2005. The experimental results have proven that this method achieved an F-measure of 95.0% and higher R_oov (85.2%) and R_iv (97.9%).