Chinese text feature extraction method based on bigram

One of the most important issues in Chinese text categorization is Chinese Word Segmentation. Conventional Chinese Word Segmentation modules usually consume too much time. In this paper, a novel method is proposed where bigram is applied to quantify the text and improved information gain algorithms are used to collect appropriate feature during text categorization. Experiments show that the bigram feature collected characterise the categorization fairly well. Compared to using word feature, the method proposed consumes much less time while maintaining a certain required categorization accuracy. Hence, this method can be applied to time-sensitive systems.