A New Algorithm for Text Clustering Based on Projection Pursuit

Vector Space Model ( VSM ) is usually used to express text features in text mining with huge dimension, but it can not show the structure of the text set obviously and costs much in computing. A new pursuit projection based text clustering algorithm is proposed. With minimizing (or maximizing) a projecting index, Projection Pursuit searches for an optimal projection direction and projects text feature vectors from high-dimensional into low-dimensional ( 1 to 3 dimensions ) space. The linear and non-linear structures and features of the original high-dimensional data can be expressed by its projection weights in the optimal projection direction. The optimal projection direction is looked for by genetic algorithm, and the distribution of texts can be visualized. Pursuit projection based text clustering does not need to set cluster number previously like in k-means clustering, and opens out non-linear structure not like in latent semantics analysis only discovering linear structure. Experiments demonstrated that this algorithm is effective to cluster texts.