Frequent term based text document clustering: A new approach

Document clustering is used to organize the documents into groups. VSM (Vector Space Model) is a technique used to represent the document as a vector. Working with VSM to cluster the documents is easier. The main problem with text documents clustering is very high dimensionality of data. A term in the document represents a dimension. To reduce the dimensions of the document vector space, it is preprocessed. The main techniques involved are stemming and term filtering for dimensions reduction of document vectors. After dimensions reduction, term frequency vectors corresponding to each document are generated, where each cell in the term frequency vector represents frequencies of a term. Using proposed method in the paper, each pair of term frequency vectors are compared to find out the similarity value between every two corresponding documents. In this way, three similarity matrices minimum_match, maximum_match and average_match are generated which are further used in various clustering techniques to produce clusters. Clusters produced using proposed approach are compared with that of clusters produced based on cosine similarity in terms of F-measure. Higher values of F-measure for clusters produced using proposed method shows that proposed algorithm is better.