Semi-hard c-means clustering with application to classifier design

Fuzzy c-means (FCM) clustering is the method for partitioning data into clusters by minimizing an objective function. Therefore, it is important to devise an objective function from which a simple clustering algorithm can be derived. An entropy term was introduced by S. Miyamoto in the FCM objective function. We proposed an objective function of the fuzzy counterpart of Gaussian mixture models (GMMs) clustering. The objective function is based on Kullback-Leibler divergence instead of the entropy. S. Miyamoto derived a hard clustering algorithm by linearizing the K-L divergence term of the objective function. In the hard c-means (HCM) clustering approach, covariance matrices are decision variables. For quick and stable convergence of FCM-like clustering, this paper proposes the semi-hard clustering approach by constraining the membership in an interval [ab]. The semi-hard clustering result is used for a classifier design. The membership function suggested by the generalized FCM and K-L based FCM is used for the classifier. The values of hyperparameters are searched by particle swarm optimization (PSO). In terms of classification performance on UCI benchmark data, the classifier is comparable to the support vector machine (SVM) and surpasses the k-nearest neighbor (k-NN) classifier. The computation time of FCM classifier for training a large scale data set is smaller than that of the SVM classifier with decomposition and working set selection algorithms.