A New Validity Index for Fuzzy Clustering

A new validity index is proposed to determine the optimal number of clusters for fuzzy clustering. In a good partition, the similarity between data points within a cluster should be maximized and the clusters should be separated. Intra-cluster variation is defined to measure the similarity within a cluster; it should be minimized for a good cluster. Inter-cluster overlap is defined to measure the separation between clusters; a larger value indicates better separation between clusters. Based on the two measurements, the validity index is proposed. Experimental results on four artificial datasets and two real datasets show the effectiveness and robustness of the proposed validity index.