An hybrid validity index for dynamic cut-off in hierarchical agglomerative clustering

Clustering aims at partitioning unlabelled data samples into clusters so that the samples within a cluster are close to each other. One of the most challenging issues in cluster analysis is the determination of true clusters. This challenge becomes more intriguing in case of hierarchical clustering techniques that requires dynamic termination of the clustering process at true conception of clusters. In this paper, a dynamic cut-off for hierarchical agglomerative clustering has been proposed which automatically terminates the clustering process once optimal number of clusters are obtained. The cut-off is based on the computation of validity index to assess the global proximity and local proximity of the clusters at every level of the hierarchy. The decision of optimality is rendered during the construction of the hierarchical tree. This eliminates the need for the computation of the complete tree. Experimental analyses on real datasets from UCI machine learning repository demonstrate the efficiency of the proposed method in detecting true clusters.