RaCoCl: Robust rank correlation based clustering - An exploratory study for high-dimensional data

The curse of dimensionality, which refers to both the combinatorial explosion in dimensions and the concentration of distances or norms in high dimensions, affects most of the clustering techniques. Recent studies on the concentration of norms suggest the use of a correlation measure instead of distances to more effectively judge (dis)similarity in high dimensions. In this work, based on these observations, we propose a robust rank correlation based clustering method. Specifically, we employ the recently proposed fuzzy gamma rank correlation measure. We show that this intuitively simple algorithm has the following advantages: (i) It requires very few parameters to be set, (ii) the number of clusters need not be apriori known, (iii) while there is an indirect dependence on the underlying distance measure, its makes use of both global and local information, (iv) it can be robust to noise depending on the correlation measure employed and, (v) as it is shown, performs well with high dimensional data. We illustrate the algorithm on some datasets where the traditional Fuzzy C-Means algorithm is known to fail.