Penalty-based cluster validity index for class discovery from cancer data

In order to perform successful diagnosis and treatment of cancer, discovering and classifying cancer types correctly is essential. One of the challenges in cancer class discovery is to estimate the number of classes given a set of unknown microarray data. In the paper, we propose a new cluster validity criterion called Penalty-based Disagreement Index (PDI) based on the perturbation technique to estimate the number of classes in microarray data, PDI not only considers the disagreement between the partition results obtained from the original data and those obtained from the perturbed data, but also includes a penalty measure which is a function of the number of classes. Our experiments show that PDI successfully estimates the true number of classes in a number of challenging real cancer datasets.