Extracting Partitional Clusters from Heterogeneous Datasets using Mutual Entropy

Clustering has traditionally been used for partitioning the objects of a single dataset. Some applications may require the clustering of multiple related heterogeneous datasets where it may not be easy to compute a useful and effective integrated feature space. In this paper, we present an algorithm called CEMENT (Cluster Ensemble using Mutual ENTropy) to address the problem of clustering two related datasets where the datasets represent the same or overlapping sets of objects but use different feature sets. The algorithm takes the partitional clusters generated from two datasets as input and uses a constraint-based approach to generate a single set of clusters. CEMENT is an EM (expectation maximization) approach where the objective function is the mutual entropy between the two sets of clusters. The algorithm was applied to the problem of clustering a document collection consisting of journal abstracts from ten different Library of Congress categories. These documents were pre-processed using several NLP (natural language processing) steps to extract syntactic and semantic feature sets. We present empirical results and statistical tests showing that CEMENT yields higher quality clusters with this dataset than several baseline clustering approaches.