Semi-Supervised Covariance Estimation Using Clustering for Small Sample Size Problem

Small sample size (SSS) problem is still a puzzle in classification. For the generally used Maximum Likelihood Classifier (MLC), the small size of training sample with respect to the high-dimensional data deteriorates the performance of parameter estimation. In the extreme case, MLC could be disabled by the singularity of the covariance matrices. Generally speaking, predecessors experimented in two aspects to solve the matter. One aspect is dimensionality reduction for higher dimensional data; the other is covariance estimation. Classical methods to estimate covariance matrix calculate the common covariance matrix of various forms to replace or modify the sample covariance. However, most methods work in the supervised pattern and consider little about the useful information contained in the vast number of unlabeled samples. In view of this, we propose a semi-supervised scheme for covariance estimation. The method mainly involves clustering of all samples to obtain C clusters and the corresponding covariance matrices(C is the class number), and combination of the matrices generated by clustering and original sample covariance to modify the latter. Four experiments are carried out to assess the performance of proposed method and the results show the method performs well.