Incremental recursive fisher linear discriminant for online feature extraction

In this paper, we propose a new online feature extraction algorithm called Incremental Recursive Fisher Linear Discriminant (IRFLD). In the conventional Linear Discriminant Analysis (LDA), the number of discriminant vectors is limited to the number of classes minus one due to the rank of a between-class covariance matrix. However, the proposed IRFLD can remove this limitation. That is, an arbitrary number of discriminant vectors up to input dimensions can be obtained to construct a feature space. In the proposed IRFLD, the Pang et al.´s Incremental Linear Discriminant Analysis (ILDA) is extended such that effective discriminant vectors are recursively searched for the complementary space of a conventional discriminant space. In addition, a suitable number of effective discriminant vectors are automatically determined using a cross-validation method, where several representative training data are held as validation data and they are updated using the k-means clustering whenever a chunk of new training data are given. The performance of IRFLD is evaluated for 5 benchmark data sets. The experimental results show that the final classification accuracies of IRFLD are always better than those of ILDA. We also reveal that this performance improvement is attained by adding discriminant vectors in a complementary discriminant space.