Self-organizing neural networks for class-separability features

We describe novel networks and self-organizing learning algorithms to extract features that are effective for preserving class separability. These are entirely different from features used for data representation such as the principal component analysis (PCA) features. The features discussed in this study are: (1) multivariate linear discriminant analysis (LDA) features in the multi-class case, and (2) linear features from the Bhattacharyya distance measure in a two-class case. Contributions in this study are: (1) we describe a novel adaptive network training algorithm that is proven to converge to the above features with probability one; (2) the training procedure is well-suited for online applications, and the network inputs are considered as a flow of data. The means and covariances of the training data are also computed adaptively; (3) we discuss two-layer linear networks to extract the LDA and Bhattacharyya distance features; (4) we prove the convergence of the two-layer networks to the respective features with probability one by stochastic approximation theory; (5) we present a new adaptive solution for the generalized eigenvectors of two correlation matrices. This is a generalization of the existing PCA algorithms; and (6) we present examples of the new networks for multi-class random data