Self-organization of Gaussian mixture model for learning class pdfs in pattern classification

This paper proposes a new approach to pattern classification problems using a neural network, BAYESNET. The network is designed to identify the class of an unlabelled pattern using the Bayesian decision theory. Since the theory explicitly requires the information of pattern distributions, the network has the capability of learning probability density functions (pdfs) of classes. To estimate pdfs, we adopt parametric estimation with Gaussian mixture model: a class is assumed to be composed of a number of subclasses, each of whose patterns has a Gaussian distribution. The BAYESNET learning includes two subprocesses, the initialization process and the main learning process. In the initialization process, the number of subclasses for each class by automatic generation and elimination of subclasses. Generation occurs when samples assigned to a subclass turns out to have an unexpected distribution, checked by the Chi-square test. A subclass is eliminated when the subclass contributes negligibly to forming the class pdf. The role of the main learning process is to fine-tune parameters of classes and subclasses. Though the learning is based on the winner-take-all scheme, one of the novelties of BAYESNET lies in the undeterministic winner selection. Unlike the usual nearest-neighbor selection, our network selects the winner according to the probability obtained by the current estimation. Due to the selection rule, the final parameter values are assured to agree with the maximum likelihood estimation. Besides, biases are added to the winner selection in order to help avoiding local minima states.