Unsupervised information theory-based training algorithms for multilayer neural networks

The author describes a novel learning algorithm for multilayer neural networks. The training neural networks are used as a vector quantizer (VQ) in a hidden Markov model (HMM)-based speech recognition system. The approach offers the following innovations: (1) it represents an unsupervised learning algorithm for multilayer neural networks. (Usually, multilayer neural networks are only trained in supervised mode); (2) information theory principles are used as learning criteria for the neural networks; and (3) the neural networks are not trained using the standard backpropagation algorithm, by using instead a new unsupervised learning procedure. The use of a neural network as a VQ trained with the new algorithm in combination with an HMM-based speech recognition system results in a 25% error reduction compared to the same HMM system using a standard k-means vector quantizer