Mutual information neural networks for dynamic pattern recognition tasks

This paper presents a new probabilistic neural network paradigm, which is especially suitable for dynamic pattern recognition problems. Such problems, with time-varying patterns of arbitrary length occur in many important pattern recognition tasks, e.g. in speech recognition, handwriting recognition, or image sequence identification. It is demonstrated that hybrid systems are very efficient tools for solving dynamic pattern recognition tasks. Such hybrid systems consist of the combination of neural networks and statistical pattern classifiers. It is proved that neural networks trained on the maximum mutual information principle are optimal for the construction of hybrid systems. The theoretical foundations of the resulting hybrid system are explained, as well as the basic principles of the information theory-based neural network learning algorithms. Furthermore, it is shown how those algorithms are implemented and that the resulting hybrid systems achieve superior performance in various applications involving the identification of time-varying patterns