A unifying viewpoint of multilayer perceptrons and hidden Markov models

A generic iterative model for artificial neural networks (ANNs) is proposed which covers a wide variety of existing neural networks: single-layer feedback networks, multilayer feedforward networks, hierarchical competitive networks, and hidden Markov models. From the phase-retrieve point of view, the hidden Markov models described by the trellis structure can be regarded as a homogeneous (recurrent) multilayer perceptron with nonlinear squashing activation function. From the learning-phase point of view, it is shown that the additive gradient descent (ascent) approaches can be used to derive the back-propagation learning in the multilayer perceptrons. On the other hand, the multiplicative gradient descent (ascent) approach can be successfully applied to the trellis structure and used to derive the Baum-Welch reestimation formulation in the hidden Markov models