Speech recognition using segmental neural nets

The authors present the concept of a segmental neural net (SNN) for phonetic modeling in continuous speech recognition (CSR) and demonstrate how this can be used with a multiple hypothesis (or N -Best) paradigm to combine different CSR systems. In particular, the authors developed a system that combines the SNN with a hidden Markov model (HMM) system. In a speaker-independent, 1000-word CSR test using a word-pair grammar, the error rate for the hybrid system dropped 25% from that of a state-of-the-art HMM system alone. By taking into account all the frames of a phonetic segment simultaneously, the SNN overcomes the well-known conditional-independence limitation of HMMs. The hybrid SNN/HMM system generates likely phonetic segmentations from the HMM N-best list, which are scored by the SNN. The HMM and SNN scores are then combined to optimize performance