Analysis-by-synthesis frame dropping algorithm together with a novel speech recognizer using time-varying hidden Markov model

In distributed speech recognition applications, variable frame rate (VFR) analysis is a technique that can reduce the channel bandwidth and computation resources. In this method, slowly changing frames that provide little information are abandoned. Rapidly changing frames, on the other hand, that are more related to speech perception are preserved. In this paper, we proposed an analysis-by-synthesis (AbS) frame dropping algorithm together with a novel VFR decoding method for hidden Markov modeling of speech. A recursive formula for the calculation of forward probability function of the VFR observations was derived and was used to form a time-varying hidden Markov model (tvHMM) with transition probabilities that are depended on the time difference between successive observations. A generalized Viterbi decoding algorithm was developed to decode the VFR observations. We also use an example to explain the decoding process for a particular VFR observation sequence. Experiments were conducted to investigate the effectiveness of the proposed AbS-tvHMM method. The experimental results show that our method can achieve essentially the same accuracy as full frame rate observations at frame rate of only 40 % and significantly reduces the computation time.