Investigation of tandem deep belief network approach for phoneme recognition

This paper proposes using tandem DBN approach - a hierarchical architecture that consists of two or more deep belief networks (DBNs) in tandem manner - for phoneme recognition task on TIMIT. First we describe the standard DBN approach applied in phoneme recognition and discuss the motivation of combining it with tandem classifier approach. We then perform series of experiments to find out the best configuration for the DBN in the second level and discover the full potential of this method. The experiments show that for the DBN in the second level, (a) 2048 units in each hidden layer is better than 1024 and 512 units, (b) for sufficient length of temporal context, two hidden layers are better, (c) the one gives best performance on development set shows 4% relative improvement on coretest set.