Entropy of delta-coded speech

The paper presents an analysis of the information properties of delta-coded speech with channel-encoding applications. Predictive coding techniques for reducing the entropy of the average distribution of the signal elements are described, and evaluated by comparison with computations of the relative entropy of Markov process approximations to the message-generating process, for orders up to 9. The optimal digital fixed-structure predictors are established for orders up to 7, and the optimal group codes are established for block lengths of 2 to 6 elements. The redundancy is shown to be typically about one half, and a predictor success probability of 0.9 is attainable with a practical 6th-order discrete structure. The entropies of the sequences, which are generated by modulo-2 addition of the predictions and source elements, are found to be much closer to the process entropies than are the corresponding performance characteristics for group encoding. In order to achieve message compression, for encoding the predictor error sequence, 5-element group encoding (attaining a compression factor of 0.5 with a 300-word buffer) is found to be superior to run-length encodings. The combination transformation is much more efficient than direct exact coding of blocks of source elements.