Title :
Kneser–Ney Smoothing With a Correcting Transformation for Small Data Sets
Author_Institution :
Smart Desktop, Seattle
Abstract :
We present a technique which improves the Kneser-Ney smoothing algorithm on small data sets for bigrams, and we develop a numerical algorithm which computes the parameters for the heuristic formula with a correction. We give motivation for the formula with correction on a simple example. Using the same example, we show the possible difficulties one may run into with the numerical algorithm. Applying the algorithm to test data we show how the new formula improves the results on cross-entropy.
Keywords :
entropy; maximum likelihood estimation; optimisation; probability; smoothing methods; speech recognition; Kneser-Ney smoothing algorithm; bigrams; correcting transformation; cross-entropy; heuristic formula; maximum-likelihood estimation; numerical algorithm; probability; small data sets; speech processing; speech recognition; Character recognition; Entropy; Handwriting recognition; Maximum likelihood detection; Maximum likelihood estimation; Optical character recognition software; Smoothing methods; Speech recognition; Testing; Speech processing; speech recognition;
Journal_Title :
Audio, Speech, and Language Processing, IEEE Transactions on
DOI :
10.1109/TASL.2007.900090
