Minimum error entropy estimation and entropic prediction filtering: An optimal predictive coding scheme

The minimum error entropy (MEE) estimation problem on discrete, finite, or countably infinite ensembles of arbitrary statistical nature is considered. A combinatorial approach is taken to solve the minimization problem. The solution is obtained in algorithmic form. The numerical complexity of the MEE algorithm is analyzed. An entropic prediction filtering (EPF) coding scheme that utilizes the MEE algorithm to minimize the per symbol error entropy is introduced. Simulation has shown that the EPF can effectively be used to reduce the average codeword length required to encode data. Other applications of the MEE algorithm in signal interpolation and extrapolation are also discussed.