Minimum entropy estimation as a near maximum-likelihood method and its application in system identification with non-Gaussian noise

We derive the minimum entropy estimation (MEE) method from information theory to show the similarity of this method to the maximum likelihood method for the linear regression problem. The result is a nonparametric-based identification technique that can be applied in any case with iid noise that outperforms estimators in this case, including the popular LS method and a recently-developed (and limited) version of the MEE. Performance-wise, the MEE method is comparable to the expectation-maximization (EM) method. Its application to FIR system identification produces a very efficient implementation of this technique.