The optimum error nonlinearity in LMS adaptation with an independent and identically distributed input

The class of LMS algorithms employing a general error nonlinearity is considered. The calculus of variations is employed to obtain the optimum error nonlinearity for an independent and identically distributed input. The nonlinearity represents a unifying view of error nonlinearities in LMS adaptation. In particular, it subsumes two recently developed optimum nonlinearities for arbitrary and Gaussian inputs. Moreover, several more familiar algorithms such as the LMS algorithm, the least-mean fourth (LMF) algorithm and its family, and the mixed norm algorithm employ (non)linearities that are actually approximations of the optimum nonlinearity.