Log adaptive filters. Structures and analysis for the scalar case

Feed-forward multi-layer neural networks (MLNNs) are complex nonlinear learning systems which can be trained by well-known rules such as back-propagation (BP). The resulting adaptation procedures are extremely difficult to analyze for stochastic training data. Significant analytic results have been obtained for the single-layer case and for some simple two-layer cases. Previously, a structural simplification has been studied which models each threshold function as a linear device. This linearized MLNN can only create hyperplane decision rules after convergence. However, the multiplicative behavior of the layers may offer some performance advantages over linear adaptive algorithms (LMS or RLS) when used for a linear problem. A new log-domain linear MLNN adaptive structure is proposed and analyzed here. The log operation converts the layer multiplications into additions whereupon linear analysis techniques can be used. The transient and steady-state statistical behavior of the log linear MLNN is analyzed for Gaussian training data. Deterministic recursions are derived for the mean and fluctuation behavior of the new algorithm. These recursion are shown to be in excellent agreement with Monte Carlo simulations