Recursive neural filters and dynamical range transformers

A recursive neural filter employs a recursive neural network to process a measurement process to estimate a signal process. This paper reviews and presents new results on recursive neural filters through introducing those based on a multilayer perceptron with output feedbacks and proposing dynamical range reducers and extenders for applications where the range of the measurement or signal process expands in time or is too large for the recurrent neural network to handle for the filtering resolution required. As opposed to the conventional analytic approach to deriving filter equations, recursive neural filters are synthesized from exemplary realizations of the signal and measurement processes, which are obtained by either computer simulations or actual experiments. No assumptions such as linear dynamics, Gaussian distribution, additive noise, and Markov property are required. A properly trained recursive neural filter with a proper architecture carries the most "informative" statistics in its dynamical state and approximates the optimal performance to any accuracy. The recursive neural filter is a massively parallel algorithm ideal for real-time implementation. Dynamical range reducers and extenders proposed are preprocessors and postprocessors of a recurrent neural network that, respectively, reduce the range of the exogenous input and extend that of the output of the recurrent neural network. While a dynamical range reducer reduces the exogenous input range by subtracting an estimate of the exogenous input from it, a dynamical range extender extends the output range by adding an estimate of the output to it. Three types of dynamical range reducer by estimate subtraction and five types of dynamical range extender by estimate addition are provided, which have different levels of effectiveness and computational costs. Selection from the different types for an application is determined by the tradeoff between the effectiveness and computational cost. Two types of dynamical range extender by estimate addition use an extended Kalman filter (EKF) to generate the added estimate. They can be viewed as methods of using a recursive neural filter only to make up the nonlinear part ignored by the EKF.