Improvement of generalization ability for identifying dynamic systems by using universal learning networks

This paper studies how the generalization ability of models of dynamic systems can be improved by taking advantages of the second order derivatives of the outputs of networks with respect to the external inputs. The proposed method can be regarded as a direct implementation of the well-known regularization technique using the higher order derivatives of the universal learning networks (ULNs). ULNs consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The method for computing the second order derivatives of ULNs is discussed. A new method for implementing the regularization term is presented. Finally, simulation studies on identification of a nonlinear dynamic system with noises were carried out to demonstrate the effectiveness of the proposed method