On improved learning algorithms with adaptive parameter regulation in feedforward nets

This paper should contribute to a structured and theoretical view of the backpropagation algorithm and some of its well-known extensions. Based on a mathematical investigation of the algorithms conditions for structured improvements and developments of these techniques are described. The construction of adaptive parameter regulations for learning and momentum rate follow. These parameter regulations allow the presentation of adaptive learning techniques. It is shown that off-line versions of these techniques represent minimization methods which are exact in mathematical sense. Under consideration of complexity conditions on-line algorithms are preferred and described in detail. Finally their numerical behaviour is investigated and simulation results are presented in comparison with standard algorithms