Aspects of instantaneous on-line learning rules

In neural and fuzzy learning systems, instantaneous learning rules have often been proposed for use within online adaptive modelling and control schemes. However many aspects of this work remain unexplained or only partially known such as: how do these learning rules deal with singular systems? what happens when the data are inconsistent? how is on-line parameter convergence related to that of standard gradient descent rules? and is momentum beneficial to the parameter estimation procedure? This paper investigates all of these topics, suggests modifications to the basic procedures where necessary and describes some of the reformulations which have been previously proposed.