Understanding the behaviour and optimising the performance of back-propagation neural networks: an empirical study

In recent years, back-propagation neural networks have become a popular tool for modelling environmental systems. However, as a result of the relative newness of the technique to this field, users appear to have limited knowledge about how ANNs operate and how to optimise their performance. In this paper, the stages observed when training a back-propagation neural network are examined in detail for a particular case study; the forecasting of salinity in the River Murray at Murray Bridge, South Australia, 14 days in advance. Particular attention is paid to the behaviour of the network as it approaches a local minimum in the error surface. The effect of the presence of infrequent patterns in the training set on generalisation ability is investigated. The nature of the error surface in the vicinity of local minima is examined and options for optimising network performance (i.e. training speed and generalisation ability) are presented for real time forecasting situations.