A network pruning algorithm for combined function and derivative approximation

This paper describes newly discovered types of overfitting that occur when simultaneously fitting a function and its first derivatives with multilayer feedforward neural networks. We analyze the overfitting and demonstrate how it develops. These types of overfitting occur over very narrow regions in the input space, thus a validation set is not helpful in detecting them. A new pruning algorithm is proposed to eliminate these types of overfitting. Simulation results show that the pruning algorithm successfully eliminates the overfitting, produces smooth responses and provides excellent generalization capabilities. The proposed pruning algorithm can be used with any single-output, two-layer network, which uses a hyperbolic tangent transfer function in the hidden layer.