A novel parameter update procedure based on minimizing the empirical probability of error [pattern classification applications]

This paper proposes a novel parameter update procedure based on minimizing the empirical probability of error. For a two-class classifier using a discriminant function to classify patterns, the decision boundary is given by the condition that the discriminant function is equal to zero. Therefore, the probability of error is determined by the discriminant function. In many applications, the empirical probability of error is used to estimate the probability of error. The new algorithm was directly derived from the empirical probability of error when a neural network with non-conventional output functions for hidden units is used to approximate the step function. Numerical examples showed that the network with the new algorithm can achieve accuracy comparable with conventional back propagation. The results also showed that the network converges very quickly. Therefore, the new algorithm can be a very useful alternative to back propagation.