Speed training: improving the rate of backpropagation learning through stochastic sample presentation

Artificial neural networks provide an effective empirical predictive model for pattern classification. However, using complex neural networks to learn very large training sets is often problematic, imposing prohibitive time constraints on the training process. We present four practical methods for dramatically decreasing training time through dynamic stochastic sample presentation, a technique we call speed training. These methods are shown to be robust to retaining generalization accuracy over a diverse collection of real world data sets. In particular, the stochastic presentation with error threshold technique achieves a training speedup of 4278% on a large OCR database with no detectable loss in generalization