Improvements on handwritten digit recognition by genetic selection of neural network topology and by augmented training

The paper presents two ways of improving the handwritten digit recognition ability of a neural network. First by selection of the number of hidden units in the neural network, and second, by training using an augmented set of patterns. The handwritten digit recognition application is performed on feed-forward, fully connected neural networks with two hidden layer architectures. A genetic algorithm is used to search among configurations of two unequal hidden layer networks to find the optimum number of hidden units. Training procedures involving augmented sets of training patterns are produced by two methods: by shifting and by magnification every handwritten digit of the original training set. Results show that the best two hidden layer network, 178×26 units, trained for 10 different random starting weight sets in centered mode, i.e., no shifting, resulted in 82.74% average recognition rate with standard deviation, STD=0.66. The best two hidden layer network, 154×58, trained for 11 different random starting weight sets in shifting mode, resulted in 92.09% average recognition rate with STD=0.37. A comparison of recognition rates for centered versus shifting training modes for the genetic selection resulted statistically significant with p<0.001