Neural versus difference equation modeling for 2D pattern recognition problems

The article study compares neural network based models and difference equation based models in 2D pattern recognition of handwritten digits. Pattern classifier models based on two neural network models were implemented using multi-layer feed forward neural networks: single layer perceptron and a multi-layer perceptron with one hidden layer. These two neural network models have nonlinear sigmoidal threshold units. A classifier model based on difference equations with linear weighted sum of all inputs was also implemented. Beside linear inputs, nonlinearities were introduced in all of the models by including products of inputs as new inputs. All models were trained/adjusted and tested using a handwritten digit database composed of a training set of 2360 digits and a testing set of 1320 digits. Each handwritten digit is composed of 15×23 binary pixels and each pixel is considered as a single input to the classifier. Classifier performance was measured as the correct classification rate on the testing database. Results show that the difference equation model with only linear inputs yielded the worse results (72.4%). Nevertheless, as products among inputs were included as new inputs, the classification performance increased (84.1%), thus exceeding the results of the single-layer perceptron with linear inputs (72.7%). The single-layer perceptron with nonlinear inputs gave an improvement of the classification result to 86.3%. The products among inputs can be interpreted as convolutional filtering over the input image to detect lines in different directions producing an overall improvement in the classification performance of all models