Image processing with optimum neural networks

Although neurons in intermediate layers (hidden units) are very important components of most neural networks, there is a lack of general rules specifying how many hidden layers and how many hidden units per layer should be used to achieve optimum performance of a network. This lack of rules has its roots in the difficulty in judging the performance of the hidden units. The most widely used training algorithm, error back-propagation, allows the measurement of the performance of hidden units only indirectly by propagating their output to the output units, and then propagating the deviation from a desired output back to the hidden units under examination. To overcome these disadvantages, a performance measure, based on information theory, which allows every group of units (in particular groups of hidden units) to be judged, is developed. This performance measure enables us a network to be trained layer-by-layer without referring to particular connections in deeper layers