Application of coding theory to neural net capacity

There are different definitions of neural net capacity. One of these definitions is what is called statistical pattern capacity. This capacity is defined as the average number of random patterns with random binary desired responses that can be “recognized” using a neural net. Another definition is what we may call worst case capacity. Consider a multilayer net with M input nodes and k output nodes. The “storage” capacity of this net is defined as the maximum number of input patterns for which the network can produce all possible output binary k-tuples. We use information theory, especially Shannon capacity theorem, to relate the neural net capacity to the channel capacity. As a practical example we demonstrate the effectiveness of error correcting codes to mitigate the imperfections of neural nets