On the capacity of three-layer networks

A new lower bound on the capacity of three-layer neural networks is given by a constructive method which uses the sparse encoding scheme. The capacity is closely related to the learning ability from examples, giving an estimate of the number of examples to be required for learning a function. It is shown that the capacity m of a neural network with h hidden elements and n input lines satisfies (in the sense of order) nh/log (nh²)<m <nh log h