Learning capability and storage capacity of two-hidden-layer feedforward networks

The problem of the necessary complexity of neural networks is of interest in applications. In this paper, learning capability and storage capacity of feedforward neural networks are considered. We markedly improve the recent results by introducing neural-network modularity logically. This paper rigorously proves in a constructive method that two-hidden-layer feedforward networks (TLFNs) with 2√(m+2)N (≪N) hidden neurons can learn any N distinct samples (x_i, t_i) with any arbitrarily small error, where m is the required number of output neurons. It implies that the required number of hidden neurons needed in feedforward networks can be decreased significantly, comparing with previous results. Conversely, a TLFN with Q hidden neurons can store at least Q²/4(m+2) any distinct data (x_i, t_i) with any desired precision.