On computational limitations of neural network architectures

Recently, there have been many attempts to develop neural network architectures which are capable of intelligent behavior. Thorough analyses of two-layer nets have been provided in the past. However, analyses of the computational abilities of multi-layer nets have been suffered so far from their inherent complexity of interaction. The paper introduces a powerful method for analyzing the computational abilities of neural nets which is based on algorithmic information theory. The method shows that the idea of many interacting computing units, as present in neural networks, does not essentially facilitate the task of constructing intelligent systems. The same holds for building powerful learning systems