Partitioning capabilities of two-layer neural networks

It has been observed that feedforward neural nets with a single hidden layer are capable of forming either convex decision regions or nonconvex but connected decision regions in the input space. In this correspondence, it is shown that two-layer nets with a single hidden layer are capable of forming disconnected decision regions as well. In addition to giving examples of the phenomenon, it is explained why and how disconnected decision regions are formed. Through the hypothesization of the existence of additional virtual cells formed by the first layer, it is shown how the decision regions formed by the second layer can indeed be disconnected. It is shown that the number of such disconnected regions can be very large. Using a recent theoretical result about the sufficiency of two layers to approximate arbitrary decision regions in a finite portion of the space, an example is given of how that is possible with the use of virtual cells