The two spirals benchmark: lessons from the hidden layers

A classic benchmark of nonlinear discrimination, the two-dimensional pattern of two intertwined spirals, is employed to demonstrate an approach to neural network performance improvement. It is generally recognized that the simple three layer backpropagation network does not succeed in solving the problem. By graphically displaying the hidden unit outputs and their resulting spirals, it becomes clear that this basic network correctly identifies the curved shape dimension but fails to recognize the spatial relationship between the spirals. These results suggest variations on the architecture and transformations of the input that lead to a better solution. The best results are obtained by extending the input vectors with a sum-of-squares value and its reciprocal, and creating peer-level hidden layers. An economic cost-benefit discussion of the networks justifies the model selection