Structural drivers of function in information processing networks

Structural configuration drives functionality in a range of different natural and artificial information processing systems. In this study, we use feedforward neural networks to evaluate the impact of structural variations on the ability of a network to learn and retain representations of external information. Performance is evaluated by statistically analyzing the error in the representations produced by parallel and layered networks during supervised, sequential function approximation. By varying the initial network state and the time given to learn the information, we identify tradeoffs between configurations that optimize for the best versus worst case scenarios and for the production of accurate versus retainable and generalizable representations of information. We show that these tradeoffs are maintained in larger networks and for variations in the information presented to the networks. By characterizing the curvature, depth, and participation of network connections about local error landscape minima, we find that variations in landscape structure give rise to the observed tradeoffs in performance. Consistently deep, narrow minima enable parallel networks to produce highly accurate solutions at the cost of more frequent failure in retention and generalizability. In contrast, variability in the depth and curvature of local minima enables layered networks to produce coarse but generalizable solutions at the cost of hindering consistent accuracy. Identifying structural drivers of functional performance is crucial for understanding both successes and limitations of information processing systems.