Relations between fault tolerance and internal representations for multi-layer perceptrons

Fault tolerance is often mentioned as an important, intrinsic property of neural networks but it has not often been the subject of directed study. Such fault tolerance as does exist must depend strongly upon the exact nature of the internal representations captured during training, and the way these are distributed across the network. A representative pattern-recognition task is used to assess the fault tolerance of feedforward neural nets as a function of hidden-layer size. Damage resistance is found to increase with the number of hidden neurons, although this finding is sensitive to the exact performance metric employed. The technique of augmentation, which can increase the fault tolerance of a net of given size, is described. To understand the relations between the obtained measures of fault tolerance and the nets´ internal representations, a number of analyses are used. These techniques have more usually been used to simplify network structure by identifying and removing redundancies, but they are equally applicable to the study of fault tolerance