A training-time analysis of robustness in feed-forward neural networks

The paper addresses the analysis of robustness over training time issue. Robustness is evaluated in the large, without assuming the small perturbation hypothesis, by means of randomised algorithms. We discovered that robustness is a strict property of the model -as it is accuracy- and, hence, it depends on the particular neural network family, application, training algorithm and training starting point. Complex neural networks are hence not necessarily more robust than less complex topologies. An early stopping algorithm is finally suggested which extends the one based on the test set inspection with robustness aspects.