Theoretical investigation of the robustness of multilayer perceptrons: analysis of the linear case and extension to nonlinear networks

In this paper, we address the problem of robustness in multilayer perceptrons. We present the main theoretical results in the case of linear neural networks with one hidden layer in order to go beyond the empirical study we previously made. We show that the robustness can greatly be improved and that even without decreasing performance in normal use. Finally, we show how this behavior, clearly demonstrated in the linear case, is an approximation of the behavior of nonlinear networks