Some experiments on training multilayer feedforward neural networks of hard-limiting units using random weights

Training multilayer feedforward neural networks of hard-limiting units remains a problem, because of the nondifferentiable output functions. However, if one adds Gaussian noise (with zero means) to each weight the weights will be random variables with smooth distribution functions. So one can use gradients to optimize the weights. This method was first presented by P.L. Bartlett and T. Downs (1992). In order to evaluate the effectiveness of the method, the authors did some experiments on training one-, two- and three-layer feedforward neural networks. The experiment results show that the method of Bartlett and Downs is effective for training one- and two-layer networks, but is very time-consuming for training networks with three (or more) layers