Novel noise-filtering ability of heterogeneous five layered neural network trained identity mapping using BP algorithm

Shows that the heterogeneous five-layered network has much more robustness than the conventionally used three-layered network in spite of the cost for constructing the five-layered network, where each network is realizing identity mapping. It can detect the feature of the input-output relationship at the third layer units in the learning process and unfold the feature to the output layer units, so that adaptive outputs are obtained at the output layer units even if the input patterns corrupted with noise are given. The ability largely depends on the steepness of the sigmoid function used at the hidden layer(s). While this is true for both types of networks, the steepness of the activation function of the hidden layer units nearest to the input layer influences the five-layered network most, and selection of suitable steepness yields the best performance. The reason why the heterogeneous five-layered network has much more robustness than the heterogeneous three-layered network is investigated. How the noise-filtering is done at the hidden layer units with a steep sigmoid function is examined