Gain elimination from backpropagation neural networks

It is shown that the gain of the sigmoidal activation function, as used in backpropagation neural networks, can be eliminated since there exists a well-defined relationship between the gain, the learning rate, and the set of initial weights. Similarly, it is also possible to eliminate the learning rate by adjusting the gain and the initial weights. This relationship is proven and extended to various variations of the backpropagation learning rule as well as applied to hardware implementations of neural networks