A simultaneous learning method for both activation functions and connection weights of multilayer neural networks

This paper proposes a simultaneous learning algorithm for both activation functions and connection weights. The activation function is composed of several basic functions, such as sigmoidal function, Gaussian function and so on. In order to avoid local minima, the activation functions are controlled and randomly disturbed every some epochs. The activation functions are automatically optimized for each application. Probability and speed of learning are higher than the conventionals