Back propagation based on selective attention for fast convergence of training neural network

A back propagation neural network based on selective attention (SABP) is proposed for improving the learning speed of multi-layer artificial neural networks with sigmoid activation function. The algorithm embedded the selective attention model in the NN training. The key to this algorithm lies in the pre-attention function, in which primitive features of the stimulus were detected in parallel by what might be called feature detectors by an improved GA, and the attention function, in which detail features were handled with the attention spotlight by the BP with an enlarged error item in the output layer. Therefore, the SABP obtained the ability of searching the global optimum solution relying on the pre-attention function that made a lower fault rate to drop into local optimum, and the attention function updated the weights of neural network effectively by enlarging the error in the output layer, which kept a high leaning rate to meet the convergence criteria quickly. The simulation experiments demonstrate that the algorithm is more powerful than Momentum algorithm or Delta-bar-Delta rule in learning, and it can effectively avoid failure training caused by randomizing the initial weights and thresholds, and solve the slow convergence problem resulting from the Flat-Spots when the error signal becomes too small.