Neighbor annealing for neural network training

An extremely simple technique for training the weights of a feedforward multilayer neural network is described and tested The method, dubbed ldquoneighbor annealingrdquo is a simple random walk through weight space with a gradually decreasing step size. The approach is compared against backpropagation and particle swarm optimization on a variety of training tasks. Neighbor annealing is shown to perform as well or better on the test suite, and is also shown to have pragmatic advantages.