Theory of Monte Carlo sampling-based Alopex algorithms for neural networks

We propose two novel Monte Carlo sampling-based Alopex (ALgorithm Of Pattern EXtraction) algorithms for training neural networks. The proposed algorithms naturally combine the sequential Monte Carlo estimation and Alopex-like procedure for gradient-free optimization, and the learning proceeds within the recursive Bayesian estimation framework. Experimental results on various problems show encouraging convergence results.