Optimize real-valued RBM with Bidirectional Autoencoder

Deep learning increasingly attracted attention after the fast training method of Restricted Boltzmann Machine(RBM) is proposed[1]. Many researches directly constructed deep architecture with stack RBMs to learn the representation of the data, few studied the optimization method to get good RBM parameters. Here proposes a new optimization method for real-valued RBM by minimizing the reconstructed error. Firstly, build and initialize Bidirectional Autoencoder(Bi-Ae). Secondly, minimize the cost function with Stochastic Gradient Descent (SGD) to get the parameters. Thirdly, convert the Bi-AE into RBM with the most suitable parameters. Experiments are executed on the MNIST dataset. Compared with PSO and likelihood maximum optimization methods, the reconstructed errors of the proposed method is 4.02% smaller than the result from error[1], which is advanced.