DocumentCode :
1783199
Title :
Using hierarchical dirichlet processes to regulate weight parameters of Restricted Boltzmann Machines
Author :
Wenbing Huang ; Fuchun Sun
Author_Institution :
Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China
fYear :
2014
fDate :
28-29 Sept. 2014
Firstpage :
1
Lastpage :
8
Abstract :
Restricted Boltzmann Machines (RBM) have been widely applied to solve various problems in machine learning. Much research has been performed to study the structures of RBM, such as sparsity and probabilistic distributions of hidden units. However, little attention has been paid to investigating the features of weight components that connect visible and hidden layers. In this paper, we formulate a nonparametric Bayesian RBM model, in the sense that Hierarchical Dirichlet Process (HDP) is imposed as a prior of weights. Thus, the original RBM is decomposed as a group-structured machine, where the groups are revealed by HDP. The clustering effect of HDP is helpful to simplify the structure of RBM and the hierarchical structure of our model is advantageous to maintain the diversity of weight components within each group. The Monte Carlo EM (MCEM) algorithm is adopted to perform weight training and hyperparameter estimation. Various experiments verify the effectiveness of our proposed model.
Keywords :
Boltzmann machines; Monte Carlo methods; belief networks; learning (artificial intelligence); parameter estimation; HDP; MCEM algorithm; Monte Carlo EM; hierarchical Dirichlet process; hyperparameter estimation; machine learning; nonparametric Bayesian RBM model; restricted Boltzmann machines; weight parameters; weight training; Bayes methods; Computational modeling; Data models; Equations; Mathematical model; Monte Carlo methods; Probabilistic logic; Hierarchical Dirichlet Process; Restricted Boltzmann Machines; nonparametric method; weights regulation;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Multisensor Fusion and Information Integration for Intelligent Systems (MFI), 2014 International Conference on
Conference_Location :
Beijing
Print_ISBN :
978-1-4799-6731-5
Type :
conf
DOI :
10.1109/MFI.2014.6997741
Filename :
6997741
Link To Document :
بازگشت