DocumentCode :
1026961
Title :
Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor autodetermination
Author :
Xu, Lei
Author_Institution :
Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, China
Volume :
15
Issue :
4
fYear :
2004
fDate :
7/1/2004 12:00:00 AM
Firstpage :
885
Lastpage :
902
Abstract :
The nature of Bayesian Ying-Yang harmony learning is reexamined from an information theoretic perspective. Not only its ability for model selection and regularization is explained with new insights, but also discussions are made on its relations and differences from the studies of minimum description length (MDL), Bayesian approach, the bit-back based MDL, Akaike information criterion (AIC), maximum likelihood, information geometry, Helmholtz machines, and variational approximation. Moreover, a generalized projection geometry is introduced for further understanding such a new mechanism. Furthermore, new algorithms are also developed for implementing Gaussian factor analysis (FA) and non-Gaussian factor analysis (NFA) such that selecting appropriate factors is automatically made during parameter learning.
Keywords :
Bayes methods; Helmholtz equations; information theory; learning (artificial intelligence); maximum likelihood estimation; variational techniques; Bayesian Ying-Yang learning; Gaussian factor analysis; Helmholtz machines; automatic model selection; bit-back approach; generalized projection geometry; harmony learning; independent factor autodetermination; information geometry; information-theoretic perspective; maximum likelihood method; minimum description length; variational approximation; Algorithm design and analysis; Bayesian methods; Independent component analysis; Information geometry; Machine learning; Maximum likelihood estimation; Mean square error methods; Predictive models; Principal component analysis; Solid modeling; Algorithms; Artificial Intelligence; Bayes Theorem; Computer Simulation; Decision Support Techniques; Factor Analysis, Statistical; Information Storage and Retrieval; Information Theory; Models, Statistical; Neural Networks (Computer); Pattern Recognition, Automated; Probability Learning;
fLanguage :
English
Journal_Title :
Neural Networks, IEEE Transactions on
Publisher :
ieee
ISSN :
1045-9227
Type :
jour
DOI :
10.1109/TNN.2004.828767
Filename :
1310361
Link To Document :
بازگشت