Title :
Optimal model inference for Bayesian mixture of experts
Author :
Ueda, Naonori ; Ghahramani, Zoubin
Author_Institution :
NTT Commun. Sci. Labs., Kyoto, Japan
Abstract :
We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective function derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of an MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method
Keywords :
Bayes methods; deterministic algorithms; inference mechanisms; Bayesian mixture of experts; deterministic algorithm; mixture of experts model; model structure; objective function; optimal model inference; variational Bayesian framework; Approximation algorithms; Bayesian methods; Clustering algorithms; Educational institutions; Inference algorithms; Laboratories; Maximum likelihood estimation; Monte Carlo methods; Random variables; Training data;
Conference_Titel :
Neural Networks for Signal Processing X, 2000. Proceedings of the 2000 IEEE Signal Processing Society Workshop
Conference_Location :
Sydney, NSW
Print_ISBN :
0-7803-6278-0
DOI :
10.1109/NNSP.2000.889405
