DocumentCode :
1898253
Title :
Score matching for models with latent variables
Author :
Dikmen, Onur ; Cemgil, A. Taylan
Author_Institution :
CNRS LTCI, Telecom ParisTech, Paris, France
fYear :
2011
fDate :
20-22 April 2011
Firstpage :
801
Lastpage :
804
Abstract :
Undirected graphical models such as Markov random fields or Boltzmann machines prove useful in many signal processing and machine learning tasks. However, parameter estimation in these models is difficult due to the intractable normalising constant in their probability density functions. One powerful technique for parameter estimation in such models is score matching. This technique makes use of an objective function which is independent of the normalising constant and constitutes locally consistent estimators for the parameters of such models. However, score matching is only applicable to fully-observed models. In this paper, we extend the applicability of score matching to models with latent variables. Our estimators are unbiased, based on Monte Carlo integration. Unbiased gradient estimators open the way to optimisation through stochastic approximation. We demonstrate the performance of our methodology on two synthetic problems.
Keywords :
Monte Carlo methods; approximation theory; gradient methods; parameter estimation; probability; signal processing; stochastic programming; Boltzmann machine; Markov random field; Monte Carlo integration; intractable normalising constant; latent variable; machine learning task; objective function; optimisation; parameter estimation; probability density function; score matching; signal processing; stochastic approximation; unbiased gradient estimator; Atmospheric modeling;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Signal Processing and Communications Applications (SIU), 2011 IEEE 19th Conference on
Conference_Location :
Antalya
Print_ISBN :
978-1-4577-0462-8
Electronic_ISBN :
978-1-4577-0461-1
Type :
conf
DOI :
10.1109/SIU.2011.5929772
Filename :
5929772
Link To Document :
بازگشت