Title :
A universal learning rule that minimizes well-formed cost functions
Author :
Mora-Jiménez, Inma ; Cid-Sueiro, Jesús
Author_Institution :
Dept. of Signal Theor. & Commun., Univ. Carlos III de Madrid, Leganes-Madrid, Spain
fDate :
7/1/2005 12:00:00 AM
Abstract :
In this paper, we analyze stochastic gradient learning rules for posterior probability estimation using networks with a single layer of weights and a general nonlinear activation function. We provide necessary and sufficient conditions on the learning rules and the activation function to obtain probability estimates. Also, we extend the concept of well-formed cost function, proposed by Wittner and Denker, to multiclass problems, and we provide theoretical results showing the advantages of this kind of objective functions.
Keywords :
gradient methods; learning (artificial intelligence); maximum likelihood estimation; neural nets; probability; cost function minimization; neural network; nonlinear activation function; posterior probability estimation; stochastic gradient learning rule; universal learning rule; Amplitude modulation; Artificial neural networks; Bayesian methods; Cost function; Entropy; Logistics; Medical diagnosis; Optimization methods; Stochastic processes; Sufficient conditions; Generalized soft perceptron (GSP); stochastic learning rule; strict sense Bayesian (SSB) cost function; well-formed cost function; Algorithms; Cluster Analysis; Computer Simulation; Computing Methodologies; Decision Support Techniques; Models, Biological; Models, Statistical; Neural Networks (Computer); Numerical Analysis, Computer-Assisted; Pattern Recognition, Automated; Stochastic Processes;
Journal_Title :
Neural Networks, IEEE Transactions on
DOI :
10.1109/TNN.2005.849839
