Cost functions to estimate a posteriori probabilities in multiclass problems

Author

Cid-Sueiro, JesÙs ; Arribas, Juan Ignacio ; Urbán-Munoz, Sebastián ; Figueiras-Vidal, Aníbal R.

Author_Institution

Dept. de Teoria de la Senal y Comunicaciones e Ing. Telematica, Univ. de Valiadolid, Spain

Volume

10

Issue

3

fYear

1999

fDate

5/1/1999 12:00:00 AM

Firstpage

645

Lastpage

656

Abstract

The problem of designing cost functions to estimate a posteriori probabilities in multiclass problems is addressed. We establish necessary and sufficient conditions that these costs must satisfy in one-class one-output networks whose outputs are consistent with probability laws. We focus our attention on a particular subset of the corresponding cost functions which verify two common properties: symmetry and separability (well-known cost functions, such as the quadratic cost or the cross entropy are particular cases in this subset). Finally, we present a universal stochastic gradient learning rule for single-layer networks, in the sense of minimizing a general version of these cost functions for a wide family of nonlinear activation functions

Keywords

entropy; estimation theory; learning (artificial intelligence); neural nets; pattern classification; probability; cost functions; cross entropy; estimate theory; multiclass problems; nonlinear activation functions; pattern classification; probability; separability; single-layer neural networks; stochastic gradient learning; symmetry; Bayesian methods; Convergence; Cost function; Entropy; Minimization methods; NP-complete problem; Neural networks; Pattern classification; Stochastic processes; Sufficient conditions;

fLanguage

English

Journal_Title

Neural Networks, IEEE Transactions on

Publisher

ieee

ISSN

1045-9227

Type

jour

DOI

10.1109/72.761724

Filename

761724