Binary classification by stochastic neural nets

We classify points in R^d (feature vector space) by functions related to feedforward artificial neural networks. These functions, dubbed “stochastic neural nets”, arise in a natural way from probabilistic as well as from statistical considerations. The probabilistic idea is to define a classifying bit locally by using the sign of a hidden state-dependent noisy linear function of the feature vector as a new (d+1)th coordinate of the vector. This (d+1)-dimensional distribution is approximated by a mixture distribution. The statistical idea is that the approximating mixtures, and hence the a posteriori class probability functions (stochastic neural nets) defined by them, can be conveniently trained either by maximum likelihood or by a Bayes criterion through the use of an appropriate expectation-maximization algorithm