A statistical analysis of neural computation

Summary form only given. This paper presents an architecture and learning algorithm for a feedforward neural network implementing a two pattern (image) classifier. By considering the input pixels to be random variables, a statistical binary hypothesis (likelihood ratio) test is implemented. A linear threshold separates p[X|H₀] and p[X|H ₁], minimizing a risk function. In this manner, a single neuron is considered as a BSC with the PDF error tails under the threshold yielding the cross-over probability ε. A single layer of neurons is viewed as a parallel bank of independent BSC´s. Which is equivalent to a single effective BSC representing that layer´s hypothesis testing performance. A multiple layer network is viewed as a cascade of BSC channels, and which again collapses into a single effective BSC. The effective BSC channel capacity is examined as the information theoretic ability of the network to extract the single bit of information encoded in the multidimensional input vector and pass the resulting fractional information bit of information to the one dimensional output “decision” variable