On the generalisation ability and storage capacity of logical neural networks

The authors describe the supervised learning of Boolean functions in multilayer networks of probabilistic logic nodes. They focus on the generalization ability and storage capacity, which are both independent of the particular training algorithm used. The questions considered are: (1) given a net and a certain amount of training data which are samples of a function that the net can represent, how accurately will the net classify unseen data which are samples of the same function? and (2) given a net and a certain amount of training data which are samples of a random function, what is the probability of error-free storage? The first question is answered by using a simple model of the generalization process that gives an expression for the generalization error as a function of the network connectivities and number of training examples. This is compared with computer simulations of some small two-layer networks used to learn a particular function. The second question is addressed by giving an upper bound on the statistical storage capacity. Some basic definitions and concepts are introduced