On limited fan-in optimal neural networks

This paper analyses the influence of limited fan-in on the size and VLSI optimality of highly interconnected nets. Two different approaches show that VLSI- and size-optimal discrete neural networks can be obtained for small fan-in values. They have applications to hardware implementations of neural networks. The first approach is based on implementing a certain sub-class of Boolean functions, F_n,m functions. We show that this class of functions can be implemented in VLSI-optimal (i.e., minimising AT²) neural networks of small constant fan-ins. The second approach is based on implementing Boolean functions for which the classical Shannon´s decomposition can be used. Such a solution has already been used by Alon-Bruck (1991) to prove bounds on neural networks with fan-ins limited to 2. We generalise the result presented there to arbitrary fan-in, and prove that the size is minimised by small fan-in values, while relative minimum size solutions can be obtained for fan-ins strictly lower than linear. Finally, a size-optimal neural network having small constant fan-ins is suggested for F_n,m functions