A family of hybrid neural networks

The focus of this study is on a family of hybrid architectures for feed-forward multi-layer neural networks and issues that arise in their design. The main objective in the design of this family has been to reduce the complexity of hardware, and hence make possible the implementation of larger networks for practical applications, by two main ideas: trading time for circuit complexity by a multiplexing scheme and a modular characteristic that allows multi-chip realizations without a prohibitive number of interconnections. In this paper, we propose to bring the various forms of this architecture together, which are at this time scattered in the literature. After presenting the main points in its operation, we will proceed to permutations and trade-offs, some of which have not been published in accessible literature so far. We start with the introduction of the basic architecture. We then present modifications and discuss some I/O issues. Matching neural transfer characteristics is important to the performance of the system and we address this problem with a set of second order improvements. Another version of the architecture, with external weight memory, is introduced which allows interaction with a host computer, and finally, a pipelined version of the architecture is presented that improves system speed with a small increment in overall complexity