High speed parallel hardware performance issues for neural network applications

Neural network applications push the envelope of high speed computers in several areas. Pattern recognition applications often involve large amounts of training data and large neural networks. Significant preprocessing of the data is often necessary and real time operation is the ultimate goal. Each one of these steps stress different components of the computer system architecture. Since neural network architecture is inherently parallel, efficient mapping of the computations required for learning and classification should be very well suited for fast, high speed parallel computers. Two of the most used neural networks, multilayer backpropagation networks and competitive learning (Kohonen layer) neural networks, are examined and analyzed for parallel implementation. The common mathematical thread observed in the training and use of these networks is that the algorithms use standard linear algebra functions that operate on vectors and matrices. This common thread is observed also in radial basis function networks and probabilistic neural networks. The implications of this observation for parallel computer architecture are presented