A scalable learning system for video recognition

Learning has become an essential part of many image and video processing systems, but it is not often used as an end-to-end solution. Some of the most successful demonstrations of end-to-end learning have been with convolutional, or shared weight networks. We are interested in how this approach can scale and have developed a flexible framework for implementing and training large scale convolutional networks called Harpo. We present an overview of the Harpo framework and describe a multilevel learning strategy used to optimize convolutional networks for particular features of interest in video data streams. Harpo is designed to exploit reconfigurable hardware to accelerate massively parallel convolutional network components and achieve real-time processing speeds. In this paper, we present initial software experiments which use the system to segment exhaust plumes coming from military vehicles in unmanned aerial vehicle video data