A 1.9nJ/pixel embedded deep neural network processor for high speed visual attention in a mobile vision recognition SoC

An energy-efficient Deep Neural Network (DNN) processor is proposed for high-speed Visual Attention (VA) engine in a mobile vision SoC. The proposed embedded DNN realizes VA to rapidly find ROI tiles of potential target objects reducing ~70% of recognition workloads of vision processor. Compared to previous VA, the DNN VA reduces execution time by 90%, which results in 73.4% overall OR time reduction. Highly-parallel 200-way PEs are implemented in the DNN processor with 2D image sliding architecture, and only 3ms of DNN VA latency can be obtained. Also, the dual-mode PE configuration is proposed for both DNN and multi-layer-perceptron (MLP) to share same hardware for high energy efficiency. As a result, the proposed work achieves only 1.9nJ/pixel energy efficiency which is 7.7x smaller than state-of-the-art VA accelerator.