Maximum Frame Rate Video Acquisition Using Adaptive Compressed Sensing

Compressed sensing is a novel technology to acquire and reconstruct sparse signals below the Nyquist rate. It has great potential in image and video acquisition to explore data redundancy and to significantly reduce the number of collected data. In this paper, we explore the temporal redundancy in videos, and propose a block-based adaptive framework for compressed video sampling. To address independent movement of different regions in a video, the proposed framework classifies blocks into different types depending on their inter-frame correlation, and adjusts the sampling and reconstruction strategy accordingly. Our framework also considers the diverse texture complexity of different regions, and adaptively adjusts the number of measurements collected for each region. The proposed framework also includes a frame rate selection module that selects the maximum achievable frame rate from a list of candidate frame rates under the hardware sampling rate and the perceptual quality constraints. Our simulation results show that compared to traditional raster scan, the proposed framework can increase the frame rate by up to six times depending on the scene complexity and the video quality constraint. We also observe a 1.5-7.8 dB gain in the average peak signal-to-noise ratio of the reconstructed frames when compared with prior works on compressed video sensing.