Structure and rank awareness for error and data flow reduction in phase-shift-based ToF imaging systems using compressive sensing

Phase-shift-based Time-of-Flight (ToF) imaging systems estimate the distances from the camera to the scene points from the phase shift undergone by a modulated light signal, projected onto the scene, instead of actually performing time measurements. The phase shift is typically computed from several values of the cross-correlation between the light signal received by each pixel and a reference signal at the pixel level. This means that several acquisitions per depth image are needed, producing a series of raw images, which have to be transmitted to a processing unit to generate the depth image. It is well known that these raw images admit a sparse representation in an appropriate domain and that such representation is often not completely free, but follows a certain structure, e.g., tree structure of natural images in wavelet domain. Furthermore, we show that raw images share the same support in its sparse representation. The structured sparsity allows raw images to be efficiently recovered from few measurements using Compressed Sensing (CS), while the common support makes feasible simultaneous recovery of all raw images in a Multiple Measurement Vector (MMV) framework. Conventional depth estimation methods might require gathering measurements that are redundant, i.e., some of them could be represented as linear combinations of the others. This means that the matrix of measurements in a MMV recovery framework is rank-deficient, making rank awareness an important point of the approach. In this paper we present a modification of the Rank Aware Order Recursive Matching Pursuit (RA-ORMP) algorithm that accounts for structured sparsity and apply it to recover raw data from a Photonic Mixer Device (PMD) depth sensor. Our results show a clear noise reduction, both in the recovered images and the final depth estimation, achieved by means of a robust joint support estimation, while enabling considerable data flow reduction.