Recursive projected sparse matrix recovery (ReProSMR) with application in real-time video layer separation

In this work, we propose an online algorithm Recursive Projected Sparse Matrix Recovery (ReProSMR) for recovering a time sequence of sparse matrix S_t and a time sequence of dense matrix L_t from their sum M_t = L_t + S_t when L_ts´ lies in a slowly changing low dimensional tensor subspace. A key application where this problem occurs is in video layer separation where the goal is to separate a video sequence into a slowly changing low dimensional background sequence and a sparse foreground sequence. Mathematically, a 2D image can be thought of as a second order tensor. ReProSMR is a modification of Recursive Projected Compressive Sensing (ReProCS) based on tensor PCA. Experimental comparisons demonstrating the advantages and computation gain of ReProSMR are shown for both simulated and real videos.