Recursive On-Line ${(2{\rm D})}^2{\rm PCA}$ and Its Application to Long-Term Background Subtraction

In this paper, we propose a novel background subtraction method which enables reliable detection of foreground objects in a long surveillance video stream. Recently, although much progress has been made in the field of background subtraction, there are still challenging scenarios (e.g., high frequency motion of dynamic texture, non-stationary motion of camera, abrupt changes of illumination, etc.) in the long surveillance videos in which even state-of-the-art methods are often prone to fail. To cope with these challenging scenarios effectively, in the proposed method, a background model is initialized in a low-dimensional subspace and then updated periodically based on a novel recursive on-line (2D)²PCA algorithm developed in this paper. Moreover, a threshold map is also updated in a scene-adaptive manner for labeling each pixel in a scene either foreground or background independently. Based on this on-line framework, the background of a surveillance video stream is reconstructed over time, thereby facilitating the detection of foreground objects reliably. In extensive experiments, we demonstrate that the proposed background subtraction method can cope with the aforementioned challenging scenarios more favorably than the state-of-the-art methods.