Bayesian foreground segmentation and tracking using pixel-wise background model and region based foreground model

In this paper we present a segmentation system for monocular video sequences with static camera that aims at foreground/background separation and tracking. We propose to combine a simple pixel-wise model for the background with a general purpose region based model for the foreground. The background is modeled using one Gaussian per pixel, thus achieving a precise and easy to update model. The foreground is modeled using a Gaussian mixture model with feature vectors consisting of the spatial (x, y) and colour (r, g, b) components. The spatial components of this model are updated using the expectation maximization algorithm after the classification of each frame. The background model is formulated in the 5 dimensional feature space in order to be able to apply a maximum a posteriori framework for the classification. The classification is done using a graph cut algorithm that allows taking into account neighborhood information. The results presented in the paper show the improvement of the system in situations where the foreground objects have similar colors to those of the background.