Joint space-time image sequence segmentation based on volume competition and level sets

We address the issue of joint space-time segmentation of image sequences. Typical approaches to such segmentation consider two image frames at a time, and perform tracking of individual segments across time. We propose to perform this segmentation jointly over multiple frames. This leads to a 3D segmentation, i.e., a search for a volume "carved out" by a moving object in the (3D) image sequence domain. We pose the problem in a Bayesian framework and use the MAP criterion. Under suitable structural and segmentation/motion models we convert MAP estimation to a functional minimization. The resulting problem can be viewed as volume competition, a 3D generalization of region competition. We parameterize the unknown surface to be estimated, but rather than solving for it using an active-surface approach, we embed it into a higher-dimensional function and use the level-set methodology. We show experimental results for the simpler case of object motion against a still background although, given suitable models, the general formulation can handle complex motion too.