Video sequence segmentation for object-based coders using higher order statistics

In this paper we propose a segmentation method aimed at separating the moving objects from the background in a generic video sequence. This task, accomplished at the coder site, is intended to support some new functionalities oriented to access and to decode single objects of a video sequence, foreseen by innovative multimedia scenarios, such as those focused during the MPEG4 work. The proposed segmentation method comprises a motion detection, that produces a preliminary segmentation map, and a subsequent regularization phase. The motion detection is essentially based on a Higher Order Statistics (HOS) test that employs a temporally, nonlinearly filtered version of the video sequence; this choice is motivated by HOS detection properties. The subsequent regularization phase provides a local connectivity constraint on the background-foreground map, and plays an important role in eliminating misclassifications due to noise, motion estimation ambiguities etc., of the original video sequence. The segmentation algorithm performance is illustrated by some experimental results carried out in the framework of the “ISO MPEG4 Core Experiments” activity in which the authors are currently involved