Mean shift clustering-based moving object segmentation in the H.264 compressed domain

This study presents a mean shift clustering-based moving object segmentation approach in the H.264 compressed domain. The motion information extracted from H.264 compressed video, including motion vectors (MVs) and partitioned block size, are used as segmentation cues. The MVs are processed by normalisation, weighted 3D median filter and motion compensation to obtain a reliable and salient MV field. The partitioned block size is used as a measure of motion texture in the process of the MV field. Based on the processed MV field, the authors employ the mean shift-based mode seeking in spatial, temporal and range domain to develop a new approach for compact representation of the MV field. Then, the MV field is segmented into different motion-homogenous regions by clustering the modes with small spatial and range distance, and each object is represented by some dominant modes. Experimental results for several H.264 compressed video sequences demonstrate good performance and efficiency of the proposed segmentation approach.