Unsupervised video segmentation by dynamic volume growing and multivariate volume merging using color-texture-gradient features

We propose a new unsupervised technique for segmentation of digital video that partitions its constituents by identifying homogeneous sub-volumes within the data treated as a three dimensional (3-D) spatio-temporal volume. Our approach is commenced by subjecting the input video to a 3-D gradient detection method that determines the magnitude of color changes across the volume. The computed gradient is utilized to guide a volume growing procedure, initiated at spatio-temporal locations with small gradient magnitudes and concluded at locations with large gradient magnitudes, to yield an initial set of homogeneous sub-volumes. These partitions are further refined by integrating them with an entropy-based texture descriptor as well as color and gradient features in a multivariate volume merging procedure that fuses sub-volumes with similar attributes, to yield the final segmentation. Our approach was tested on several simple-to-complex video sequences with favorable results.