Automatic region of interest detection in multi-view video

This paper presents a novel algorithm in automatic region of interest detection for multi-view video sequences. We first group video frames along and across views as a group of picture (GOP). Key points or feature vectors representing textures existing in video frames in GOP are extracted using Scale-Invariant Feature Transform (SIFT). Key points are clustered using the K-means algorithm. Visual words are assigned to all key points based on their clusters. Patches represented small areas with textures are generated using the Maximally Stable Extremal Regions (MSER) operator. One patch can contain more than one key point, which leads to more than one visual word. Therefore, the patch can be represented by different visual words in different degrees. Motion detection algorithm is used to determine movement regions in video frames. Patches in the movement regions have higher likelihoods to be parts of the region of interest. With the developed spatial modeling, appearance modeling, depth estimation as well as the motion detection, we compute the likelihood which patches will belong to the region of interest. Depth estimation algorithms are used for grouping the homogeneous region with the same depth. With the depth estimation and motion information, the foreground plane will give an object region. The group of patches in the same depth with high likelihoods is clustered and indicated as the region of interest. The experimental results show that our proposed algorithm can automatically discover the regions of interest in multi-view video sequences correctly.