Discriminative Feature Learning for Video Semantic Segmentation

In this paper, we propose a novel deep learning based method for video semantic segmentation. Specially, we utilize 3D convolution neural network (3D CNN) to learn discriminative hierarchical features from spatial-temporal volumes for accurate pixel labelling. The learned features are capable of capturing both appearance and motion information. To align the pixel labels along real object boundaries, as well as maintain local consistency, we further perform graph-cut on a graph constructed on coherent 3d regions, or super-voxels, extracted from input video. Experiments demonstrate that due to the discriminative capability of learned features, our approach can obtain competitive labelling accuracy compared to the state-of-art in absence of sophisticated inference models, even with few training samples.