Facilitating motion-based vision applications by combined video analysis and coding

In order to jointly optimise the quality of video coding on one hand and video analysis on the other, this paper proposes a novel approach to enhance the reusable information content in compressed video domain. By introducing a hierarchical content driven motion estimation mechanism at the encoder, complemented by a statistical prediction of region-of-interest, this approach reduces the complexity and yet increases robustness of the compressed domain vision analysis applications. Taking the object tracking application as an example, we demonstrate that the motion vectors generated by the proposed method can be directly used to extract object information, achieving tracking performance comparable with a pixel domain approach. In addition, we show that the incurred rate distortion (RD) overheads and the effect on encoder complexity are minimal, especially when compared to the reduction of processing required for video analysis targeting a wide spectrum of computer vision applications.