Compressed spatio-temporal descriptors for video matching and retrieval

The contents of a video can be described in terms of appearance and motion of the scenes. In this paper, we propose a compressed spatio-temporal descriptor that is suitable for video matching and retrieval tasks. We use a modified wavelet based compression technique that exploits the temporal redundancy of the data using optical flow. In order to achieve a compact flow representation, a spline based technique is used. The optical flow field gives the directions along which the gray levels have regular variations in time. Wavelet decomposition along these directions results in fewer coefficients and thus higher compression. We demonstrate that the wavelet coefficients and flow parameters can be efficiently used for 1) video retrieval and matching, and 2) calculating spatio-temporal similarity between articulated objects. The results are demonstrated on several sequences.