Content-based video retrieval based on object motion trajectory

This paper proposed a content-based video retrieval system based on object motion trajectory. An algorithm for tracking moving objects in MPEG-compressed domain is developed. It is to link individual macroblocks in the temporal domain first and then prune and merge the formed paths by considering spatial adjacency of MBs. In this way, the difficult spatial segmentation problem of traditional methods is avoided and tracking of multiple deformed objects can be achieved. Also, our system is capable of eliminating global motion so that camera motion is allowed. The extracted object motion trajectory is then converted into a form conformable to MPEG-7 motion descriptor (keypoints + interpolating functions). Both interfaces of query-by-example and query-by-sketch are provided and problems in descriptor matching (e.g., mismatch in keypoint interval and video time duration) are solved to achieve robustness and a high recall rate. We have tested our algorithm with real video clips, including fixed- or moving-camera, rigid or deformed, single or multiple objects, varying object size during motion, etc. Exeriments show that the tracking and retrieval results are satisfactory and suitable for further applications.