Motion field modeling for video sequences

In this paper, we propose a model for the interframe correspondences existing between pixels of an image sequence. These correspondences form the elements of a field called the motion field. In our model, spatial neighborhoods of motion elements are related based on a generalization of autoregressive (AR) modeling of time-series. We also propose a joint spatiotemporal model by including spatial neighborhoods of pixel intensities in the motion model. A fundamental difference of our approach with most previous approaches to modeling motion is in basing our model on concepts from statistical signal processing. The developments in this paper give rise to the promise of extending well-understood tools of signal processing (e.g., filtering) to the analysis and processing of motion fields. Simulation results presented show the excellent performance of our models in interframe prediction; specifically, on average the motion model performs 29% better in terms of mean squared error energy over a commonly used pel-recursive approach [1]. The spatiotemporal model improves prediction efficiencies by 8% over the motion model. Our model can also be used to obtain estimates of the optical flow field as simulations will demonstrate.