Nonstationary AR modeling and constrained recursive estimation of the displacement field

An approach to constrained recursive estimation of the displacement vector field (DVF) in image sequences is presented. An estimate of the displacement vector at the working point is obtained by minimizing the linearized displaced frame difference based on a set of observations that belong to a causal neighborhood (mask). An expression for the variance of the linearization error (noise) is obtained. Because the estimation of the DVF is an ill-posed problem, the solution is constrained by considering an autoregressive (AR) model for the DVF. A nonstationary AR model of the DVF is also considered. Additional information about the solution is incorporated into the algorithm using a causal oriented smoothness constraint. A set theoretic regularization approach based on this formulation results in a weighted constrained least-squares estimation of the DVF. The algorithm shows an improved performance with respect to accuracy, robustness of occlusion, and smoothness of the estimated DVF when applied to typical videoconferencing scenes