Selection of the Lagrange multiplier for block-based motion estimation criteria

In hybrid video coding, motion vectors used for motion compensation constitute an important set of decisions. Cost functions for block motion estimation that take the smoothness of the resulting motion vector field into account, in addition to the motion compensated prediction error, have been proposed. Computationally simple derivatives of sum of absolute differences and sum of squared differences-based criteria are studied in this paper. Cost functions are based on Lagrangian rate-distortion formulation, and the basic question is how the Lagrangian multiplier involved should be selected. Assumptions behind these cost functions are discussed, and a new method is derived for determining the multiplier. Comparisons with other strategies are made with experiments. The results show that the selection of the multiplier is not critical.