Observation Model Based on Scale Interactions for Optical Flow Estimation

In this paper, an original observation model for multiresolution optical flow estimation is introduced. Multiresolution frameworks, often based on coarse-to-fine warping strategies, are widely used by state-of-the-art optical flow methods. They allow the recovery of large motions by successive estimations of the flow field at several resolution levels. Although such approaches perform very efficiently and usually lead to faster minimizations, they generally consider independent problems at each resolution levels and do not exploit the existing interactions between scales (especially the influences of fine scales on larger ones). In this paper, we tackle this issue by proposing a flexible framework, inspired from fluid mechanics, able to partly counter these limitations. For each resolution level, our process filters the equations of interest and decomposes the key variables into resolved (i.e., at a given resolution) and unresolved (i.e., at finer resolutions) components. This enables to derive a new data term that takes into account, at coarse resolutions, the influence of their unresolved parts. From this new term, we propose two different estimation strategies, depending on whether we explicitly know the type of relations between the different scales (as for physical processes) or not. In order to test the efficiency of this new observation model, we have embedded it in a simple multiresolution Lucas-Kanade estimator. Comparing the usual optical flow constraint equation with this new term in the same motion estimation procedure, it clearly appears that the proposed term leads to more consistent estimates and prevents from errors propagation apparition during the estimation. In all situations (synthetic, real, physical images or not), our new term is able to greatly improve the results compared with usual conservation constraints.