Region Tree Based Sparse Model for Optical Flow Estimation

Nonlocal regularization has been verified as an effective way to estimate optical flow. Most work in this line constructs the regularizer by only considering the structure of regular grid-like nonlocal neighborhood, but not explicitly takes advantage of the global structure. In this paper, we propose to construct a super pixel based region tree to explicitly incorporate the global structure information into the regularizer. To make use of this non-regular nonlocal (NRNL) regularizer to obtain region-wise smooth and discontinuity preserving flow filed, we first reconstruct the flow for each super pixel by sparse representation, and then dynamically select the super pixel flow with the lowest energy as the optimally-recovered flow field, which corresponds to the optimal sub-region tree. Finally, we update the flow alternatively through continuous optimization. Incorporating the super pixel and sparse representation method not only constrains the nonlocal information that comes from homogeneous region, but also removes the intermediate flow field noise. Experiments on the Middlebury benchmark demonstrate the effectiveness of our method.