Exemplar-based frame rate up-conversion with congruent segmentation

A novel motion-compensated frame interpolation algorithm to increase video frame rates is proposed in this work, which employs texture optimization techniques to refine inaccurate motion vector fields. We enforce the congruent segment constraint in the motion refinement so that matching objects have similar local image structures. More specifically, we use the congruence energy as well as the appearance energy in the motion optimization to estimate high quality motion vectors. Simulation results show that, by preserving complex object shapes and texture, the proposed algorithm provides more faithful intermediate frames than conventional algorithms.