Frame rate up conversion via image fusion based on variational approach

In this paper, we propose a novel framework for motion compensated frame rate up conversion. The proposed algorithm consists of two steps: generation of interpolated images and fusion of them. In the first step, we generate four different interpolated frames between the given two frames. Those frames are obtained through motion compensated interpolation, by using a different set of optical flow fields that are estimated from the four consecutive frames. In the second step, we fuse the four interpolated images into one by using a variational structure so that we can effectively remove outliers appearing due to false optical flows in occlusion regions. Experimental results demonstrate that the proposed algorithm improves objective and subjective visual qualities compared to the existing algorithms.