Stationary subtitle processing for real-time frame rate up-conversion

Frame rate up-conversion (FRUC) is a popular video processing technique used extensively to enhance the temporal resolution of video sequences. In order to obtain a high quality interpolation, the motion vector field (MVF) between frames must be well-estimated. However, many current block-based motion estimation (BME) techniques are prone to errors in blocks with subtitles and cause artifacts in or around the subtitle regions, because they do not consider the possibility of stationary subtitles. In this paper, we propose a pre-post processing algorithm with subtitle detection and in-painting to improve the quality of FRUC. By using the proposed algorithm, stationary subtitles are erased to eliminate their negative effects on the BME. The motion compensated frame (MCF) is finally overlaid with the erased subtitle. Unlike many conventional text detection or removal algorithms, the proposed algorithm is designed to be suitable for implementation of real-time FRUC.