A novel approach for detection and localization of caption in video based on pixel pairs

This paper proposed a method for localization of caption text, particularly Kannada caption text from video sequence. The proposed method uses multiple frames integration approach based on three characteristics namely character location, edge distribution and pixel contrast. The novelty of our approach lies on the pairing of pixels by finding double edge maps of the original and its rotated image. To highlight the text area based on the video temporal information, a Roberts´ edge detector is applied to reference gray scale frames of both original and rotated images, followed by combination of the two edge maps and later on a multiple frame integration based on the above three characteristics by employing logical AND operator that keeps only pixels that are invariant among the frames. The morphological operations are applied on the edge map to connect the text characters and discard nontext components. The result is then smoothed and overlaid on one of the original reference images to extract text candidate block. The experimental results carried out on sample video data of Kannada TV channel (Commercial) show that the proposed approach achieves a high precision and recall rate.