Automated virtual camera during mobile video playback

This paper details an approach for novel video viewing experience on small-screen mobile devices. Users are allowed to choose a region-of-interest via a touch-based gesture, following which a virtual camera algorithm takes control in order to automatically zoom and pan within the video scene. The selected region-of-interest is kept within a retargeted viewport despite change in the position of the region-of-interest. Scene transformations such as change in the region-of-interest, disappearance, and re-appearance of a region-of-interest are handled gracefully. The system works on sampled frames and hence is favorable in conditions requiring low power consumption. The system implements (1) a pattern match based region tracking algorithm that works on frames sampled at as low as 1fps, and (2) a virtual camera algorithm that simulates virtual camera zoom/pan operations at 30fps. We evaluated the performance of the algorithm in terms of precision and recall scores. We show that the system has a high precision of 95% and a recall of 80% for a diversity of video scenes.