Addressing Visual Consistency in Video Retargeting: A Refined Homogeneous Approach

For the video retargeting problem which adjusts video content into a smaller display device, it is not clear how to balance the three conflicting design objectives: 1) visual interestingness preservation; 2) temporal retargeting consistency; and 3) nondeformation. To understand their perceptual importance, we first identify that the latter two play a dominating role in making the retargeting results appealing. Then a statistical study on human response to the targeting scale is carried out, suggesting that the global preservation of contents pursued by most existing approaches is not necessary. Based on the newly prioritized objectives and the statistical findings, we design a video retargeting system which, as a refined homogeneous approach, addresses the temporal consistency issue holistically and is still capable of preserving high degree of visual interestingness. In particular, we propose a volume retargeting cost metric to jointly consider the retargeting objectives and formulate video retargeting as an optimization problem in graph representation. A dynamic programming solution is then given. In addition, we introduce a nonlinear fusion based attention model to measure the visual interestingness distribution. The experiment results from both image rendering and subjective tests indicate that our proposed attention modeling and video retargeting system outperform their conventional methods, respectively.