Spatial-temporal video browsing for mobile environment based on visual attention analysis

In this paper, we propose an attention based method to provide convenient browsing experience on mobile devices. To achieve this, we extract video summary in both temporal and spatial domains. Firstly, an improved video attention analysis method is performed to obtain attention distribution of video. Then, temporal summarization is carried out by extracting attended frames from attended shots to make the summary representative and fluent. And spatial summary is further generated by detecting the Regions of Interest (ROIs) of temporal summary frames. At this stage, to improve the viewing experience, a curve fitting based smoothing method is proposed to avoid drastic change of ROIs´ positions and sizes. Finally, we will obtain the spatial-temporal summary to be displayed on screens of mobile devices. Experimental results verify the effectiveness of this method.