Visual attention modeling for video quality assessment with structural similarity

Saliency map demonstrates the regions where the human eye will typically focus, and the key step of proposed video assessment metrics is to generate saliency map of each frame over the video sequence. However, it remains a challenge of the accuracy of saliency detection. In this paper, a lightweight multiscale approach is presented to extract spatial and temporal attention. The final saliency map is obtained by integrating two attention models. Spatial attention modeling takes into account the characteristics of the human visual system (HVS). In temporal attention modeling, we use a novel muItiscale approach method to extract motion feature, which is based on segmentation of objects in video sequence. As the temporal pooling schemes used in existing video quality assessment are only direct average or Minkowski summation of image quality scores over the video sequence, we use a procedure which takes HVS mechanism on video sequence into account adequately. In addition, the results demonstrate that the proposed method in this paper can improve the performance of video quality metrics obviously.