Saliency Inspired Full-Reference Quality Metrics for Packet-Loss-Impaired Video

This paper explores the application of saliency information for perceptual quality assessment of packet-loss-impaired videos. We propose and validate two categories of error measures for full-reference saliency based quality metrics. The first category uses a weighted average of pixel errors between original and distorted videos, where the weight at a pixel depends on its visual saliency determined by Itti´s saliency detection method after motion information incorporated. Motivated by the observation that packet-loss induced errors often change the spatial-temporal visual attention and correspondingly the saliency map, the second category measures the spatial deviation in saliency values between the original and distorted videos, and the temporal variation of saliency map of the distorted video, and further uses the products of both measures. We combine multiple error measures from the previous two categories using stepwise linear regression analysis. The final combined model includes three factors and provides significant gain over using the best single factor and other non-saliency based measurements.