Comparison of subjective viewing test methods for image quality assessment

This paper presents a comparison study on subjective quality scores obtained by both single stimulus (without reference) and triple stimulus (with reference) methods. The TID2013 database is reevaluated by single stimulus approach, which is realized by absolute category rating (ACR). And the mean opinion score (MOS) provided along with TID2013 represents the results from triple-stimulus pair comparison (3-stimulus PC) method. In the end, the correlation coefficient and hypothesis testing are used to determine if there is a significant difference between both sets of scores. The experimental results show that the differences exist and are significant for some specific distortion types or image contents. We believe the findings in this work can benefit and facilitate the future subjective visual quality viewing tests.