On confidence and response times of human observers in subjective image quality assessment

Mean opinion scores obtained in subjective image quality experiments are widely accepted as measures of perceived visual quality. They have, however, a strong limitation regarding the reliability of the rated quality, since there is no explicit information as to whether the human observer experienced difficulties when judging image quality. We thus suggest that additional information about the observers confidence should be provided along with the actual quality measure. In this paper, we analyse two ways of obtaining this confidence measure; firstly as a confidence score given by the human observer and secondly as an indirect measure of the observers response time to provide the quality score. We reveal strong relationships of confidence scores and response times to the quality scores. We further propose a model to predict observer confidence based on the quality scores and response times.