Quantifying sequential context effects in subjective quality evaluation

ITU-T P.800 Absolute Category Rating (ACR) listening tests are an important tool for evaluating the audio quality of codecs for telephony. However, listener responses in these tests do not only depend on the audio quality of the items heard but also on the preceding items and the given ratings given to them. Here we investigate how strong these effects are in typical ACR listening tests and to what extent they can alter the measured quality of the codecs. We find that the preceding codec and the preceding rating influence the ratings in opposite ways. High quality stimuli or low ratings tend to provoke lower ratings for the following stimuli. Preceding high ratings or low quality stimuli lead to higher ratings. It is shown that effects are generally small and will have little effect on the predicted overall quality differences between the tested conditions if the stimuli order is randomized for individual subjects.