Subject bias: Introducing a theoretical user model

We propose a model for rating behavior based on subject bias and subject error. Evidence for subject bias can be found in freely available subjective experiments. When subject bias is removed from ratings, the sensitivity of statistical comparisons between stimuli usually improves. According to our model, subject biases characterize the subject pool. These between-subject differences are important when analyzing and comparing people. On the other hand, it is advantageous to remove subject bias when analyzing mean opinion score. We conclude that bias acts like a random variable within ratings.