User-Model-Based Evaluation for Interactive Image Retrieval

User-system interaction is sometimes a cumbersome element of non-textual information access. Image retrieval systems now incorporate various interaction mechanisms. However, there is no standard or established way of interacting that has been proven to be most effective. Therefore, many system designers are introducing or eliminating various functionalities. The bottleneck in the development effort is finding an effective and fair way to evaluate these systems. For static information access, several test collections have commonly been used. For interactive information access, expensive laboratory testing is needed. This paper presents a user semi-automatic user-model-based evaluation method that is designed to reduce cost while maintaining objectivity.