Novelty and topicality in interactive information retrieval

The information science research community is characterized by a paradigm split, with a system-centered cluster working on information retrieval (IR) algorithms and a user-centered cluster working on user behavior. The two clusters rarely leverage each otherʹs insight and strength. One major suggestion from user-centered studies is to treat the relevance judgment of documents as a subjective, multidimensional, and dynamic concept rather than treating it as objective and based on topicality only. This study explores the possibility to enhance usersʹ topicality-based relevance judgment with subjective novelty judgment in interactive IR. A set of systems is developed which differs in the way the novelty judgment is incorporated. In particular, this study compares systems which assume that usersʹ novelty judgment is directed to a certain subtopic area and those which assume that usersʹ novelty judgment is undirected. This study also compares systems which assume that users judge a document based on topicality first and then novelty in a stepwise, noncompensatory fashion and those which assume that users consider topicality and novelty simultaneously and as compensatory to each other. The user study shows that systems assuming directed novelty in general have higher relevance precision, but systems assuming a stepwise judgment process and systems assuming a compensatory judgment process are not significantly different.