On Evaluating the Contribution of Validation for Question Answering

Validation is arising as a crucial component of new architectures aimed at improving Question Answering technologies. Hence, there is a strong need of appropriate measures for evaluating Validation and, even more, its impact on question answering results. However, common Validation measures do not allow a clear study of this impact, and they might even lead researchers to obtain wrong conclusions. We propose a new approach for evaluating Validation technologies, which offers clear and useful information about the impact on QA performance. We compare our proposal with classic evaluation measures, showing the benefits of our scheme.