An empirical evaluation of an IR-based strategy for chat-oriented dialogue systems

This work reports the evaluation of a data-driven chatbot strategy, which is based on the vector space model IR framework. The evaluation is conducted by means of an empirical comparison between the proposed strategy and a baseline system that implements a similar, but naïve, strategy. The proposed chatbot strategy combines semantic evidence from both the current user input and the previous recent chatting-history, while the baseline system only uses the current user input as source of evidence. As a result of the conducted comparative evaluation, the proposed chatbot strategy happened to outperform the baseline strategy by a small, but statistically significant, difference.