Measurement and Modeling of A Web-based Question Answering System

Question-answering (QA) is one of the most natural and effective channels for specific information acquisition. QA systems emerged several decades back. In this paper, we perform a measurement study of a popular, large scale Web-QA system, i.e. iAsk, and systematically investigate various behavior patterns and the system performance. To evaluate such a Web-QA system, we propose three performance metrics, namely reply-rate, reply-number, and reply-latency, which are most closely related to the QoS of a Web-QA system. Based on extensive measurement results, we propose a mathematical framework for the three performance metrics that capture our observation precisely. The framework reveals that the QoS of a Web-QA system actually heavily depends on three key factors: the user scale, user reply probability and a system design artifact (related to Webpage layout). We study their respective impacts on the system performance. Finally, we propose several ways through which current Web-QA system can be improved. To the best of our knowledge, this is the first piece of work that studies the performances of Web-QA systems