Title of article :
An evaluation of adaptive filtering in the context of realistic task-based information exploration
Author/Authors :
Daqing He، نويسنده , , Peter Brusilovsky، نويسنده , , Jaewook Ahn، نويسنده , , Jonathan Grady، نويسنده , , Rosta Farzan، نويسنده , , Yefei Peng، نويسنده , , Yiming Yang، نويسنده , , Monica Rogati، نويسنده ,
Issue Information :
دوماهنامه با شماره پیاپی سال 2008
Pages :
23
From page :
511
To page :
533
Abstract :
Exploratory search increasingly becomes an important research topic. Our interests focus on task-based information exploration, a specific type of exploratory search performed by a range of professional users, such as intelligence analysts. In this paper, we present an evaluation framework designed specifically for assessing and comparing performance of innovative information access tools created to support the work of intelligence analysts in the context of task-based information exploration. The motivation for the development of this framework came from our needs for testing systems in task-based information exploration, which cannot be satisfied by existing frameworks. The new framework is closely tied with the kind of tasks that intelligence analysts perform: complex, dynamic, and multiple facets and multiple stages. It views the user rather than the information system as the center of the evaluation, and examines how well users are served by the systems in their tasks. The evaluation framework examines the support of the systems at users’ major information access stages, such as information foraging and sense-making. The framework is accompanied by a reference test collection that has 18 tasks scenarios and corresponding passage-level ground truth annotations. To demonstrate the usage of the framework and the reference test collection, we present a specific evaluation study on CAFÉ, an adaptive filtering engine designed for supporting task-based information exploration. This study is a successful use case of the framework, and the study indeed revealed various aspects of the information systems and their roles in supporting task-based information exploration.
Keywords :
User Study , Intelligence analysts , Task-based information exploration , Exploratory search , adaptive filtering , CAFE , Evaluation framework
Journal title :
Information Processing and Management
Serial Year :
2008
Journal title :
Information Processing and Management
Record number :
1228745
Link To Document :
بازگشت