Title :
Challenges for IF performance evaluation in practice
Author :
Van Laere, Joeri
Author_Institution :
Inf. Res. Centre, Univ. of Skovde, Skovde, Sweden
Abstract :
This study presents a literature analysis of 52 publications on the topic of information fusion performance evaluation. The aim is to identify whether information fusion researchers are aware of the problematic nature of performance evaluation in practice, which is substantially different from performance evaluation in test environments. As expected, only 6% of the publications do actually report on studies conducted in practice, and therefore more such studies are needed. However, 58% of the publications do critically reflect on the problematic nature of performance evaluation in practice. As ground truth is lacking in practice, there is a need for (a set of) performance measures which are both comprehensive and adaptable to domain or situation context and changing circumstances over time. More studies need to address the overall question how multiple performance measures can function in concert and how exactly can be judged whether adaptation has been an improvement or not.
Keywords :
decision making; sensor fusion; decision making; information fusion; performance evaluation; Aquaculture; Decision making; Informatics; Information analysis; Licenses; Marine animals; Performance analysis; Performance evaluation; Testing; Time measurement; Performance evaluation; literature survey;
Conference_Titel :
Information Fusion, 2009. FUSION '09. 12th International Conference on
Conference_Location :
Seattle, WA
Print_ISBN :
978-0-9824-4380-4