An integrative evaluation framework for intelligent decision support systems

Evaluation of the overall effectiveness of decision support systems (DSS) has been a research topic since the early 1980s. As artificial intelligence methods have been incorporated into systems to create intelligent decision support systems (IDSS), researchers have attempted to quantify the value of the additional capabilities. Despite the useful and relevant insights generated by previous research, existing evaluation methodologies offer only a fragmented and incomplete view of IDSS value and the contribution of its technical infrastructure. This paper proposes an integrative, multiple criteria IDSS evaluation framework through a model that links the decision value of an IDSS to both the outcome from, and process of, decision making and down to specific components of the IDSS. The proposed methodology provides the designer and developer specific guidance on the intelligent tools most useful for a specific user with a particular decision problem. The proposed framework is illustrated by evaluating an actual IDSS that coordinates management of urban infrastructures.