Title :
A method for evaluating elicitation schemes for probabilistic models
Author :
Wang, Haiqin ; Dash, Denver ; Druzdzel, Marek J.
Author_Institution :
Decision Syst. Lab., Pittsburgh Univ., PA, USA
fDate :
2/1/2002 12:00:00 AM
Abstract :
We present an objective approach for evaluating probability and structure elicitation methods in probabilistic models. The main idea is to use the model derived from the experts´ experience rather than the true model as the standard to compare the elicited model. We describe a general procedure by which it is possible to capture the data corresponding to the expert´s beliefs, and we present a simple experiment in which we utilize this technique to compare three methods for eliciting discrete probabilities: 1) direct numerical assessment, 2) the probability wheel, and 3) the scaled probability bar. We show that for our domain, the scaled probability bar is the most effective tool for probability elicitation
Keywords :
belief networks; expert systems; knowledge acquisition; probability; Bayesian network; data capture; direct numerical assessment; discrete probabilities; elicitation scheme evaluation; expert beliefs; expert experience; objective approach; probabilistic models; probability elicitation; probability wheel; scaled probability bar; structure elicitation methods; Bayesian methods; Buildings; Career development; Databases; Electronic mail; Engineering profession; Frequency; Intelligent systems; Laboratories; Wheels;
Journal_Title :
Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on
DOI :
10.1109/3477.979958