A method for evaluating elicitation schemes for probabilistic models

Author

Wang, Haiqin ; Dash, Denver ; Druzdzel, Marek J.

Author_Institution

Decision Syst. Lab., Pittsburgh Univ., PA, USA

Volume

32

Issue

1

fYear

2002

fDate

2/1/2002 12:00:00 AM

Firstpage

38

Lastpage

43

Abstract

We present an objective approach for evaluating probability and structure elicitation methods in probabilistic models. The main idea is to use the model derived from the experts´ experience rather than the true model as the standard to compare the elicited model. We describe a general procedure by which it is possible to capture the data corresponding to the expert´s beliefs, and we present a simple experiment in which we utilize this technique to compare three methods for eliciting discrete probabilities: 1) direct numerical assessment, 2) the probability wheel, and 3) the scaled probability bar. We show that for our domain, the scaled probability bar is the most effective tool for probability elicitation

Keywords

belief networks; expert systems; knowledge acquisition; probability; Bayesian network; data capture; direct numerical assessment; discrete probabilities; elicitation scheme evaluation; expert beliefs; expert experience; objective approach; probabilistic models; probability elicitation; probability wheel; scaled probability bar; structure elicitation methods; Bayesian methods; Buildings; Career development; Databases; Electronic mail; Engineering profession; Frequency; Intelligent systems; Laboratories; Wheels;

fLanguage

English

Journal_Title

Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on

Publisher

ieee

ISSN

1083-4419

Type

jour

DOI

10.1109/3477.979958

Filename

979958