Improving expert judgment by coherence weighting

This paper extends previous efforts to increase accuracy of probability estimates by weighting multiple probabilistic judgments according to their coherence. The experiment we report involves elicitation of probability estimates that belong to sets in which only one probability is of primary interest; the other estimates serve to measure the individual judges´ coherence within sets. Our method shows that asking for only two additional judgments can achieve significant increases in accuracy over a simple linear average. In the aggregation, we adjust the judgments to be coherent and give more weight to judgments that require less adjustment.