Consensus diagnosis: a simulation study

Consensus diagnoses arise when several experts contribute their opinions about the relative merits of a series of competing hypothesis, and a single decision maker combines their responses and makes a decision without further discussion among the contributors. Consensus diagnoses were simulated by allowing an oracle to generate `opinions´ based on universal background knowledge and all available information about the specific problem being diagnosed. Contributors´ opinions were generated by perturbing the oracle´s opinion; the size of the perturbation depended on the contributor´s degree of expertise. Several different aggregation functions were then used to reclaim the oracle´s opinion from those of the contributors. The performance of these functions was compared as panel size and hypothesis-set size varied from two to ten. Comparative and individual analyses indicated that for panels assembled under circumstances similar to those of this study, small, simple methods work best