Fusion of machine intelligence and human intelligence for colonic polyp detection in CT colonography

In this paper, we proposed a novel method to improve colonic polyp detection in computed tomographic colonography. Utilizing the human knowledge workers via the Amazon Mechanical Turk (MTurk) webservice, we distributed polyp detections from a computer-aided detection system (CAD) to anonymous online knowledge workers and asked them to distinguish true and false polyp candidates. We combined decisions from the CAD system (machine intelligence) and the MTurk workers (human intelligence) using alpha-integration. Preliminary experimental results indicated that the combined decisions were superior to either alone, with area under the receiver operating characteristic curve improving by 5.8% and 7.0% compared with CAD and MTurk workers alone, respectively.