Active learning for image quality assessment by model observer

In medical imaging, it is widely accepted that image quality should be evaluated using a task-based approach in which one evaluates human observer performance for a given diagnostic task. Unfortunately, human observer studies with expert readers are costly and time-demanding. To confront this problem, model observers (MO) have been used as surrogates for human observers. MOs typically can accurately predict human diagnostic performance but some types of MOs require sets of images and human observer scores for tuning (training). Current literature does not provide guidance on how to choose the training data set. Therefore, in this work we present a heuristic active learning approach, using uncertainty sampling, to the problem of selecting good MOs training data sets. The presented results indicate that the proposed data set selection approach, together with a learning model observer based on the relevance vector machine, has excellent performance in predicting human observer performance as measured by the area under the receiver operating curve (AUC).