Controlling true positive rate in ROC analysis

ROC analysis is a widely used method for evaluating the performance of classifiers. In analysis involving scarce data sets leave-one-out resampling techniques might be appropriate. This introduces a problem in terms of computing average ROC curves necessary to determine variance in the true positive and negative rates. A method to determine decision regions for a specified true positive rate is presented. The method is based on estimating the class specific probability density functions for the two classes. The functions are discretised. Dividing these yields a function where values above or below a specific threshold value corresponds to deciding class one or two respectively. It is shown how a gradual lowering of the threshold value corresponds to an increase in the true positive rate, and how a true positive rate can be specified and the corresponding threshold determined. An example with simulated data is used to demonstrate the method.