DocumentCode :
769651
Title :
Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach
Author :
Yousef, W.A. ; Wagner, R.F. ; Loew, M.H.
Author_Institution :
Center for Devices & Radiol. Health, Food & Drug Adm., Rockville, MD
Volume :
28
Issue :
11
fYear :
2006
Firstpage :
1809
Lastpage :
1817
Abstract :
This paper considers binary classification. We assess a classifier in terms of the area under the ROC curve (AUC). We estimate three important parameters, the conditional AUC (conditional on a particular training set) and the mean and variance of this AUC. We derive, as well, a closed form expression of the variance of the estimator of the AUG. This expression exhibits several components of variance that facilitate an understanding for the sources of uncertainty of that estimate. In addition, we estimate this variance, i.e., the variance of the conditional AUC estimator. Our approach is nonparametric and based on general methods from U-statistics; it addresses the case where the data distribution is neither known nor modeled and where there are only two available data sets, the training and testing sets. Finally, we illustrate some simulation results for these estimators
Keywords :
decision theory; pattern classification; binary classification; data distribution; independent data sets; nonparametric approach; Decision theory; Medical diagnosis; Parameter estimation; Probability density function; Random variables; Statistical analysis; Statistical distributions; Testing; Training data; Uncertainty; Classification; ROC analysis.; nonparametric statistics; Algorithms; Artificial Intelligence; Cluster Analysis; Databases, Factual; Image Enhancement; Image Interpretation, Computer-Assisted; Information Storage and Retrieval; Pattern Recognition, Automated; ROC Curve;
fLanguage :
English
Journal_Title :
Pattern Analysis and Machine Intelligence, IEEE Transactions on
Publisher :
ieee
ISSN :
0162-8828
Type :
jour
DOI :
10.1109/TPAMI.2006.218
Filename :
1704836
Link To Document :
بازگشت