ESTIMATION OF AUC AND ITS SIGNIFICANCE IN THE ASSESSMENT OF CLASSIFICATION MODELS
The performance of a diagnostic test when test results are measured on a binary or ordinal scale can be evaluated using the measures of sensitivity and specificity. In particular, when it is measured on a continuous scale, the assessment of the performance of a diagnostic test is always over the range of possible cut-off points for the predictor variable. This is achieved by the use of a receiver operating characteristic (ROC) curve which is a graph of sensitivity against 1-specificity across all possible decision cut-offs values from a diagnostic test result. This curve evaluates the diagnostic ability of tests to discriminate the true state of subjects especially in classification models. These tasks of assessing the predictive accuracy of classification models is always better achieved using a summary measure of accuracy across all possible ranges of cut-off values called the area under the receiver operating characteristic curve (AUC). In this paper, we propose a simple nonparametric method of calculating AUC from predicted probability of positive response to a condition which involves multiple prediction rules. This method is based on the non-parametric Mann-Whitney U statistic. The estimation methods for AUC and their significance was assessed using some classification models. The proposed method when applied on real data and compared with other existing methods of calculating AUC was shown to be better in assessing classification models. The method offers reliable statistical inferences and circumvents the difficulties of deriving the statistical moments of complex summary statistics seen in the parametric method. The proposed method as a non-parametric estimation is recommended for calculating the AUC.