Statistical Pattern Recognition by Andrew Webb

Author:Andrew Webb [Webb, Andrew]
Language: eng
Format: epub
Published: 2011-02-19T05:00:00+00:00

260

Performance

8.2.3

ROC curves for two-class rules

Introduction

The receiver operating characteristic (ROC) curve was introduced in Chapter 1, in the context of the Neyman–Pearson decision rule, as a means of characterising the performance of a two-class discrimination rule and provides a good means of visualising a classifier’s performance in order to select a suitable decision threshold. The ROC curve is a plot of the true positive rate on the vertical axis against the false positive rate on the horizontal axis. In the terminology of signal detection theory, it is a plot of the probability of detection against the probability of false alarm, as the detection threshold is varied. Epidemiology has its own terminology: the ROC curve plots the sensitivity against 1

Se, where Se is the specificity.

In practice, the optimal ROC curve (the ROC curve obtained from the true class-conditional densities, p.xj! i /) is unknown, like error rate. It must be estimated using a trained classifier and an independent test set of patterns with known classes, although, in common with error rate estimation, a training set reuse method such as cross-validation or bootstrap methods may be used. Different classifiers will produce different ROC curves characterising performance of the classifiers.

Often however, we may want a single number as a performance indicator of a classifier, rather than a curve, so that we can compare the performance of competing classifier schemes.

In Chapter 1 it was shown that the minimum risk decision rule is defined on the basis of the likelihood ratio (see equation (1.15)); Assuming that there is no loss with correct classification, x is assigned to class !1 if

p.xj!1/

½

> 21 p.!2/;

(8.7)

p.xj!2/

½12 p.!1/

where ½ ji is the cost of assigning a pattern x to ! i when x 2 ! j , or alternatively ½21

p.!1jx/ >

(8.8)

½12 C ½21

and thus corresponds to a single point on the ROC curve determined by the relative costs and prior probabilities. The loss is given by (equation (1.11))

L D ½21 p.!2/ž2 C ½12 p.!1/ž1

(8.9)

where p.! i / are the class priors and ž i is the probability of misclassifying a class ! i object. The ROC curve plots 1

ž1 against ž2.

In the ROC curve plane (that is, the (1

ž1, ž2) plane), lines of constant loss (termed

iso-performance lines by Provost and Fawcett, 2001) are straight lines at gradients of ½21 p.!2/=½12 p.!1/ (see Figure 8.4), with loss increasing from top left to bottom right in the figure. Legitimate values for the loss are those for which the loss contours intercept the ROC curve (that is, a possible threshold on the likelihood ratio exists). The solution with minimum loss is that for which the loss contour is tangential with the ROC curve–

the point where the ROC curve has gradient ½21 p.!2/=½12 p.!1/. There are no other loss contours that intercept the ROC curve with lower loss.

For different values of the relative costs and priors, the loss contours are at different gradients, in general, and the minimum loss occurs at a different point on the ROC curve.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.