Receiver Operating Characteristic Analysis - A Primer

Eng, J., 2005. Receiver Operating Characteristic Analysis: A Primer1. Academic Radiology, 12(7), pp.909—916.

Introduction

This article summarizes the use of the receiver operating characteristic (ROC) for binary classification tasks. The ROC visualizes the trade-off between sensitivity (recall for true positives) and specificity (recall for true negatives).

The area under the ROC curve (abbreviated AUC) is commonly used as an indicator for the classifier's performance.

Construction of the ROC curve

1. obtain classifications and the corresponding confidence (e.g. definitely negative, probable negative, possible negative, ...)
2. compute the sensitivity and specificity at different cutoff levels (e.g. only consider definite negative classifications as negative).
3. Visualize the trade-off between sensitivity (true positives) and 1-specificity (false positives).