Insyt
All charts in the library

Uncertainty and models

ROC curve

Shows sensitivity against false-positive rate as a binary classifier threshold changes.

  • Plot
  • Model evaluation
  • Threshold comparison

What it answers

How does sensitivity trade off against false alarms?

Shape
True-positive versus false-positive curve
What it shows
Discrimination and threshold trade-offs.

Example

The question

How well does our churn model separate leavers from stayers?

Churn model: true positive rate against false positive rate (ROC curve)

Churn model: true positive rate against false positive rate (ROC curve)The curve shows true against false positive rate as the cut-off moves; the area under it is 0.86 (0.5 would be random). At threshold 0.6 the model catches 55% of leavers with 10% false alarms.True positive rate0%25%50%75%100%0%25%50%75%100%False positive rateThreshold 0.6AUC 0.86Random guess
Churn model: true positive rate against false positive rate (ROC curve)The curve shows true against false positive rate as the cut-off moves; the area under it is 0.86 (0.5 would be random). At threshold 0.6 the model catches 55% of leavers with 10% false alarms.True positive rate0%25%50%75%100%0%25%50%75%100%False positive rateThreshold 0.6AUC 0.86Random guess
The curve shows true against false positive rate as the cut-off moves; the area under it is 0.86 (0.5 would be random). At threshold 0.6 the model catches 55% of leavers with 10% false alarms.

Illustrative data. It belongs to no company.

When to use it

Use it when

Evaluating binary ranking scores on suitable held-out data.

Avoid it when

Only one class present or treating ROC as calibration.

How to read it

Read false-positive rate on X and true-positive rate on Y across thresholds.

Note

Both classes and continuous scores are needed unless rates are precomputed.

The data you need

Layout
Long table
Fields
CaseID: text; Actual: 0/1; Score: number
One row is
One case with its true binary outcome and prediction score. What is the grain?
Rules
Define the positive class and score direction. Use held-out predictions for evaluation. Calibration needs probabilities in [0,1].
Optional
Weight. ROC/PR alternatives: precomputed Threshold,TP,FP,TN,FN with complete threshold coverage.
Preparation
Use held-out binary outcomes and scores; sweep thresholds to compute true/false-positive rates.
Units
TPR=TP/(TP+FN); FPR=FP/(FP+TN). Both outcome classes need nonzero support.
Example of the input layout: Long tableA long table: one row per observation, few columns. New data adds rows.CaseIDActualScoreAABBC
A long table: one row per observation, few columns. New data adds rows.

Worked example

Scores positives 0.9,0.65; negatives 0.55,0.2. Atthreshold 0.6, TPR=1 and FPR=0 in this toy sample; do not generalise fromfour cases.

CaseIDActualScore
A10.90
B00.20
C10.65
D00.55
All charts in the library