Uncertainty and models

Confusion-matrix heatmap

Shows counts or normalised rates for true and predicted class combinations.

Table
Model evaluation
Error comparison

What it answers

Which classes are confused with which?

Shape: Actual-by-predicted class grid
What it shows: Error types and class imbalance.

Example

The question

Where does our order-category model get confused?

Actual against predicted category, 1,000 test orders (confusion matrix)

Rows are the actual category, columns the predicted one; the diagonal is correct. 833 of 1,000 are right. The biggest confusion is between garden and tools (35 and 40 orders mixed up).

Illustrative data. It belongs to no company.

When to use it

Use it when

Auditing classification errors.

Avoid it when

Unlabelled orientation or normalisation that hides small class sizes.

How to read it

Read actual/predicted axis labels and cell counts before any percentages.

Note

Label axis orientation and normalisation.

The data you need

Layout: Long table
Fields: CaseID: text; ActualClass: text; PredictedClass: text
One row is: One case with its actual and assigned class. What is the grain?
Rules: Use a declared class list and handle missing labels explicitly. Preserve counts before normalisation.
Optional: Preaggregated alternative: ActualClass,PredictedClass,Count.
Preparation: Cross-tabulate actual versus predicted classes; calculate any normalisation only after preserving counts.
Units: Counts ornormalised proportions; label whether each row, column or the full matrix provides the denominator.

A long table: one row per observation, few columns. New data adds rows.

Worked example

Cases(actual, predicted):(Yes, Yes),(No, Yes),(No, No). CountsTP1, FP1, TN1, FN0; positive precision is 1/2.

CaseID	ActualClass	PredictedClass
A	Yes	Yes
B	No	Yes
C	No	No

All charts in the library