How One Threshold Changes Precision and Recall

Learning goal: Calculate how changing a model's cutoff changes its review queue, precision and recall.

Before you begin: You need fractions and percentages. No coding is required.

Maya helps run a school photography club. Members upload pictures, and a model gives each picture a score for being blurry. Maya wants the model to flag pictures for a person to check, rather than deleting anything automatically.

The model's scores are already fixed. Her next decision is where to put the cutoff. A threshold turns a score into an action: in this lesson, flag every picture with a score greater than or equal to the threshold. Changing that rule changes the predictions without retraining the model. scikit-learn's threshold guide

Start with twelve pictures

These pictures and scores are invented for the lesson. A person has checked the actual labels. Five pictures are blurry and seven are clear. Treat the scores as ranking signals; we have not established that a score of 0.80 means an accurately estimated 80% probability.

At a threshold of 0.80, Maya flags A, B, C and D. Three really are blurry. One is clear. Two blurry pictures, E and G, are missed.

A confusion matrix counts these outcomes. Here, rows are the actual labels and columns are the model's decisions. “Positive” means flagged as blurry.

Precision asks: of the pictures flagged, what fraction really are blurry? Here it is 3 ÷ (3 + 1) = 75%.

Recall asks: of all the blurry pictures, what fraction did we flag? Here it is 3 ÷ (3 + 2) = 60%. The denominators differ because the questions differ. scikit-learn's metric definitions

Move the cutoff and count again

Maya lowers the threshold to 0.50. Pictures A through H now enter the review queue.

Precision becomes 5 ÷ (5 + 3) = 62.5%. Recall becomes 5 ÷ (5 + 0) = 100%.

Maya catches two additional blurry pictures, but also sends two additional clear pictures for review. The queue doubles from four pictures to eight. Whether that is worthwhile depends on her purpose and available review time. These twelve examples cannot establish what will happen across every future upload.

Notice what did not change: the pictures, their labels and their scores. Only the decision rule changed. A threshold is therefore part of the system's behavior, not a minor display setting.

Avoid an overly tidy rule

Lowering a threshold on these fixed scores cannot remove an already flagged picture. Recall can increase or stay unchanged. Precision often falls as less promising pictures enter the queue, but it does not have to fall at every step. A newly included picture might be a true positive. You will see that in the exercise.

Try it

Set the threshold to 0.70. Which pictures enter the queue? Calculate precision and recall.

Answer: A, B, C, D and E are flagged. The counts are four true positives, one false positive, one false negative and six true negatives. Precision is 4 ÷ 5 = 80%; recall is also 4 ÷ 5 = 80%. Compared with 0.80, both improve because the one extra picture is genuinely blurry.

Before choosing a cutoff, write down which mistakes matter and how many items people can review. Then count the consequences instead of assuming that one default threshold suits every job.

Optional: keep a final check separate

If you plan to use a threshold in a real project, choose it using one set of labeled examples, then check the chosen rule on a different set that you have not used to make that choice. The first set helps you improve the rule; the second gives a more honest final check. If you keep changing the threshold after seeing that final result, it is no longer an untouched check. Technical guides call the first set validation data and the second a test set. scikit-learn’s threshold-tuning precautions