Answers and explanations · Follow one complete prediction investigation

Lesson 11 · Worked integration

Exercise 11.1

At threshold 0.40, jobs three through six are alerted. TP = 2, FP = 2, FN = 1 and TN = 1. Cost is 2 + 4 times 1 = 6 units; alert fraction is 4/6, approximately 66.7%, which fails the 40% cap.

At threshold 0.80, jobs five and six are alerted, including the probability equal to the threshold. TP = 1, FP = 1, FN = 2 and TN = 2. Cost is 1 + 4 times 2 = 9 units; alert fraction is 2/6, approximately 33.3%, which passes. Of these two candidates, only 0.80 is eligible. This does not prove that 0.80 is optimal over the full declared grid; that would require checking all candidates and ties.

Exercise 11.2

For the supplied reference run, regression selects Ridge by lowest mean MAE across the four training-only temporal folds. Logistic regression selects C = 1 with no class weighting by training-CV negative log loss. Sigmoid calibration uses the separate 144-row calibration block. The threshold is 0.20, selected on the 144 policy rows by cost and capacity with the declared tie rules.

The recorded policy counts are TN = 84, FP = 20, FN = 3 and TP = 37; they sum to 144. Positives are 3 + 37 = 40, alerts are 20 + 37 = 57, and alert fraction is 57/144, approximately 39.583%. Cost is 20 + 4 times 3 = 32 units. Confirm these against outputs/reference_results.json in your run. A changed input or implementation must have its own results rather than copying these values.

Exercise 11.3

A good freeze note identifies the exact source revision, hashes, data partitions, target definitions, feature schema, pipeline(s), selected parameters, calibration block/method, threshold rule/value, seeds, environment and report plan. It states that no further choice will use final labels before the report. If final capacity fails, report the fraction and recommend withholding operational use pending a revised study with fresh evaluation evidence. If code changes after freeze, log the reason, which stages were affected and whether test outcomes were already seen. Reference results describe reference code; the learner must provide independently generated evidence from the learner implementation.

Exit check 11.A

Base parameters and selected settings use training and its inner folds. The probability mapping uses the calibration block while the base pipeline remains fixed. The threshold uses the later policy block after its separate embargo. The final test evaluates the frozen composite procedure.

Exit check 11.B

Capacity is measured on a particular sample and probability distribution. Future inputs, outcome relationships and scores can differ, and finite-sample variability also changes counts. A fixed threshold is not a hard queue-length control.