Answers and explanations · Start with a prediction-time contract

Lesson 01 · Prediction contract

Exercise 01.1

An acceptable contract says: “One row represents one accepted synthetic depot job. Predict duration in hours at intake using the eight approved intake measurements, including the original promised hours. The outcome becomes available after completion, with label availability recorded explicitly. The prediction supports a fictional human review/planning study. It does not authorize automatic staffing changes or consequential scoring of people. Evaluate on later jobs using the prescribed chronological protocol and report uncertainty and limitations.”

A mid-job model has a different prediction time, available information and likely remaining-duration target. An observed progress update might be legitimate then while being forbidden at intake. It requires a new contract and evaluation reconstruction, not an extra feature silently added to this model.

Exercise 01.2

The training sum is 30 and there are five values, so the mean predictor is 6 hours. The sorted middle value is 3, so the median predictor is 3 hours. For actual durations 2, 4 and 12, the mean predictor's absolute errors are 4, 2 and 6; MAE is 12/3 = 4 hours. The median predictor's errors are 1, 1 and 9; MAE is 11/3, approximately 3.667 hours.

The long training durations pull the mean upward more than the median. The median minimizes training absolute error among constant predictions, while the mean minimizes training squared error. This fact does not guarantee which constant wins on every future sample. Neither constant may be recalculated using these evaluation outcomes.

Exercise 01.3

An always-on-time baseline obtains 90% accuracy if 10% are late, while catching no delays. Accuracy alone therefore hides a complete failure of the intended late-job review function. Report recall to show the fraction of late jobs found, precision to show the fraction of alerts that are late, and alert count/fraction to quantify the review workload. At least one additional measurement is required by the exercise; a strong answer connects more than one to the actual decision. Probability quality is a separate consideration if predicted probabilities are communicated.

Exit check 01.A

Fitting can use historical labels that were available by its fit boundary. Intake prediction must use only approved intake inputs and the already learned procedure; it cannot require the new job's eventual duration, lateness or post-outcome fields. Historical labels also must have been available in the simulated historical fitting situation.

Exit check 01.B

A model was optimized on training data, so training improvement can include memorization or adaptation to noise. Future jobs may differ. An appropriate held-out design measures performance of the learned rule on observations that did not select it; a baseline makes the improvement interpretable.