Start with a prediction-time contract

The question comes before the estimator

A depot receives a job and must decide whether to put it on a human review list. At intake, the team knows the distance, queue size, item count, requested priority, zone, depot, weekday and promised hours. It does not yet know the job's duration. Our first task predicts duration_hours; our second estimates the probability that duration will exceed the promise recorded at intake. The decision is a fictional operational review, never a score about a person's worth, employment, credit, health or access to services.

You already know how to inspect a table from Python, preserve grain from SQL and question denominators from Statistics. Machine learning adds a different obligation: demonstrate that a rule learned from earlier observations remains useful on observations it could not have seen. A sophisticated model fitted to information from the future fails that obligation, even if its error is tiny.

A prediction-time contract is a short specification of the situation in which a prediction will exist. Write the population, one-row grain, prediction time, allowed information, target, outcome-observation time, intended action and evaluation plan. Here the population is accepted synthetic depot jobs, the grain is one job, and the prediction time is intake. An ID connects evidence but is not an explanatory measurement. The contract also says who would inspect an alert and what happens if data are invalid. A model is not the whole workflow.

Features, targets and decisions are different objects

We use X to mean the input table and y to mean the target values. A fitted model is a reusable function: new X goes in, predictions come out. During fitting, the model uses paired historical X and y. During prediction, y must be unavailable; otherwise there would be no prediction problem.

duration_hours is a numerical target, making the first task regression. late is a binary target, making the second task classification. Its definition is duration greater than promised_hours, with equality counted as on time. The promise is both a target-definition ingredient and an approved predictor because it is recorded at intake. The classifier can distinguish an identical job with a short promise from one with a generous promise. Its availability must be verified, and the original intake promise must be preserved. If promise-setting practices change, the lateness problem can change even when delivery operations do not.

A probability is not a decision. A probability of 0.70 is a model's estimated risk under its learned data conditions. A review rule might flag probabilities at or above a threshold. The threshold depends on missed delays, unnecessary reviews and capacity; it is not automatically 0.50. We postpone its selection until we have separate data for it.

Worked miniature: build a baseline before adding features

These four training durations are an arithmetic example, not results from the generated course data: 2, 4, 4 and 10 hours. The mean is 20 divided by 4, or 5. The median is 4. A mean baseline predicts 5 for every future job; a median baseline predicts 4. Neither needs distance or queue size. That simplicity is useful because it tells us what a feature-based model must improve on.

Now consider a separate miniature evaluation set with durations 3, 5 and 8. The mean baseline's absolute errors are 2, 0 and 3, giving mean absolute error, or MAE, of 5/3, approximately 1.67 hours. Its squared errors are 4, 0 and 9, giving mean squared error of 13/3. Taking the square root gives RMSE of approximately 2.08 hours. The median baseline has absolute errors 1, 1 and 4, so MAE is 2 hours. It would be a mistake to conclude that the training median was calculated incorrectly: a training-optimal constant need not win on every new sample.

Notice the direction of information. We calculated each constant on training durations, then evaluated it unchanged. Replacing it with the evaluation mean would make the score an estimate of a rule that was allowed to see the answers. In the executable workflow, DummyRegressor provides the same kind of reference. The versioned API documents the mean and median strategies.

For classification, imagine training contains 8 late jobs out of 40. A constant probability baseline can predict 0.20 for every job. A majority-label baseline always predicts on time. Its high accuracy on a mostly on-time sample says little about whether it catches delays. Later lessons compare both probability quality and review decisions against appropriate baselines.

Define success without promising an outcome

Choose the primary metric before seeing candidate scores. MAE is easy to interpret in hours. RMSE reveals whether a few large errors dominate. For classification, ranking, probability quality and alert burden answer different questions. Write one main selection criterion and a small set of checks, rather than selecting whichever metric makes a favored model look good.

A model that beats a baseline by a negligible amount may not justify maintenance. Conversely, a useful average improvement can hide a poor result for a small zone. Evaluation should report the sample size, period and conditions as well as the number. The Statistics course's distinction between practical importance and numerical difference still applies. This course's simulated data establish a learning exercise, not business value at a real depot.

Guided execution and interpretation

Run python examples/01_contract.py from the course root after setup. Before running, write which fields should be predictors and which should be retained only for auditing or label construction. Read the printed contract. Locate baseline results in the generated outputs when the complete workflow has run; actual values belong in outputs/reference_results.json. Do not substitute the miniature numbers above.

Keep a short decision log: question, chosen baseline, metric, reason and date. The log should let someone distinguish an assumption made before analysis from an explanation invented after the results. You will extend it rather than rewrite history at every lesson.

Independent practice

1. 01.1 · Specify the moment. Write a six-sentence contract for duration prediction, including grain, intake time, allowed inputs, outcome time, use and one non-use. Explain why an intake prediction and a mid-job prediction are different products.
2. 01.2 · Calculate without fitting software. Training durations are 1, 3, 3, 9 and 14. Calculate the mean and median constant predictors. On new durations 2, 4 and 12, calculate each MAE. Explain why the two training constants differ.
3. 01.3 · Challenge the objective. A colleague proposes selecting the classifier solely by accuracy because only 10% of jobs are late. Write a response that includes a baseline, a failure that accuracy can hide and one additional measurement needed for the review workflow.

Exit checks

• 01.A: What information may fit use that a real intake-time predict call must not require?
• 01.B: Why does beating a baseline on training data fail to establish usefulness on future jobs?