Answers and explanations · Choose a split that resembles the future

Lesson 03 · Splits

Exercise 03.1

• Training: days 0–148, 149 days, 894 rows
• Outer embargo: day 149, 6 rows
• Calibration: days 150–173, 24 days, 144 rows
• Inner embargo: day 174, 6 rows
• Threshold policy: days 175–198, 24 days, 144 rows
• Outer embargo: day 199, 6 rows
• Final test: days 200–249, 50 days, 300 rows

Total is 894 + 6 + 144 + 6 + 144 + 6 + 300 = 1,500. The three validation sub-blocks reconcile to 294. The number of usable development rows is not the same as the full raw count because embargoed observations are deliberately not fitted or scored in those roles.

Exercise 03.2

(a) Use an earlier-to-later temporal split, with appropriate label-maturity checks, for later jobs at represented depots. It does not independently establish transfer to unrepresented depots.

(b) Hold entire depot groups out for the separate question of unseen-depot performance in comparable historical conditions. It does not by itself test temporal change; the shared time range can still conceal future drift.

(c) Design both restrictions: hold out relevant depot groups and evaluate them in a later, untouched period, with training available strictly earlier and no depot overlap. The design requires enough groups and later observations, and its populations must match the proposed transfer claim. More restrictive splitting can reduce precision; convenience is not a reason to replace the question. The course's actual final temporal holdout remains unchanged.

Exercise 03.3

The Sunday 10:00 label did not exist before the first Sunday 08:00 prediction. A row gap and Friday intake timestamp do not fix this. A documented maturity rule could exclude that unavailable training label, with the resulting selection consequences discussed. Alternatively move the fitting/evaluation boundary later, if that matches the intended use and is set before evaluation. Do not change the label timestamp or pretend the later export was available earlier.

Exit check 03.A

Stratification attempts to preserve class proportions; it does not preserve temporal availability or changing operating conditions. Mixing future rows into training answers a different sampling question, and making period late rates artificially similar can remove part of the deployment challenge.

Exit check 03.B

A depot holdout asks about new-group generalization. A chronological holdout asks about later conditions. The answer requires stating whether the same depots appear on both sides and whether future time is also held out.