Early Leakage Signals

Unit ID: ML-M02-U06 Estimated active time: 25-35 minutes

Leakage can start before modelling

You do not need to train a model to find leakage risk.

Many leakage problems are visible in the column names and timing.

For a day-3 prediction, ask:

Was this value known by the end of day 3?

If not, it cannot be a feature.

Future-information columns

In this dataset, these columns are future information for a day-3 prediction:

• completed_module1_by_day10
• completion_recorded_day
• final_quiz_score

The target is allowed to be future information because it is what we are trying to predict. But it must not be used as an input feature.

final_quiz_score is especially dangerous. It may strongly relate to completion, but it is only known after the learner completes or attempts the final quiz.

Post-action columns

mentor_message_sent_day4 is also risky.

The action happens after the prediction time. If we use that column as a feature, we mix the model decision, human action, and outcome together.

That makes evaluation unclear.

Practice

Make three lists:

1. Valid day-3 feature candidates.
2. Target column.
3. Excluded future or post-action columns.

Then explain why each excluded column is unsafe as a feature.

Takeaway

Leakage often hides in timing. If a column would not be known at prediction time, keep it out of the feature set.