Why Your Scaler Must Learn from Training Data Only

Learning goal: Find a subtle data leak and use a pipeline to keep the same scaling rule at training and prediction time.

Before you begin: Understand training and test sets. The calculation needs only subtraction and division; the optional code needs Python with the scikit-learn package installed. You can follow every calculation without running it.

Ravi is building a small model for a school game. One input is the time, in seconds, that a player took to finish a practice level. He wants to rescale the numbers before training.

He has heard that the test set should be kept separate. “Surely that only means hiding the answers,” he thinks. “Scaling the input numbers cannot be cheating.”

It can still leak information. A scaler learns something about the data, even though it does not learn the target labels. For a normal held-out evaluation, those learned settings must come from the training portion alone. scikit-learn's data-leakage guidance

Watch the ruler change

Our invented training times are 10, 20 and 30 seconds. One held-out test time is 50 seconds. This tiny example explains the calculation; it is not enough data to evaluate a useful model.

We will use min–max scaling with a training range of zero to one:

scaled value = (value − training minimum) ÷ (training maximum − training minimum)

The training minimum is 10 and the maximum is 30, so the denominator is 20. The three training values become:

• 10 seconds: (10 − 10) ÷ 20 = 0.
• 20 seconds: (20 − 10) ÷ 20 = 0.5.
• 30 seconds: (30 − 10) ÷ 20 = 1.

Now keep that rule unchanged for the test value: (50 − 10) ÷ 20 = 2.

Two is allowed. Min–max scaling fits the training values into the chosen range; later values can fall outside it. In scikit-learn, MinMaxScaler does not clip them by default. MinMaxScaler documentation

See what goes wrong with an early peek

Suppose Ravi combines training and test values before fitting the scaler. The maximum becomes 50. The denominator becomes 50 − 10 = 40.

The held-out picture of the world has changed how the training examples are represented. Ravi did not look at a test label, but he still used test information while constructing the model's inputs.

This does not mean every leaked scaling step will improve every score. Some models or datasets may show little change. The problem is that the evaluation no longer follows the intended training-only procedure.

Fitting a separate scaler on the test set is also wrong: then the model receives numbers defined by a different ruler. Learn the rule once from training data and reuse it.

Put the scaler inside the pipeline

A scikit-learn pipeline connects the transformation and model. Here, a transformation means changing the number using the fixed scaling rule. The pipeline keeps that step attached to the prediction step, so the two cannot accidentally use different rulers. This small, runnable example uses invented labels indicating whether the player completed the next level:

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.linear_model import LogisticRegression

X_train = [[10], [20], [30]]
y_train = [0, 0, 1]
X_test = [[50]]

model = make_pipeline(MinMaxScaler(), LogisticRegression())
model.fit(X_train, y_train)
scaler = model.named_steps["minmaxscaler"]
print(scaler.transform(X_test))  # [[2.]]
predictions = model.predict(X_test)

During fit, the pipeline learns the scaler from X_train, transforms those rows, and fits the model. During predict, it applies the stored transformation without fitting it again. Pipeline documentation

The result to check is still the same: the new value 50 becomes two under the training ruler. Do not quietly refit that ruler during prediction.

Try it

With the same training times, a new player takes 5 seconds. What is the scaled value? Should you refit the scaler to make it zero?

Answer: (5 − 10) ÷ 20 = −0.25. Do not refit during prediction. An out-of-range value is possible under the stored rule. Check unusual inputs and distribution changes separately; do not quietly change the model's ruler.

Optional: what changes when you repeat the test?

Cross-validation repeats evaluation using different training and checking groups. Each temporary checking group is called a fold. A scaler must learn only from the training rows for that round. Pass the whole pipeline and unscaled inputs to the cross-validation tool so the scaler is refitted inside each round, rather than scaling all rows in advance. A pipeline still cannot fix a badly chosen split or an input that already reveals the answer. scikit-learn’s leakage-prevention practices