Train/Test Split as a Basic Experiment Habit

Unit ID: ML-M03-U01 Estimated active time: 25-35 minutes

Why this matters

A model can look good when it is tested on examples it already saw.

That is not useful. We want to know how the model behaves on unseen examples.

So before fitting the model, we split the data.

The idea

A train/test split creates two groups:

• training rows: used to fit the model;
• test rows: held back for evaluation.

The test rows are a small experiment. They ask:

How does this model perform on rows it did not train on?

This is not perfect proof of future performance, but it is better than testing on the same data used for training.

Predict

Suppose a learner accidentally fits a model using all rows, then reports the score on those same rows.

What risk does this create?

Write one sentence before reading on.

Run or inspect

The usual pattern is:

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=RANDOM_SEED,
)

This keeps 75 percent of rows for training and 25 percent for testing.

The random_state makes the split repeatable. If the notebook is rerun, learners should get the same split.

Change one thing

For classification, if one class is much smaller than the other, a random split may put too few small-class examples in the test set.

That is why later lessons use stratified splits and richer metrics. In this module, the classification dataset has a simple enough balance for a first baseline.

Practice

Answer:

• Which rows are used to fit the model?
• Which rows are used to evaluate the model?
• Why should the target not appear inside X?
• Why do we set random_state?

Check and explain

Complete this sentence:

The test set is held back so that ______.

Takeaway

Split before fitting. Keep test rows unseen until evaluation.