Skip to course content
Free course

Machine Learning Foundations / Module 3

Module 3 lesson

What a First Model Should Prove

Unit ID: ML-M03-U00 Estimated active time: 20-25 minutes

Why this matters

The first model is not the final product.

The first model is a test of a smaller question:

Is there any useful signal in the available features, compared with a very simple baseline?

That is a humble question, and that is good. A humble first model protects us from pretending we know more than we do.

The idea

In Module 1, you framed the problem. In Module 2, you inspected the data.

Now we build the first supervised models. Supervised means the training data contains examples with known answers. The known answer is called the target.

In this module we use two targets:

  • quiz_score_day10: a number, so this is a regression task.
  • completed_module1_by_day10: a class, so this is a classification task.

Both targets are fictional synthetic outcomes from the same early learning scenario.

Predict

Before training anything, answer this:

What should a first model prove before we spend time improving it?

Choose one:

  • It should prove the course can make perfect predictions.
  • It should prove there is useful signal beyond a simple baseline.
  • It should prove the model can be used with real learners immediately.

The best answer is the middle one.

Run or inspect

A first model should be compared with a naive baseline.

For regression, a naive baseline might always predict the median quiz score from the training data.

For classification, a naive baseline might always predict the most common class from the training data.

If a trained model cannot beat this kind of baseline, the model has not shown useful signal yet.

Change one thing

What changes if the baseline is already strong?

Then beating it slightly may not be enough. We still need to ask whether the improvement is meaningful, stable, and useful for the decision.

Practice

Write one sentence for each task:

  • Regression task: What number are we predicting?
  • Classification task: What class are we predicting?
  • Baseline question: What simple baseline should we compare against?

Check and explain

Complete this sentence:

A first model is useful when it shows ______, but it does not prove ______.

Takeaway

A first model should show whether useful signal exists. It should not be treated as a finished system.