Baseline Ladder: Dummy, Simple, Interpretable, Complex
Unit ID: AMLA-M01-U03 Estimated active time: 30-40 minutes
Classroom explanation
A baseline is not a formality. It is the model's first honest opponent.
In this course, use a baseline ladder. A ladder means you climb only when the lower step is not enough.
Step 1: Dummy baseline
A dummy model asks, "What happens if we barely learn anything?"
For classification, this might predict the most frequent class. For regression, this might predict the mean or median. If a complex model barely beats this, the problem may be weak, the features may be poor, or the metric may be hiding something.
Step 2: Simple useful baseline
This is usually a simple linear, logistic, or small tree model with safe preprocessing. It should be easy to rerun and explain.
Step 3: Interpretable candidate
This model may still be simple, but it should handle the data better. Examples include regularised linear models, logistic regression with class weights, or a pruned tree.
Step 4: More complex candidate
Only now should you try heavier models such as support vector machines, random forests, boosting, stacking, or larger searches. These may help, but they cost more in tuning, runtime, and explanation.
How to use the ladder
Move up only when you can say why the next step is worth trying. Do not climb because the name sounds more advanced.
Practice
Create a four-step baseline ladder for one classification task:
- Dummy:
- Simple useful:
- Interpretable candidate:
- More complex candidate:
For each step, write one sentence explaining why it belongs there.
Takeaway
A complex model must earn its place by beating simpler evidence, not by sounding impressive.