Voting and stacking
Unit ID: AMLA-M07-U05 Estimated active time: 25-40 minutes
Classroom explanation
This unit belongs to Ensembles: Bagging, Forests, Boosting, Voting, and Stacking. The practical focus is bias, variance, diversity, bagging, forests, boosting, voting, stacking, and leakage-safe tuning.
Start from the workflow you already know: define the problem, protect the split, build a baseline, compare honestly, and state limits. The new algorithm detail in this unit should help you make a better choice, not distract you from that workflow.
Voting combines model predictions directly. Stacking trains another model to combine predictions. Both need careful validation to avoid leakage.
Why this matters
Algorithm names can sound more precise than they really are. A method is useful only when its assumptions, data needs, runtime cost, and explanation limits fit the decision.
In this unit, ask:
- What kind of evidence would make this method worth trying?
- What data shape would make it fragile?
- What simpler baseline must it beat?
- What limitation should appear in the final memo?
Worked example
A portfolio might combine logistic regression, a shallow tree, and a forest, but the combination must be judged against simpler candidates.
Use the synthetic learner-support dataset. Compare the module's candidate idea against the dummy baseline and the transparent rule baseline. The goal is not to crown a universal winner. The goal is to decide whether this method deserves a place in the candidate portfolio.
Common mistake
Do not stack using predictions from the same data used to train base models without proper cross-validation.
A second common mistake is to treat a stronger-sounding algorithm as automatically better. Avoid that by writing the candidate reason before looking at any score.
Practice
Write one reason voting is simpler than stacking.
Add one line to your algorithm comparison report explaining how this unit changes your candidate list. Include one reason to try the method and one reason to delay or reject it.
Takeaway
Voting and stacking is useful only when it improves the decision evidence enough to justify its extra assumptions, tuning, or complexity.