Activity: Choose or reject a special-setting method

Estimated active time: 45-75 minutes

Scenario

Use the synthetic learner-support dataset. The fictional platform wants to decide which model families deserve a place in an algorithm portfolio.

Task

Take the class imbalance and small-sample settings seriously: re-evaluate your best model under each, and report what changes.

Required sections

1. Problem and decision.
2. Accuracy, precision and recall for the minority class specifically.
3. The score when trained on half the data, and on a quarter.
4. How much the ranking of algorithms moves across those sample sizes.
5. Keep, delay, or reject under these conditions.
6. Limitation note.

Rules

• Never report accuracy alone on an imbalanced target — give minority-class recall.
• Repeat the small-sample runs with at least three seeds. One run is an anecdote.
• If the ranking changes with sample size, the original ranking was not a finding.

Output

Use the relevant learner-resource template from learner-resources/ or write a one-page memo with the same sections.