Activity: Choose a Threshold and Explain Errors

Activity ID: ML-M06-A01 Estimated active time: 45-75 minutes

Goal

Produce the main Module 06 evidence: Metrics report with threshold choice, confusion matrix explanation, and error-analysis notes.

Inputs

• data/module_06_metrics_thresholds_v1.csv
• data/data-dictionary.md
• notebooks/module-06-worked.ipynb
• notebooks/module-06-practice.ipynb

Instructions

1. Compute the confusion matrix at the default 0.5 threshold.
2. Compute precision and recall at three thresholds, and tabulate them.
3. State which error type costs more in this scenario, and why.
4. Choose a threshold from that cost asymmetry and record your reasoning.
5. Take the ten highest-confidence mistakes and describe what they have in common.
6. Write two limitations, one about the metric and one about the data.

Quality checklist

• [ ] The confusion matrix is reported, not just accuracy.
• [ ] Precision and recall are shown at more than one threshold.
• [ ] The chosen threshold has a stated reason tied to error cost.
• [ ] The most confident mistakes were inspected, not just counted.
• [ ] The report does not treat one metric as proof.

Submission format

Submit the notebook output or worksheet table plus a 4-6 sentence explanation.

Safety note

Use only supplied fictional or synthetic data. Do not add or upload real learner data.