Machine Learning Foundations - Code Feedback Rules

Status: Draft standard Applies to: notebooks, practice tasks, worked examples, checkpoints, and downloadable fallback files

Purpose

Learners should receive useful feedback without a server-side grader or account. Feedback must help them inspect their own work, compare against expected evidence, and explain what changed.

Feedback levels

Use these levels in every coding task:

1. Run feedback: Does the cell run without an error?
2. Shape feedback: Do the table, feature matrix, target vector, prediction array, or metric output have the expected size?
3. Reasonableness feedback: Are values in a believable range for the task?
4. Comparison feedback: Is the result compared with a simple baseline or earlier result?
5. Explanation feedback: Can the learner explain the result in one or two sentences?

Do not mark a task as successful only because a number looks high.

Standard notebook feedback pattern

Each practice notebook should include:

• a short goal before the task;
• a starter cell with clear variable names;
• one visible self-check cell using assert or a small printed checklist;
• an expected-output note that gives shape, column names, or approximate range;
• a written reflection prompt; and
• a worked notebook that explains the same task step by step.

Self-check rules

Self-checks should be transparent. Learners should be able to read what is being checked.

Use self-checks for:

• required column names;
• row counts after a split or filter;
• missing-value counts;
• metric ranges;
• prediction array lengths;
• absence of target leakage columns in X;
• reproducible random seed use; and
• whether a baseline is present.

Avoid self-checks that:

• hide the correct answer;
• depend on a single exact model score unless the calculation is deterministic and simple;
• reward overfitting;
• require internet access;
• require learner login; or
• send code, data, or results to a server.

Error-message style

When an assertion fails, the message should tell the learner what to inspect next.

Good:

assert "completed_course" not in X_train.columns, "Remove the target from the feature table before fitting."

Weak:

assert score > 0.8

Feedback for model tasks

Every modelling task should ask for:

• the simple baseline result;
• the trained model result;
• the metric used;
• the reason that metric fits the decision;
• at least one limitation; and
• one thing that should not be concluded from the result.

Feedback for beginner code

If the learner code is correct but hard to read, the feedback should prefer clarity over cleverness. Suggest clearer variable names, smaller cells, and direct pandas or scikit-learn steps before suggesting abstraction.

Completion rule

A coding task is complete when the learner can rerun it, inspect the output, pass the visible checks, and explain the result. Passing a check cell is not enough by itself.