Run an Evaluation and Improvement Loop

Unit ID: PAW-M12-U03 Estimated active time: 25-30 minutes

Three runs, not one

A single run tells you very little — you cannot tell a prompt problem from a bad draw.

Three runs on three real instances tell you what is consistent, and consistency is what you are building.

Score every run the same way

Use the rubric from your canvas, unchanged, and score each run before reading the next.

Changing the rubric between runs makes the comparison meaningless, which is the commonest way this exercise goes wrong.

The improvement loop

EVALUATION LOOP

Run   Version  Accuracy  Complete  Useful  Honest  Tone  Edits I made
---   -------  --------  --------  ------  ------  ----  ---------------
1     v1
2     v1
3     v1

Consistent weakness across runs:
One change to make (one only):        -> v2

4     v2
5     v2
6     v2

Did the change fix it?     yes / no / partly
Edits still required:
Stop when: no edit is needed more than once in four runs

Change one thing at a time

Changing three lines and seeing improvement tells you nothing about which line did it.

One change per version is slower for two rounds and much faster after that, because you accumulate knowledge instead of a prompt nobody understands.

Know when it is good enough

The stop rule in the sheet is a reasonable default: stop when the same edit is not needed more than once in four runs.

Chasing perfection past that point costs more than the remaining edits do.

Mini practice

Run three instances on v1, score them, and identify the single consistent weakness.

Make one change and run three more.

Record whether it worked in your improvement log.

Mark complete