Everyday AI · Module 7

Test a normal case and a changed case

Suggested time: 6 minutes.

A procedure that works once may depend on an unstated assumption. A changed-input test checks whether the process responds to the new evidence rather than copying the first result.

For our second pack, the invitation-review deadline changes and the venue-confirmation owner is missing. Your procedure should keep the new deadline and expose the missing owner. It should not carry over the earlier values.

Worked comparison

First input:

“Invitation review: Leela, due 14 November 2026.”

Changed input:

“Invitation review: Leela, due 18 November 2026.”

Bad second output:

“Leela will review the invitation by 14 November 2026.”

The instruction may have been reused correctly, but the factual output is stale. Check the second result against the second pack, not against your memory of the first run.

Better second output:

“Review invitation — Leela — due 18 November 2026.”

Keep the test understandable

Use a fresh chat for the changed run if possible, and include the complete saved brief with only the new source pack. This reduces reliance on the previous conversational exchange; it does not guarantee correctness. If you reuse a chat, make the current pack boundary and source-only rule explicit, then check carefully for old facts.

When you change the procedure itself, record the change. A test of new input and a test of a new procedure answer different questions. For the required two runs, first keep the procedure stable so you can inspect how it handles the changed evidence.