Unit 05.04: The prompt that keeps the model inside the evidence

A grounding prompt is worth writing carefully. It is also worth being precise about what it is: a request that reduces certain failures, not a mechanism that prevents them.

Each instruction maps to a specific failure

A good grounding prompt is not a paragraph of encouragement. It is a short list, and every line on it should trace to a failure you can name.

Below is such a prompt and then maps each rule to the failure it addresses.

Answer only from the CONTEXT below.
- Cite the chunk id for every claim.
- If the context does not contain the answer, say exactly:
  "The available documents do not cover that."
- Do not use knowledge from outside the context, even if you are confident.
- If two chunks disagree, say so and cite both.

CONTEXT:
[c1] Refunds are allowed within 7 days of purchase.
  answer only from context     -> prevents answering from model weights
  cite every claim             -> prevents unverifiable assertions
  exact refusal wording        -> prevents a fluent guess when nothing was retrieved
  say when chunks disagree     -> prevents silently picking one side of a contradiction

"Cite the chunk id for every claim" targets unverifiable assertions. The exact refusal wording targets fluent guessing when retrieval came back empty - and it is *exact* so that an eval set can assert on the string. "Say when chunks disagree" targets the silent contradiction from Module 3.

The instruction to ignore outside knowledge "even if you are confident" is the one people leave out, and it addresses the case where the model has genuine general knowledge that contradicts your documents. Your documents are the authority on your policies, whatever the model has read.

The mistake this prevents

The mistake is treating the prompt as the grounding mechanism and skipping the checks. Prompts reduce the rate of these failures; they do not bound it. The only way to know the rate is to measure it, which is what the eval set in Module 8 is for.

Takeaway

Write the grounding prompt as a short list where every line names a failure. Then measure how often each failure still occurs - the prompt moves the number, and the eval set is what tells you by how much.

Mark complete