Unit 01.04: Writing down what the assistant must never guess

The single highest-value artefact in a RAG project is a short document saying what the assistant answers, what it refuses, and what it hands to a human. It takes an afternoon and it prevents the failures that end projects.

A boundary document is a test specification

Every question falls into one of three buckets: answer from documents, refuse, or escalate. Most teams design the first bucket carefully and let the other two emerge by accident, which means they emerge as "the model says something."

Below is a boundary document with all three buckets filled in, plus the two rules that bind them together: cite everything, and say so when nothing matches.

{
  "answers_from": [
    "published support policies",
    "product documentation"
  ],
  "never_answers": [
    "anything about an individual customer's account",
    "legal advice",
    "questions whose answer is not in the retrieved documents"
  ],
  "must_cite": true,
  "on_no_match": "state that the documents do not cover it, and stop",
  "escalate_when": [
    "two documents disagree",
    "the question implies a complaint"
  ]
}

Read never_answers and escalate_when as a list of test cases rather than as policy prose. "Two documents disagree" is something you can construct in an eval set. "Questions whose answer is not in the retrieved documents" is the refusal case Module 6 tests directly.

on_no_match is the field people skip, and it is the one that decides whether the system has a refusal path at all. Fixing its exact wording now means you can assert on that exact string later.

The mistake this prevents

The mistake is writing this after launch, in response to an incident. By then the boundaries are set by whatever the system happened to do, and you are arguing about a specific bad answer rather than about policy. Write it while nothing is on fire, and enforce as much of it as you can in routing rather than in prompt text.

Takeaway

Scope is defined by what a system refuses, not by what it attempts. Write the boundary document first and treat every line in it as a test that has to pass before launch.

Mark complete