Unit 11.04: The brief someone else could build from

The final artefact of this course is a document, not a system. Its test is whether a competent stranger could build from it without asking you a question.

Everything, in one place, with the numbers filled in

Scope, ingestion, metadata, retrieval parameters, answering rules, checks, escalation, evaluation, accepted failures, next steps. Ten sections, each one traceable to a module of this course.

Below is the complete brief for the running example.

{
  "scope": "support policy questions for agents, during live conversations",
  "out_of_scope": [
    "customer accounts",
    "legal advice",
    "unretrieved questions"
  ],
  "ingestion": {
    "chunk": "one policy statement",
    "max_words": 60,
    "excluded": "anything naming an individual"
  },
  "metadata": [
    "id",
    "source_document",
    "updated",
    "visibility",
    "section"
  ],
  "retrieval": {
    "filters_first": true,
    "k": 6,
    "rewrite": true,
    "threshold": 0.35
  },
  "answering": {
    "cite_every_claim": true,
    "refusal": "The available documents do not cover that."
  },
  "checks": [
    "claim supported by citation",
    "conflict between chunks",
    "staleness over 365 days"
  ],
  "escalation": [
    "conflict",
    "top two within 0.02",
    "unsupported claim"
  ],
  "evaluation": {
    "cases": 8,
    "kinds": 7,
    "refusal_share": 0.375,
    "scored": [
      "retrieval",
      "citation",
      "answer"
    ]
  },
  "accepted_failures": [
    "distant paraphrase may miss",
    "staleness not auto-detected",
    "English only"
  ],
  "next_steps": [
    "add multilingual refusal",
    "automate staleness flag",
    "grow eval set to 40"
  ]
}

Test of a finished brief: could someone build this without asking you a
question? If any field would prompt one, that field is not specified yet.

Every parameter carries a value: k is 6, the threshold is 0.35, the eval set has 8 cases across 7 kinds with a 37.5% refusal share. A brief with k: TBD is not a brief - it is an outline, and the decision it defers is the one the builder will make by accident.

next_steps matters as much as the specification. It tells a reader what you know is missing and roughly in what order you would fix it, which prevents a reviewer from spending their time listing gaps you had already identified.

The mistake this prevents

The mistake is writing this as a summary of the course rather than as a buildable specification. The test is mechanical: read each field and ask whether an implementer would need to ask you something. Every field that prompts a question is a field that is not specified yet.

Takeaway

A finished brief has a value in every field, names its own limits, and could be handed to someone else. That is the deliverable this course has been building towards - the system is what follows from it.

Mark complete