Unit 09.03: Audit trails for what an assistant showed

After an incident, the question is never "what did the assistant say." It is "what did it have access to, and what reached the user."

The two fields you cannot reconstruct

retrieved and shown_to_user answer that question and nothing else does. Together they tell you whether a restricted chunk was ever a candidate, and whether it made it into the context window.

Below is one audit record with every field an investigation needs.

{
  "request_id": "r-8841",
  "asked_at": "2026-07-29T10:14:02Z",
  "user_groups": [
    "public",
    "staff"
  ],
  "question": "What is the refund window?",
  "retrieved": [
    {
      "id": "c1@v2",
      "score": 0.91
    },
    {
      "id": "c2@v1",
      "score": 0.44
    }
  ],
  "shown_to_user": [
    "c1@v2"
  ],
  "answer": "Refunds are allowed within 7 days. [c1@v2]",
  "refused": false
}

The two fields that matter most after an incident are `retrieved` and
`shown_to_user`. They answer a question you cannot reconstruct later: was a
restricted chunk ever a candidate, and did it reach the context window?

Log the chunk VERSION, not just the id. "c1" is ambiguous once c1 has been
updated; "c1@v2" tells you exactly what the user was shown.

Note that chunk ids carry versions: c1@v2, not c1. Once c1 has been updated, a bare id is ambiguous - you cannot tell which text the user actually saw, which is the whole point of keeping the record.

user_groups is the other field worth insisting on. It is what lets you verify after the fact that the access filter behaved, rather than trusting that it did. Without it, an access question becomes a code review instead of a query.

The mistake this prevents

The mistake is logging the question and the answer and calling it an audit trail. Those two fields tell you what happened and nothing about why. The retrieval candidates are where an access failure would be visible, and they are gone unless you wrote them down at the time.

Takeaway

Log the retrieved candidates, what reached the context window, the user's groups, and versioned chunk ids. It is the difference between investigating an incident and speculating about one.

Mark complete