Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 02.03: Permissions belong on the chunk, not the prompt

"Do not reveal confidential documents" in a system prompt is not access control. It is a note to a component that has already been handed the confidential document.

The order of operations is the control

If a restricted chunk reaches the context window, the restriction has already failed. Whether the model then mentions it is a matter of probability, and probability is not what you present to an auditor.

The alternative is a filter applied before ranking, using a field stored on the chunk. The example below shows both, run against the same query.

CHUNKS AND WHO MAY SEE THEM
  c1   Refunds within 7 days.            visible to: everyone
  c2   Escalate refunds over 500.        visible to: staff

WHO IS ASKING          WHAT THE FILTER LETS THROUGH
a public user          c1
a staff user           c1, c2

THE WRONG APPROACH
  retrieve c1 and c2 for everyone, then instruct the model:
  "do not mention internal documents"

That places a restricted chunk in front of the model and relies on it to look
away. The filter never retrieves it at all.

The prompt-based version puts the restricted chunk into the context and then asks nicely. The filter version never retrieves it. The second is deterministic, testable and explainable; the first is none of those.

The filter also has to run *before* ranking, not after. Filtering the top-k means a restricted chunk can occupy a slot, pushing out a permitted chunk the user should have seen - so the user gets a worse answer and you get a security finding.

The mistake this prevents

The mistake is layering prompt instructions on top of a permissive retriever and calling it defence in depth. Defence in depth means multiple controls; a prompt is not a control, because you cannot state what it guarantees. Add the prompt instruction if you like, but only after the filter exists.

Takeaway

Store visibility on the chunk, intersect it with the user's groups before ranking, and treat any restricted chunk that reaches the context window as an incident. Prompts cannot enforce what retrieval has already handed over.