Unit 09.02: Data exfiltration paths

Only one of the ways data leaves is the one people think of.

Six paths, five of them around the assistant

Restricted chunks in answers, prompts in error messages, over-permissive trace stores, cross-user cache hits, model-composed URLs, and unredacted logs.

The code lists each with its control.

PATHS = [
    ("answer quotes a restricted chunk",
     "retrieval-time ACL filter, applied before ranking"),
    ("error message includes the system prompt",
     "generic error text; details to logs only"),
    ("trace store readable by more people than the corpus",
     "same access control as the source data"),
    ("cache serves one user's answer to another",
     "user groups in the cache key"),
    ("a tool renders a URL the model composed",
     "no model-composed URLs; allowlist destinations"),
    ("logs capture the full retrieved context",
     "redact on write, retention limit"),
]
print(f"{'path out':46} control")
for path, control in PATHS:
    print(f"{path:46} {control}")

print(f"""
{len(PATHS)} paths, and only the first is the one people think of. The other five
are all systems built around the assistant rather than the assistant itself.

The URL one is worth dwelling on: a model asked to include a link can compose
one carrying retrieved text in its query string, so rendering it sends that
text to whoever owns the domain.
""")

The URL path is the one worth dwelling on. A model asked to include a link can compose one carrying retrieved text in its query string, and rendering that link sends the text to whoever owns the domain - with no tool call, no obvious action, and nothing in the answer that looks wrong.

Five of the six are systems built around the assistant rather than the assistant itself: the trace store, the cache, the logs, the error handler. Each was built by someone thinking about a different problem.

The mistake this prevents

The mistake is auditing the assistant and not its surroundings. The retrieval filter can be perfect while the cache in front of it, the trace store behind it, and the error handler beside it each expose the same data by a different route.

Takeaway

Audit every path data can leave by, not just the answer. Caches, traces, logs, error messages and model-composed URLs are all exfiltration paths built by people solving other problems.

Mark complete