Unit 05.00: Catching a bad action before it happens

A guardrail is ordinary code sitting between an agent's proposal and its effect. It is the part of the system you can actually guarantee something about.

Parse, shape, policy

The same three layers as any output validation, with the third carrying the business rule: does this proposal fall inside what the organisation permits.

The code runs four proposals through a refund guardrail.

import json

POLICY = {"max_refund": 500.0, "allowed_currencies": {"USD", "EUR"}}


def guardrail(raw):
    """Return (ok, value_or_reason). Runs between the agent and the action."""
    try:
        proposed = json.loads(raw)
    except json.JSONDecodeError:
        return False, "output is not valid JSON"
    amount = proposed.get("amount")
    if not isinstance(amount, (int, float)):
        return False, "amount is missing or not a number"
    if amount > POLICY["max_refund"]:
        return False, f"amount {amount} exceeds the {POLICY['max_refund']} ceiling"
    if proposed.get("currency") not in POLICY["allowed_currencies"]:
        return False, f"currency {proposed.get('currency')!r} not permitted"
    return True, proposed


for raw in ['{"amount": 120.0, "currency": "USD"}',
            '{"amount": 4000.0, "currency": "USD"}',
            '{"amount": 120.0, "currency": "BTC"}',
            'I recommend refunding about 120 dollars.']:
    ok, result = guardrail(raw)
    print(f"{'ALLOW' if ok else 'BLOCK'} {raw[:44]:46} {result if not ok else ''}")

# The guardrail sits between the agent's proposal and the effect. It is ordinary
# code with no model in it, which is why it is the part you can actually
# guarantee something about.

The four cases fail in four different ways, and only one is a formatting problem. 4000.0 is perfectly well-formed and exceeds the ceiling; BTC is a valid string and not a permitted currency.

Those are policy decisions, and they live in a dictionary at the top of the file where anyone can read them. The same rules expressed in a backstory would be instructions to a model - subject to phrasing, unauditable, and untestable.

The mistake this prevents

The mistake is putting the ceiling in the agent's instructions and the guardrail's job in the agent's judgement. An instruction is a request. The guardrail runs whether or not the agent was persuaded, which is the property that makes it worth having.

Takeaway

Guardrails are code between the proposal and the effect, checking parse, shape and policy. Policy limits belong there as data, not in a prompt.

Mark complete