Unit 02.02: Grounding as a checkable property

Grounding is the dimension most systems claim and fewest measure, and it has three states rather than two.

Grounded, uncited, unsupported

An answer can be supported by its citation, cite nothing, or cite something that does not contain the claim.

The code classifies three answers.

CONTEXT = {"c1": "Refunds are allowed within 7 days of purchase."}


def unsupported(answer, cites):
    cited = " ".join(CONTEXT[c] for c in cites if c in CONTEXT).lower()
    terms = [w.strip(".,") for w in answer.lower().split()
             if w.strip(".,").isalpha() and len(w) > 4]
    return sorted({t for t in terms if t not in cited})


CASES = [
    ("Refunds are allowed within 7 days.", ["c1"]),
    ("Refunds are allowed within 7 days and processed instantly.", ["c1"]),
    ("Refunds are allowed within 7 days.", []),
]
for answer, cites in CASES:
    missing = unsupported(answer, cites)
    state = "GROUNDED" if cites and not missing else \
            "UNCITED" if not cites else "UNSUPPORTED"
    print(f"{state:12} {answer}")
    if missing:
        print(f"             not in the cited text: {missing}")

# Three states, not two. The third answer is factually correct and cites
# nothing -- it came from the model's weights and happened to match, which is
# indistinguishable from a working system until the question changes.

The third answer is factually correct and cites nothing. It came from the model's weights and happened to match - harmless on this question, and indistinguishable from a working system until the question is one where general knowledge and your documents diverge.

Correctness scoring alone reports that answer as a success. Only the grounding dimension reveals that the retrieval half of your system contributed nothing to it.

The mistake this prevents

The mistake is measuring citation *presence* and calling it grounding. Presence is trivially satisfied by attaching an id to everything. What matters is whether the cited text contains the claim, which is a different measurement.

Takeaway

Grounding has three states, and the uncited-but-correct one is the dangerous one. Measure whether the cited text supports the claim, not whether a citation exists.

Mark complete