Unit 07.04: The model's own guess becoming a fact

The most dangerous thing in a conversation history is the model's own earlier guess.

Track how each fact was established

An unverified claim entering the history and becoming a premise.

The code shows the exchange and the fact table.

HISTORY = [("user", "I'm on the enterprise plan, right?"),
           ("assistant", "Yes, your account is on the enterprise plan."),
           ("user", "so what is my refund window?")]
for role, text in HISTORY:
    print(f"[{role:9}] {text}")

facts = [{"fact": "account is enterprise", "source": "the assistant's guess",
          "verified": False},
         {"fact": "two charges on 14 June", "source": "the billing record",
          "verified": True}]
print()
for f in facts:
    print(f"carry forward {f['fact']!r}: {f['verified']}  (source: {f['source']})")

# The unverified guess is now in the context and the next turn treats it as
# established -- so the refund answer will be an enterprise answer, given
# confidently, for an individual account. Track how each fact was established.

The model said the account was enterprise. Nothing verified it, it is now in the context, and the refund answer will be an enterprise answer given confidently for an individual account.

Carrying forward only source-verified facts is a one-line filter and it prevents a whole class of compounding error.

The mistake this prevents

The mistake is carrying the full history forward on the grounds that more context is better. More context includes every unverified claim the system has made, and each becomes more entrenched the longer it survives.

Takeaway

Record how each fact was established and carry forward only what a source supports. The model's own guess, left in the history, becomes a premise.

Mark complete