Unit 08.04: Memory that quietly changes answers

The most dangerous thing in a conversation history is the assistant's own earlier guess.

Self-confirmation, one turn at a time

An unverified answer enters the history and the next turn treats it as established.

The code shows a three-turn exchange where a wrong plan assignment becomes a premise.

HISTORY = [
    ("human", "I'm on the enterprise plan, right?"),
    ("ai", "Yes, your account is on the enterprise plan."),   # wrong, unverified
    ("human", "So what is my refund window?"),
]

print("conversation:")
for role, text in HISTORY:
    print(f"   {role:6} {text}")

print("""
The assistant's own earlier guess is now in the context, and the next turn
treats it as established. The refund answer will be an enterprise answer,
confidently, for an individual account.

Two defences. Mark which facts were verified against a source and which were
generated, and carry only verified ones forward. And never let the assistant's
own output become an input fact without a check -- it is the same
self-confirmation loop, one turn at a time.
""")
for fact, verified in [("account is enterprise", False),
                       ("two charges on 14 June", True)]:
    print(f"  carry forward {fact!r}: {verified}")

The assistant said the account was enterprise. It was not verified, it is now in the context, and the refund answer will be an enterprise answer given confidently for an individual account.

Two defences. Mark which facts were verified against a source and which were generated, and carry only verified ones forward. And never let the assistant's own output become an input fact without a check - it is the same self-confirmation loop the multi-agent courses warn about, arriving one turn at a time.

The mistake this prevents

The mistake is carrying the full history forward on the grounds that more context is better. More context includes every unverified claim the system has made, and each one gets more entrenched the longer it survives.

Takeaway

Track how each fact was established, not only that it was, and carry forward only what was verified against a source. An assistant's own guess in the history becomes a premise.

Mark complete