Skip to course content
Free generative AI app course

Generative AI Application Development with Python

Unit 07.00: What to carry between turns

"It forgot" is almost always a fact that was never extracted into a field.

Fields, not a transcript

The same three-turn exchange as raw text and as extracted facts.

The code compares them.

TURNS = [("user", "why was I charged twice?"),
         ("assistant", "I can see two charges on 14 June, 24.50 each."),
         ("user", "refund the second one")]

transcript = "\n".join(f"{r}: {t}" for r, t in TURNS)
facts = {"account": "ACC-1187",
         "established": ["two charges on 2026-06-14", "24.50 each"],
         "request": "refund the second charge"}

print(f"transcript: {len(transcript)} chars, grows every turn")
print(f"facts     : {len(str(facts))} chars, bounded")
print()
for k, v in facts.items():
    print(f"  {k:12} {v}")

# "It forgot" is nearly always a fact that was never extracted into a field.
# The transcript still contains it; it is simply buried, and it is re-sent in
# full on every turn.

The extracted version is smaller and, more usefully, addressable. facts["account"] is a lookup; finding the account number in a transcript is a search that gets harder every turn.

It also fixes the cost curve. A transcript grows every turn and is re-sent every turn, so a long conversation costs quadratically while a fixed set of fields does not grow at all.

The mistake this prevents

The mistake is reaching for a memory feature. Memory systems store and retrieve what you give them, so a transcript stored verbatim is a transcript retrieved verbatim - the same burial problem with a retrieval step in front of it.

Takeaway

Extract established facts into named fields as they arise. Most reported memory failures are facts never extracted, and the transcript is a quadratic cost as well.