Unit 01.00: What a model knows, and what it only sounds like it knows

Ask a language model when your company's refund window closes and it will tell you. It will tell you in the same steady tone whether it read your policy, read someone else's, or produced a sentence that fits the shape of refund policies in general. The tone is identical in all three cases, and that is the problem this course starts from.

Confidence is not a signal

A model trained on text learns which words tend to follow which other words. That is enough to produce fluent, well-formed, plausible sentences about refunds. It is not enough to produce *your* refund policy, because your refund policy was a business decision, not a linguistic pattern.

The example below below puts the two side by side. One answer comes from nothing; the other comes from a two-line corpus. Read the outputs before reading the explanation.

THE DOCUMENTS SAY
  policy-1   Refunds are allowed within 7 days of purchase.
  policy-2   Enterprise plans use custom contracts negotiated per account.

QUESTION  How long do I have to request a refund?

UNGROUNDED ANSWER
  "Refunds are allowed within 30 days."
  source: none

GROUNDED ANSWER
  "Refunds are allowed within 7 days of purchase."
  source: policy-1

The ungrounded answer says 30 days. The documents say 7. Both are fluent and
confident, and only one names a source a reader can open.

The ungrounded answer says 30 days. The corpus says 7. Both outputs are grammatical, both are confident, and only one carries a source field that a reader can follow. That source field is the entire mechanical difference between the two, and building it properly is what the next ten modules are about.

Notice also what the grounded assistant does when nothing matches: it declines rather than guessing. That branch exists because someone wrote it. Nothing in a language model provides it for free.

The mistake this prevents

The mistake is reading fluency as evidence. In every other setting where you read English, fluency correlates with knowledge - a person who explains your refund policy smoothly has probably read it. Language models break that correlation completely, and your instincts do not update on their own. Treat every ungrounded answer as an unsourced claim, however well it reads.

Takeaway

A model's confidence is generated the same way its content is: as text. The only thing that separates a real answer from a plausible one is a source you can open, and that source has to be supplied by the system around the model.

Mark complete