Why a Clear Answer Can Be Wrong
Work through the explanation, apply it to the example, and complete the quick check before continuing.
Return to Lakemere
At the start of this module, the assistant gave us a date, a teacher's name, and a book about the made-up town of Lakemere.
The answer sounded like a real history note. The details were not supported.
This kind of false or invented content is often called a hallucination. In this course, we will usually use clearer words such as false, invented, or unsupported.
Why this happens
A text model is good at producing language that fits the current context and learned patterns.
It does not automatically have a separate fact checker for every sentence.
Wrong answers can happen for several reasons.
The question asks for a false thing
If a user asks about a person, event, or book that does not exist, the model may still try to complete the pattern of a useful answer.
Important context is missing
The model may not have the document, date, rule, or detail needed for the task.
The information is old
Training data may not include a recent change. The full app may also lack current search.
The context is unclear
An unclear question can lead the model towards the wrong meaning.
The model uses a weak pattern
It may join together names, dates, places, or ideas that often appear in similar writing but do not belong together in this case.
A tool or retrieval step fails
The app may find the wrong document, call the wrong tool, or use a correct tool result badly.
False citations
A model can create a citation that looks real.
It may produce:
- A believable book title.
- A real author's name with a false article.
- A link that does not exist.
- A real source that does not support the claim.
The shape of a citation is not proof that the source is real or useful.
Missing information also matters
An answer can be wrong because it leaves something out.
Return to the policy example:
Returns are accepted within 30 days if the item is unused and proof of purchase is available.
If the answer says only Yes, you are within 30 days, it misses the unused-item rule.
The sentence may be partly true and still be unsafe to use as the full answer.
Confidence is a writing style
A model may use strong language such as:
Certainly.The answer is...There is no doubt...
This tone is not a reliable measure of truth.
The system may also be uncertain without saying so.
A simple checking habit
When an answer matters, mark each claim as one of these:
- Given: stated in the prompt or supplied material.
- Supported: backed by a suitable source or tool result.
- Added: introduced by the model but not yet supported.
- Missing: an important point that should be present.
Module 5 will teach a full checking method. For now, this habit is enough to slow down false confidence.
Quick check
An answer includes a book title and author. What should you conclude?
A. The citation must be real because it is detailed. B. The source must support the claim. C. The citation needs to be checked outside the generated answer. D. The model searched a library.
Check the answer
Answer: C. A model can create a citation-shaped piece of text without a real supporting source.
Remember
- Smooth language and correct facts are different things.
- A model may add false details when the question or context is weak.
- A citation can look real and still be false or unrelated.
- Missing conditions can make an answer unsafe.
- Mark what was given, supported, added, and missing.
Next, we will separate the base model from search, tools, memory, and agent actions.