Outcome: Identify unsupported claims and turn an AI answer into a checked, usable response.
Prerequisites: Basic experience with an AI chat tool, access to the original material, and permission to use that material. No coding needed.
A well-written answer can contain an invented deadline, a wrong number, or a source that does not exist. Before you forward an AI-generated recommendation, check the claims that could change what someone does. Confident wording is never enough evidence.
Why a convincing answer can be wrong
Language models learn patterns in text and generate likely continuations. Fluency can therefore appear even when a particular fact is missing. OpenAI's research explains how uncommon, arbitrary facts are especially difficult to infer from patterns, and how evaluation systems that reward guessing can discourage expressions of uncertainty. These are contributing mechanisms, not a complete explanation of every error. OpenAI research
NIST calls confidently presented false or erroneous output “confabulation.” Its guidance also warns that fabricated citations can make an answer appear justified. A detailed explanation and a professional-looking reference can both need checking. NIST Generative AI Profile, section 2.2
Other practical problems include outdated information, incomplete documents, ambiguous instructions, and mistakes in calculations or interpretation. You do not need to diagnose the model's internal cause before checking the answer.
A worked example
The following policy and answer are illustrative, synthetic examples written for this lesson. They are not outputs from a live model test.
Fictional source, “Desk Equipment Policy, version 3, effective 1 September 2026”:
-
P1: Permanent employees may claim up to ₹3,000 per financial year for approved accessories.
-
P2: Claims require a receipt and manager approval and must be submitted within 30 calendar days of purchase.
-
P3: Contractors are outside this policy.
Illustrative answer: “All staff, including contractors, can claim ₹3,000 each month. Submit the receipt within 30 working days. Manager approval is optional under section 4.”
The answer sounds actionable. Every sentence contains a material problem.
Check it in five steps
-
Split the answer into individual claims. Record who qualifies, the allowance, its period, the deadline, required approvals, and the cited section. Checking a paragraph as one unit makes small but consequential changes easy to miss.
-
Find the controlling source. Here, use the current policy and its effective date. An old staff handbook or a colleague's recollection may differ. In real work, open the original document and verify its version. A search snippet alone is insufficient.
-
Match each claim to exact evidence. The review finds:
-
“Including contractors”: contradicted by P3. Correct to permanent employees only.
-
“₹3,000 each month”: contradicted by P1. The limit is per financial year. Monthly wording could imply ₹36,000 over twelve months.
-
“30 working days”: contradicted by P2. Preserve “calendar days” and the purchase-date starting point.
-
“Approval is optional”: contradicted by P2. Receipt and manager approval are both required.
-
“Section 4”: unsupported by the supplied source. Remove it; do not manufacture a replacement reference.
A claim absent from the source is “unsupported,” not automatically “false.” Ask the policy owner if missing information matters.
-
Recalculate anything numerical. Check arithmetic separately with a calculator or spreadsheet. Confirm units, dates, denominators, and time periods. Correct arithmetic cannot rescue an incorrect assumption, such as treating an annual allowance as monthly.
-
Rewrite, then recheck. A usable answer is: “Permanent employees may claim up to ₹3,000 per financial year for approved accessories (P1). Submit a receipt with manager approval within 30 calendar days of purchase (P2). Contractors are excluded (P3).” Compare the revision with the source again before sharing.
Ask for evidence before the first answer
Try this prompt:
“Use only the supplied policy. For each eligibility rule, amount, deadline, and approval requirement, provide the supporting paragraph. Preserve units and time periods. If the policy does not answer something, write ‘not specified.’ Separate any suggested next steps from policy requirements.”
Anthropic recommends allowing uncertainty and grounding document answers in supporting passages. These techniques can reduce errors; they cannot guarantee correctness. Claude guidance on reducing hallucinations
Mistakes and limits
Asking the same model “Are you sure?” can produce another confident answer. Agreement between two models also needs external evidence. Open citations and check whether they support the exact claim. Use approved tools for confidential documents. For consequential legal, medical, or financial decisions, obtain appropriate expert review.
Try it
A fictional note says: “Delivery is expected by Friday, subject to stock confirmation.” An AI writes: “Delivery is guaranteed on Friday.” What needs changing?
Answer: Restore both uncertainty and the stock condition: “Delivery is expected by Friday, subject to stock confirmation.” Do not promise a guarantee the source never made.
Sources: The three primary sources linked above were checked on 5 October 2026.