Unit 10.01: What it cannot see, and will answer anyway

A vision-language model will answer questions about things that are not in the image.

Six unobservable properties, all answered fluently

What is behind an object, its weight, what happened before, unreadable text, colour under a cast, identity across photographs.

The code lists the model's behaviour for each.

CANNOT_SEE = [
    ("what is behind the carton",      "will describe it plausibly"),
    ("the weight of the object",       "will estimate confidently"),
    ("what happened before the photo", "will construct a narrative"),
    ("text too small to resolve",      "will guess characters"),
    ("the exact colour under a colour cast", "will name a colour"),
    ("whether this is the same object as another photo", "will assert either way"),
]
print(f"{'not visible in the image':42} the model's behaviour")
for thing, behaviour in CANNOT_SEE:
    print(f"{thing:42} {behaviour}")

print("""
None of these produces a refusal by default. The model completes the pattern,
and a plausible completion about an unobservable property is indistinguishable
from an observation.

Every question you ask should be answerable from pixels. When it is not, the
prompt has to supply an explicit way to say so.
""")

None of these produces a refusal by default. The model completes the pattern, and a plausible completion about an unobservable property is indistinguishable in form from an observation.

The rule that follows is narrow and useful: every question you ask should be answerable from pixels. Where it is not, the prompt must supply an explicit way to say so.

The mistake this prevents

The mistake is reading confidence as evidence of visibility. The model is as fluent about the weight of an object as about its colour, and only one of those is in the image.

Takeaway

Ask only what pixels can answer. For anything else the model will produce a confident completion, and its form is identical to an observation.

Mark complete