Unit 10.02: Counting, reading small text, and spatial relations

Three specific tasks are unreliable, and the model answers all three fluently.

Counting, small text, spatial relations

Six tasks sorted by reliability with the reason for each.

The code lists them.

TASKS = [
    ("is there a carton?",              "reliable",   "presence of a large object"),
    ("what colour is it?",              "reliable",   "dominant colour"),
    ("how many cartons?",               "unreliable", "counting past ~4 degrades"),
    ("read this serial number",         "unreliable", "small text; digits transpose"),
    ("is the label left or right of the barcode?", "unreliable",
     "spatial relations are weak"),
    ("is the box damaged?",             "depends",    "needs a definition of damaged"),
]
print(f"{'task':46} {'reliability':12} why")
for task, reliability, why in TASKS:
    print(f"{task:46} {reliability:12} {why}")

weak = sum(1 for _, r, _ in TASKS if r == "unreliable")
print(f"\n{weak} of {len(TASKS)} are known weak points")

# Counting, small-text reading and spatial relations are the three that surprise
# people, because the model answers all of them fluently. Test each one on your
# own images before designing around it.

Counting degrades past about four objects. Small text produces transposed digits. Spatial relations - left of, above, behind - are weak in a way that surprises people, because the answers are grammatical and confident.

'Is the box damaged?' is in a different category: it is unreliable because 'damaged' is undefined, which is a schema problem rather than a capability one.

The mistake this prevents

The mistake is designing a workflow around counting or reading serial numbers because a demo did it correctly once. Test each on fifty of your own images and measure the rate before building on it.

Takeaway

Counting, small-text reading and spatial relations are known weak points answered fluently. Measure them on your own images before designing around them.

Mark complete