Skip to content
Aabha AI Academy

All courses · Course overview · Lesson sequence

Skip to lesson text

Answers and explanations · Explain a lexical retrieval baseline

A. Query red normalizes to (1,0). Against A, the score is approximately 0.579739; against B it is 1. B has no other terms, so its direction exactly matches the query. Cosine reduces raw length effects rather than rewarding every extra word.

B. The scores are identical because laptop is not in this index's vocabulary and is omitted from the query vector. A runnable check:

from retrieval import Document, Index
def doc(key, text):
    return Document(key, key, text, "synthetic://" + key, "1", "current", "public")
index = Index([doc("a", "red blue"), doc("b", "red")])
for question in ("red", "red laptop"):
    print(question, [(hit.document.id, hit.score) for hit in index.search(question)])

Both rank B before A with equal corresponding scores. This is a retrieval limitation; it must not be translated into a supported laptop claim.

C. The compound question retrieves only returns-v1 at k=1, giving 1/2. At k=2 it also retrieves hours-v2, giving 1. Six other answerable questions have recall 1, and the paraphrase has recall 0. Thus the first mean is (6+0.5+0)/8=0.8125; the second is (6+1+0)/8=0.875. All seven successful questions already have their first relevant hit at rank 1, so mean reciprocal rank stays 7/8=0.875. The unsupported cases remain a separate abstention/evidence-return count.

D. A tiny document with title Wireless and text Ask staff for access. should not match query wireless in the current text-only index. If titles become indexed, a new expected match could verify that behavior. The current contract explicitly says title metadata is not scored, so update documentation and compare development errors as well as that one test. A test that a title term becomes searchable establishes behavior, not improved answer support or global quality.

Return to this lesson