Module 03 Activity
Scenario
The passages exist. Now you need to understand how retrieval will behave on them before anything is built on top.
What you produce
A retrieval expectations note: what your corpus makes easy, what it makes hard, and which of your fifteen questions you expect to fail.
Steps
- For each of the fifteen questions, write down which passage you believe should be returned. That list is your answer key.
- For five of them, write the words the user used and the words the document uses. Mark the pairs that share almost no vocabulary.
- Identify any two passages in your corpus that are about the same subject and say different things. Note what a search for meaning would do with them.
- Decide how many passages should be put in front of the model, and justify the number from what your answer key shows rather than from a default.
- Name one exact condition your questions rely on - a date, a section, an audience - that a search for meaning cannot express, and specify it as a filter instead.
- Predict which of the fifteen will fail and why. Keep the prediction; you will check it in Module 8.
Evidence to hand in
- The answer key: question to expected passage.
- Five vocabulary comparisons with the mismatched pairs marked.
- Any contradictory pair you found.
- Your chosen number of passages, with its justification.
- The exact condition specified as a filter.
- Your written prediction of which questions will fail.
Review checklist
- The answer key was written before any retrieval was tried.
- At least one vocabulary mismatch is identified.
- The number of passages is justified from the answer key, not a default.
- The exact condition is specified as a filter, not as query wording.
- The prediction is specific enough to be wrong.
