Select context without losing evidence

Our retriever finds both the opening-hours passage and the return-box passage. The team celebrates complete evidence. Then a small output budget removes both passages from the packet. The ranking was correct, but the next stage receives no useful information.

This is a context-selection failure. Context here means the evidence supplied to an answer stage. It is not every source in the collection, nor necessarily every retrieved candidate. We need to account for what survives the transition from ranking to the bounded packet.

Follow one question through three budgets

Run:

python3 -B methods.py context

The question asks when the help desk opens and where books can be returned when it is closed. The two needed passages are hours-v2 and returns-v1. Retrieval ranks returns first and hours second. With both in the candidate list, labeled retrieval recall@2 is 1.

Now read the three budget results. At 80 quote characters, neither complete passage fits and packet coverage is 0. At 120 characters, returns-v1 fits but leaves insufficient room for the hours passage, so coverage is 1/2. At 400 characters, both fit and coverage is 1. The question and candidate ranking have not changed. Only the evidence passed onward differs.

The denominator comes from labels authored for evaluation. The selector does not receive the relevant set and cannot use it as an oracle. It walks ranked candidates and copies entire passages that fit. Evaluating coverage after selection tells us what this simple strategy lost.

Keep units explicit

The limit is max_quote_chars, the sum of Python string lengths of quote text. Source IDs, titles, JSON formatting, instructions and the question are excluded. This is not a total-message limit, a byte limit or a model-token limit. The name makes that small contract explicit.

A later real-model adapter would need to assemble the complete input in its actual format and count it with the actual tokenizer. It would reserve capacity for the requested output and any interface overhead. We do not supply such an adapter or pretend a fixed characters-per-token ratio can replace one. The current lab can teach budget reasoning while making a narrower measurable promise.

Skipping a large passage rather than truncating it is also a declared policy. If ranked passages have lengths 150, 30 and 40 with a budget of 80, the first is skipped and the latter two fit. The packet uses 70 quote characters. A lower-ranked result may therefore survive while a higher-ranked result is omitted. That choice is visible in budget_skipped_ids; it is not proof the smaller passages are better evidence.

Remove redundancy without merging identities carelessly

select_context first deduplicates identical document IDs. If one ID appears with different content or provenance, it raises an error instead of silently picking a version. It also removes repeated exact passage text under different IDs, retaining the first ranked copy and recording the other IDs in duplicate_text_ids.

Exact text deduplication is intentionally narrow. Two overlapping chunks with slightly different text remain distinct. Two sources might contain the same wording but have different authority; retaining the first does not establish which authority should win. This policy reduces literal repetition in the synthetic lab. A real collection needs a reviewed source policy before deduplication becomes an authority decision.

Repeated evidence can crowd out a second source. Suppose three high-ranked chunks all repeat the return-box sentence and the fourth contains hours. A cutoff of two may have high apparent similarity while lacking half the compound answer. Counting distinct source text and inspecting multi-source questions helps reveal that failure. No diversity heuristic is universally correct: some questions legitimately need several passages from one source.

Surface version ambiguity instead of guessing freshness

Consider two current records with the same source identifier, one version 1 and another 2, giving different hours. The current/public filter admits both. Our context policy blocks every candidate from that source and records the conflicting version labels. It does not assume the lexically largest version string is the authorized replacement. Version labels may be dates, release names or arbitrary identifiers; even a larger integer does not prove the record was approved.

This deliberately conservative rule catches a specific kind of ambiguity: multiple version labels for one source among the candidates. It does not detect every contradiction between different sources or between passages carrying the same mistaken version. When the rule flags a conflict, the next step is source review in the synthetic experiment. It is not permission to edit production data or relax a live access rule.

Ordering is another hypothesis to measure

Even if all passages fit, a generator may use them differently depending on placement and surrounding material. Lost in the Middle studied position effects on particular language models/tasks. Research on irrelevant context likewise motivates testing distractors. These findings do not establish the behavior of every later model.

Our quote copier has no such neural position effect: it copies selected text in order. A future experiment could keep the same evidence set, move the needed passage among distractors and compare claim-support/completeness review. It should record the actual model, prompt and repeated-run policy. We cannot report such an experiment from this package because no neural model runs.

Make the handoff inspectable

The returned context record separates the packet from reasons for omission: conflicts, duplicate text and budget skips. This is enough to trace the small examples without retaining a second archive of every run. Commands print results and exit. If you record observations for practice, use only the synthetic IDs and the minimum needed results.

Complete the context practice and separate solutions. You should be able to explain how retrieval can succeed while context coverage fails, and why using evaluation labels inside a selector would invalidate the comparison. Module 7 adds an answer stage with an honest, inspectable template mechanism.