Answers and explanations · Defend an evidence-grounded capstone

These solutions explain a defensible approach. They are not a passing submission or a source of platform credit. Keep your own observations, including results that differ because you changed the inputs or policy.

Establishing the reference

The correct evidence begins with the actual synthetic source text and its identity. A question about desk hours and closed-hours returns needs both the hours rule and return instructions. One can answer when the desk is open without explaining what happens after it closes; the second source fills that distinct part of the question. A laptop-loan request is unsupported by a passage about book loans even if both mention borrowing and fourteen days. The object of the rule changed.

The reference output provides observed IDs and metrics, not an independent authority for its own relevance labels. A reviewer should challenge a label by citing the source and requested fact. If the label is wrong, record the correction and rerun all compared configurations under the corrected dataset identity. Do not preserve an invalid label merely to keep a historical score unchanged.

Isolating changes

At a fixed budget, changing k investigates how many ranked candidates reach selection. At a fixed k, changing the quote budget investigates how much of that candidate evidence survives. Changing both at once can show a different outcome but cannot attribute the difference to a single stage. A useful comparison table contains the case, expected IDs, retrieved IDs, selected IDs and reason for each omission.

For the lesson's explicit arithmetic example, retrieval recall is 2 / 2 = 1. After selecting the 70-character passage, adding 90 would require 160 characters, so the 100-character budget excludes it. Selected-evidence coverage is 1 / 2 = 0.5. Neither value establishes semantic support. They compare IDs against labels; a later sentence could still contradict a condition, and a label itself can be mistaken.

Do not report the arithmetic lengths as actual file measurements. For the supplied corpus, use the command's real rows or calculate lengths from the exact passages. A broader budget can improve coverage while leaving an unsupported-object query unresolved. A lexical cutoff change cannot create a representation for a genuinely unseen paraphrase.

In the supplied visible reference, cq-compound needs study-room-v1 and hours-v2. At 400 characters, moving from k=1 to k=2 raises that case's packet coverage from 0.5 to 1. At k=2, increasing the budget from 80 to 400 raises it from 0 to 1. Across the four answerable cases, the corresponding packet means are 0.875 and 1 for the larger budget; both means are zero at 80 characters. Retrieval MRR remains 1, which is compatible with missing a second relevant source: reciprocal rank records the first relevant hit, not complete evidence coverage. The two unsupported cases remain outside these means, and one still receives irrelevant retrieval hits. Whether those hits survive into the packet is a separate budget-dependent observation.

Completing and extending the learner path

The supplied runnable orchestration is in capstone_reference.py; the reference selector is select_context in methods.py. Read that selector after attempting learner_select. The runner already loads the synthetic inputs, creates the index, retrieves candidates and compares retrieval metrics with packet coverage. Your hook should deduplicate and detect conflicting source versions as taught, select complete quotations within the budget and report exclusions using the expected fields. It must never turn a retrieval score into probability of correctness.

For your own four-question extension, the most valuable labels are justified labels. One possible fictional rule is: “A room may be requested for at most one hour; staff confirm availability.” The claim “a room is reserved” would go beyond that text. A second passage describing access arrangements can support a multi-source question, but it should not be silently treated as a reservation permission. If an older room policy exists, mark its version/status explicitly and verify which one entered the index. An absent bicycle-storage policy remains absent even if the source mentions a book-return box.

Authoring these examples alongside a solution makes them teaching fixtures. To conduct a later less-biased comparison, have another person write and adjudicate new cases before the candidate is frozen, withhold them during tuning, then inspect all results together. Once results guide another change, retain their history and acknowledge that the set is now development material.

Reviewing the answer

A supported human answer preserves both the rule and its condition: “The supplied passage allows a request of up to one hour, subject to staff confirmation; it does not establish that a particular room is available.” Cite the actual passage you authored. An unsupported answer such as “Your room is booked for an hour” invents both a current availability fact and an action that this project cannot perform.

Valid citation IDs establish that the named material is present. Claim-level review asks the different question: does that material justify this sentence, including its object, quantity, time and conditions? Our manual support labels are supplied judgments. A function that counts them has not independently understood the prose.

An actual LLM study would need actual model calls or an explicitly run local model, a declared version/configuration and recorded returned answers. You would then review those answers under the same claim contract and distinguish retrieval, selection and generation errors. No result from fixed vectors, a count-based toy language model or the one-question authored template substitutes for that study.

A defensible recommendation

Prefer a configuration because of its observed per-case behavior under declared constraints, not because its name sounds more sophisticated. Report retained conditions and multi-source coverage alongside false evidence returns. Explain why the remaining failure prevents a broader claim, and name the next experiment that would test your explanation.

A complete local capstone can honestly conclude that the reference is useful for inspecting synthetic evidence but unsuitable as an automatic answerer. That is not an incomplete analysis if the code, cases, comparisons and reasoning are complete. It is a bounded conclusion. The missing LLM experiment, human usability study, real-data controls, deployment and platform assessment are separate work and must remain visible.