Unit 12.01: Building the retrieval chain

The assembled chain: scored retrieval, a threshold, a grounding prompt, and a refusal path that exists in code.

Every piece from the last eleven modules

Stable chunk ids, metadata with dates, scored retrieval, an explicit threshold, formatted context carrying ids, and a decline branch.

The code answers a covered question and an uncovered one.

from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.vectorstores import InMemoryVectorStore

REFUSAL = "The available documents do not cover that."
DOCS = [Document("Refunds are allowed within 7 days of purchase.",
                 metadata={"id": "support-policies-v4.md#refunds:2",
                           "updated": "2026-06-14"})]
store = InMemoryVectorStore.from_documents(DOCS, DeterministicFakeEmbedding(size=64))

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer only from CONTEXT. Cite the chunk id for every claim."),
    ("human", "CONTEXT:\n{context}\n\nQUESTION: {question}")])
model = FakeListChatModel(
    responses=["Refunds are allowed within 7 days. [support-policies-v4.md#refunds:2]"])
generate = prompt | model | StrOutputParser()


def answer(question, k=4, threshold=0.75):
    scored = store.similarity_search_with_score(question, k=k)
    keep = [(d, s) for d, s in scored if s >= threshold]
    if not keep:
        return {"text": REFUSAL, "cites": [], "refused": True}
    context = "\n".join(f"[{d.metadata['id']}] {d.page_content}" for d, _ in keep)
    return {"text": generate.invoke({"context": context, "question": question}),
            "cites": [d.metadata["id"] for d, _ in keep], "refused": False}


for q in ["What is the refund window?", "What is the office address?"]:
    out = answer(q)
    print(f"{q:32} refused={out['refused']}")
    print(f"   {out['text']}")

The refusal is a branch in answer, not an instruction in the prompt. That is what makes it testable: the function returns refused: True and an empty citation list, both assertable without reading English.

The chunk id is support-policies-v4.md#refunds:2 - document, section and position - so it survives a rebuild and a reader can follow it. It is verbose in the output and that verbosity is the feature.

The mistake this prevents

The mistake is building this with similarity_search and adding the threshold later. The unscored version has no refusal path at all, so every test you write against it is a test of the answering path, and the refusal arrives untested.

Takeaway

Assemble scored retrieval, an explicit threshold, ids in the context, and a refusal branch in code. The refusal must be a return value, not a prompt instruction.

Mark complete