Unit 06.00: Wiring retrieval into the answer

The join between retrieval and generation is a dictionary, and reading it correctly is most of understanding a retrieval chain.

Two branches feeding one template

One branch retrieves and formats; the other carries the question through unchanged. Both run before the prompt and both fill its variables.

The code builds the full chain and answers a question.

from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.vectorstores import InMemoryVectorStore

store = InMemoryVectorStore.from_documents(
    [Document("Refunds are allowed within 7 days.", metadata={"id": "c1"})],
    DeterministicFakeEmbedding(size=64))
retriever = store.as_retriever(search_kwargs={"k": 1})


def format_docs(docs):
    return "\n".join(f"[{d.metadata['id']}] {d.page_content}" for d in docs)


prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer only from CONTEXT. Cite the chunk id."),
    ("human", "CONTEXT:\n{context}\n\nQUESTION: {question}"),
])
model = FakeListChatModel(responses=["Refunds are allowed within 7 days. [c1]"])

chain = ({"context": retriever | format_docs,
          "question": RunnablePassthrough()}
         | prompt | model | StrOutputParser())

print(chain.invoke("What is the refund window?"))

# The dict is the join: `retriever | format_docs` fills `context` while
# `RunnablePassthrough` carries the question through unchanged. Both run before
# the prompt, and both feed one template.

{"context": retriever | format_docs, "question": RunnablePassthrough()} is the whole join. The dict's keys are the template's variables, and each value is a runnable producing that variable.

format_docs is doing something easy to overlook: it puts the chunk id into the context text. Without that the model cannot cite anything, because it never saw an id - which is the most common reason a grounding prompt produces uncited answers.

The mistake this prevents

The mistake is passing the raw Document objects into the prompt. They stringify into something that includes the metadata dict, which is noisy, wastes tokens, and gives the model a citation format nobody chose. Format explicitly.

Takeaway

The dict is the join: keys are template variables, values are runnables. Format documents explicitly and put the id in the text, or the model has nothing to cite.

Mark complete