Unit 02.00: Messages, roles, and what the model really sees

The template is not what the model receives. Looking at the resolved messages once, per chain, prevents a category of confusion.

System and human are separate messages

A chat model receives a list of messages with roles, not a single string. The template renders into that list, and rendering it yourself is one line.

The code prints the resolved messages.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You answer from the policy text only."),
    ("human", "Policy: {policy}\n\nQuestion: {question}"),
])
messages = prompt.invoke({"policy": "Refunds within 7 days.",
                          "question": "How long do I have?"})

for message in messages.messages:
    print(f"[{message.type:6}] {message.content!r}")

print("\nThis is what goes over the wire. Print it once per chain you build.")

# Two things become visible only here: the system message is a separate message
# rather than a prefix, and the policy text is inside the human turn -- which
# means the model sees it as something the user said, not as a system rule.

Two things become visible only here. The system message is a genuinely separate message rather than a prefix on the user's text - providers treat it differently, and some weight it more heavily.

And the policy text sits inside the *human* turn. That means the model sees it as something the user supplied rather than as a system rule, which matters when the two conflict. If the policy is authoritative, consider putting it in the system message instead.

The mistake this prevents

There is a security consequence to this too. Never interpolate untrusted text - a user's message, an inbound email, a fetched web page - into the system message. The system message is where your instructions live, and text placed there competes with them on equal footing: "ignore previous instructions" inside a document you summarise is a real attack. Untrusted content belongs in a human turn, clearly delimited, with the system message stating that content in the user turn is data to be processed rather than instructions to be followed.

The mistake is assuming role assignment is cosmetic. Where the retrieved context lands changes how it competes with the user's own instructions, and it is the sort of thing that only shows up when a user's message contradicts the context.

Takeaway

Render the messages and look at them once per chain. Role assignment is a design decision, particularly for retrieved context that must outrank the user's own words.

Mark complete