Unit 02.02: Swapping providers without a rewrite

Provider portability is the framework's headline benefit, and it is real at the interface level and not at the behaviour level.

Same chain, different model object

The prompt, the parser and the composition are unchanged. Only the model object differs.

The code runs one chain definition against two models.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.language_models.fake_chat_models import FakeListChatModel

prompt = ChatPromptTemplate.from_messages([("human", "{question}")])
parser = StrOutputParser()

# Two "providers", same interface. In production these are ChatOpenAI,
# ChatAnthropic, and so on -- the chain around them does not change.
provider_a = FakeListChatModel(responses=["answer from provider A"])
provider_b = FakeListChatModel(responses=["answer from provider B"])

for name, model in [("provider A", provider_a), ("provider B", provider_b)]:
    chain = prompt | model | parser
    print(f"{name}: {chain.invoke({'question': 'hello'})}")

print("\nThe chain definition is identical. Only the model object changed.")

# What does NOT transfer: token limits, tool-calling formats, system-message
# handling, and pricing. The interface is uniform; the behaviour is not, so a
# swap still needs its eval set re-run.

The chain code is identical, which is genuinely useful - swapping a provider does not mean touching your retrieval, your parsing or your wiring.

What does not transfer is everything about behaviour: token limits, tool-calling formats, how the system message is weighted, refusal tendencies, and pricing. A swap changes the answers, so it needs the eval set re-run - the interface being uniform is exactly what makes it tempting to skip that.

The mistake this prevents

The mistake is treating a provider swap as a configuration change because the code did not change. The code not changing is the abstraction working; the behaviour changing is the model being a different model. Re-run the eval set and compare per case.

Takeaway

The interface is portable; the behaviour is not. Treat a provider swap as a change requiring the full eval set, not as a config edit.

Mark complete