Unit 04.02: Overlap, and what it is really for

Overlap is presented as a default setting. It is a mitigation with a cost, and it is worth knowing which problem it actually addresses.

Lower separation risk, larger index

Overlap repeats the end of each chunk at the start of the next, so a qualifier separated from its rule appears in both.

The code splits the same text with and without overlap and reports the storage multiplier.

from langchain_text_splitters import RecursiveCharacterTextSplitter

TEXT = ("Refunds are allowed within 7 days of purchase. "
        "This applies to individual plans only.")

for overlap in (0, 30):
    splitter = RecursiveCharacterTextSplitter(chunk_size=50, chunk_overlap=overlap)
    chunks = splitter.split_text(TEXT)
    total = sum(len(c) for c in chunks)
    print(f"overlap={overlap:<3} {len(chunks)} chunks, {total} chars stored "
          f"({total / len(TEXT):.2f}x the original)")
    for c in chunks:
        print(f"    {c!r}")
    print()

# Overlap buys a lower chance that a qualifier is separated from its rule, and
# it costs index size and near-duplicate hits. It is a mitigation for bad
# boundaries, not a substitute for choosing good ones.

Overlap costs index size directly - every overlapping character is stored and embedded twice - and it produces near-duplicate results, so a top-k of four may contain two views of the same passage.

What it buys is a lower chance that a rule is retrieved without its exception. That is a real risk and the right first response is better boundaries, not more redundancy: a sentence-boundary split avoids the problem rather than papering over it.

The mistake this prevents

The mistake is raising overlap when retrieval quality is poor. It inflates the index, adds duplicate hits, and still fails when a qualifier sits further away than the overlap window. Fix the boundary first, then add overlap if it measurably helps.

Takeaway

Overlap mitigates bad boundaries at the cost of index size and duplicate hits. Choose the boundary properly first; add overlap only if it improves a measurement.

Mark complete