Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 02.01: Splitting text without cutting the meaning in half

Fixed-size splitting is the default in every RAG tutorial because it is trivial to implement. It is also the reason so many systems retrieve passages that end mid-word.

Where the boundaries actually belong

Text has natural seams: sentence ends, paragraph breaks, headings, list items. A character counter knows about none of them. It cuts at position 40 - or 500, the arithmetic is the same - regardless of what sits there.

The example below splits one passage both ways and prints the chunks so you can see the damage rather than reason about it.

fixed-width 40 characters:
  'Refunds are allowed within 7 days of pur'
  'chase. This applies to individual plans '
  'only. Enterprise contracts are handled s'
  'eparately.'

on sentence boundaries:
  'Refunds are allowed within 7 days of purchase.'
  'This applies to individual plans only.'
  'Enterprise contracts are handled separately..'

The fixed-width split produces a chunk ending ...within 7 days of purchas. It is retrievable, it scores well on a query about refunds, and a reader following the citation gets a truncated fragment. The sentence-boundary split produces chunks that each finish their own thought.

Worse than truncation is separation. When a rule and its exception land in different fixed-width chunks, the rule chunk ranks - it contains the query terms - and the exception does not. The answer is then correct according to its citation and wrong in practice.

The mistake this prevents

The mistake is adding overlap as a substitute for thinking about boundaries. Overlap papers over bad splits by making them redundant, which inflates the index, produces near-duplicate results, and still fails when a qualifier sits further away than the overlap window. Fix the boundary first, then add overlap if it still helps.

Takeaway

Split on the seams the text already has. Sentence and heading boundaries cost almost nothing to detect and prevent the failure fixed-size splitting guarantees: a rule retrieved without its exception.