Module 04 Summary

What this module established

Structure discarded at load time cannot be recovered - headings and dates become metadata or they are gone. Chunk size follows from the semantic unit, and the context window constrains k rather than what belongs in one chunk.

Carry forward

• Overlap mitigates bad boundaries at the cost of index size and duplicate hits. Fix the boundary first.
• Embed the corpus once and cache by content. Only the query needs embedding per request.
• Three string checks at ingestion catch chunks that produce unfollowable citations. Alert on the rate, not the individual failures.

Before moving on

Move on when structure is in metadata, chunk size is justified by a stated semantic unit, and the stand-alone check runs on every ingestion.

Mark complete