Document chunk boundaries
Preserve heading ancestry when splitting reports
A passage needs to retain the section that gives it meaning. Build a small document model before producing the records sent to the search index.
In this article
Extract a document tree before a text stream
Treat the source as a hierarchy where the format allows it. A report has a title, sections and subsections. Those sections contain paragraphs, lists, tables and figures. Keeping that relationship makes it possible to attach the right context to a passage later.
For HTML or Markdown, some of the structure is explicit. For a PDF, a layout tool may infer headings and reading order. Store enough of the extraction result to compare it with the source when an answer looks wrong. A flattened string alone makes that investigation unnecessarily difficult.
Do not assume that a larger font always indicates a section heading. Reports often contain running headers, pull quotes and decorative titles. Use the extraction tool's supported structure information and a regression fixture representative of the actual documents.
Carry the active heading path into each passage
As the ingestion process walks the document, maintain the current section path. When it emits a passage, attach that path as metadata and include a concise form in the text supplied for retrieval or answer construction where useful.
For example, a passage about a delivery exception could retain this record. The field names are illustrative application choices.
{
"sourceId": "delivery-policy",
"sourceVersion": "2026-09",
"headingPath": ["Delivery", "Regional orders", "Exceptions"],
"page": 7,
"passageText": "Orders in this category require manual confirmation."
}The heading path explains which category the sentence refers to. Without it, the passage looks like a general rule applying to all orders. Keep the path tied to the source version so a later document reorganisation does not silently change its meaning.
Split oversized sections with explicit rules
When a section is too large, prefer boundaries between paragraphs or complete list items. Avoid separating an introductory condition from the steps it governs. For tables, repeat the relevant column labels and retain the relationship to notes or units.
Record why each passage begins and ends where it does. A simple boundary reason such as section end, paragraph group or table continuation helps explain unexpected output. It also makes parser changes easier to compare than a collection of unexplained offsets.
Apply the model's actual token limit using the appropriate tokenisation method. A character count is only an approximation. Leave room for the heading context and other metadata that will be added to the model request rather than filling the entire allowance with body text.
Verify the resulting records before indexing everything
Inspect passages from the beginning, middle and end of several documents. Check whether a new heading resets the path correctly and whether repeated page headers have been mistaken for content. Confirm that appendix headings do not inherit an unrelated section from the main report.
Test a source update that inserts a section near the beginning. The passage's source reference should still identify the correct new version rather than relying on an old character position. Rebuild affected records through a versioned process so queries do not combine incompatible metadata and text.
Once the fixture is sound, run the full collection and record extraction failures separately. A document that cannot be parsed reliably should have a visible handling path. Quietly indexing a misleading text stream turns an ingestion problem into an apparently confident answer later.
Primary sources
Microsoft Learn: layout-based chunkingMicrosoft Learn: chunking large documentsReferences checked 11 September 2026.