A practical checklist for preparing documents for RAG

Jun 9, 2026 · The markitdown.ai team · 2 min read

Most RAG debugging starts at the wrong end. Teams tune the retriever, swap embedding models, and adjust top_k — when the real problem is upstream: the text going into the index was never clean to begin with. Garbage in, confident-but-wrong out.

Here's a pre-processing checklist that fixes the input layer first.

1. Normalize every format to one clean representation

You'll ingest PDFs, Word docs, slide decks, spreadsheets, scans, and HTML. Don't write a bespoke extractor for each. Convert everything to one structure-aware format — Markdown works well — so the rest of your pipeline only has to reason about a single, predictable shape.

2. Preserve structure, not just characters

Headings and tables are signal. They tell your chunker where a topic begins and ends.

Keep heading hierarchy so you can chunk on section boundaries instead of arbitrary character counts.
Keep tables intact; a flattened table is noise that pollutes nearby chunks.
Keep reading order correct, especially for multi-column source material.

3. Strip the furniture

Page numbers, running headers and footers, navigation, ads (for HTML), and hyphenated line breaks add nothing to retrieval and quietly degrade embeddings. Remove them before chunking, not after.

4. Chunk on meaning, then size

With clean headings in place, prefer structure-aware chunking: split on sections and subsections, then sub-split only the chunks that are still too large. This keeps related ideas together far better than a blind fixed-window split. A little overlap between chunks helps preserve context across boundaries.

5. Carry metadata with each chunk

Attach source filename, section heading, and page (if available) to every chunk. It costs almost nothing and pays off twice: better filtering at retrieval time, and real citations you can show the user.

6. Make ingestion idempotent and observable

You will re-ingest. Documents change, chunking strategies change, embedding models change. Key chunks deterministically so a re-run updates rather than duplicates, and log what was skipped so "we indexed everything" is verifiable, not assumed.

Where conversion fits

Steps 1–3 are a single concern: turning an arbitrary document into clean, structured text. That's the job markitdown.ai is built for — convert a file in the browser, run a folder through batch, or call the API inside your ingestion job. Get the input layer right and the rest of the RAG stack has a much easier job.