PDF to Markdown, and why the structure is guesswork

A PDF is a print format. Its job is to put the right glyph at the right spot on the page, and it does that by storing text runs with a position and a font. Nothing in the file says which of them is a heading, where one paragraph ends, or that six lines are a bulleted list.

So a Word document converts; a PDF gets reconstructed. Everything below is a heuristic, and every heuristic has cases it gets wrong. That's why one warning appears above the output on every single PDF conversion, whether or not anything looked odd.

If you have the original .docx, use it instead. It converts properly, because the structure is really in it.

First the body text size is worked out: every line's font size is tallied, weighted by how many characters it holds, and the most common one wins. Weighting by characters rather than by lines matters — one long paragraph line says more about what the body size is than one short heading line does.

A line is then a heading if it's meaningfully bigger than that, and short. The size ratio picks the level: 1.6× or more becomes #, 1.35× becomes ##, and 1.15× becomes ###. The original heading levels are gone, so what's preserved is only the relative relationship — these headings are the same rank as each other.

The length limit is 120 characters, and it's there for a specific failure: a document whose opening paragraph is set in large type is not a heading, and font size alone can't tell the difference. Length can.

This is also the heuristic that misses most often. A document that styles its headings by weight rather than size — bold at the same size as the body — has no size signal at all, and those headings come out as paragraphs.

Headers and footers are dropped when they're both smaller than the body text and sitting within 9% of the page height from the top or bottom edge. Proportional rather than a fixed measurement, so A4 and Letter behave the same. A page number or a "Confidential" stamp repeated on every page would otherwise interrupt the text forty times.

A paragraph break is a vertical gap wider than 1.8× the line's own font size. That's how a PDF expresses one, since there's no paragraph mark to read.

Lists are found by their first characters: a bullet glyph (•, ·, ▪, ◦, ‣, ∙) or a number up to three digits followed by a dot or a closing bracket. A list drawn some other way is not a list to the converter.

PDFs hyphenate at line ends, and a naive join gives you "re- port". The lines get spliced back into one word when a line ends in a hyphen and the next line starts with a lowercase letter — that combination is a line-break hyphen almost every time.

When the next line starts with a capital, the hyphen stays. "State-of-the-art" broken after "state-" keeps its hyphen, which is correct, because that one is part of the word.

Chinese, Japanese and Korean lines are joined with no space at all. Those scripts don't separate words with spaces, so inserting one on every line break would put a gap in the middle of a sentence.

If a page is an image of text — a scan, a photo, a fax — there are no characters in the file to read, only a bitmap. The converter says so and stops, rather than handing back an empty document that looks like a bug.

Getting text out of that needs OCR, which isn't part of this tool. A PDF where some pages are scans and others aren't will convert the real pages and list the page numbers of the ones that came back empty, up to six of them.

Two ceilings: 25 MB and 500 pages. Both are about what a browser tab can do without locking up — a PDF is rendered here, in your browser, with no server involved.

Two-column layouts are the hardest case and they're best-effort. Text is read in the order the file stores it, which for a well-made two-column PDF is column by column, and for a badly-made one alternates between columns line by line. There's no reliable way to tell which you have before reading it.

Tables and formulas are the same story: a PDF table is ruled lines and text at coordinates, with nothing marking it as a table. Expect the cells as separate lines rather than a pipe table.

Turn on page marks if you're going to fix things up by hand. That inserts an HTML comment before each page, so you can find the page a mangled section came from and check it against the original.