NoWaterProgramming

What PDF Tools Actually Do: The Object Model, Stream Codecs, and Font Embedding

Compress, merge, split, linearize and OCR are five different operations on the same file format. What each one touches in the object graph, why some are fast and some are not, and the three that can quietly corrupt a document.

7 min read
Share:

Most guides treat a PDF as a black box you feed into a button. It is not. A PDF is a structured file format with a well-defined object model, and once you understand that model the everyday operations (compress, merge, split, convert, OCR, sign) stop being magic and start being predictable. This is a tour of what those tools actually do to the bytes, and of the three that can damage a document while reporting success.

References below are to ISO 32000-1, the PDF 1.7 specification, which Adobe publishes for free. PDF 2.0 is ISO 32000-2 and does not change any of the structure described here.

The Object Model Underneath Every Operation

A PDF file is a collection of numbered indirect objects: dictionaries, arrays, names, numbers, strings, and streams. At the tail of the file sits the cross-reference table (the xref, clause 7.5.4), which maps every object number to a byte offset, followed by a trailer that points at the document catalog and the start of the xref. Page content is stored in content streams: sequences of operators (BT/ET for text blocks, Tf to select a font, Tj to show text, re/f to fill rectangles).

This explains why some tools are fast and some are slow. Splitting and merging are mostly bookkeeping: a good tool copies the relevant object subgraphs, renumbers them to avoid collisions, and rebuilds one coherent xref and trailer. It does not re-render anything. That is why merging a hundred-page report should take milliseconds, and if a tool is slow at it, it is probably rasterizing pages it had no reason to touch.

It also explains the format's most surprising property, which has its own section below: a PDF can be updated by appending, so removing something from the visible document does not necessarily remove it from the file.

Why "Compress" Means Five Different Things

PDF compression is not one algorithm. Each stream carries a /Filter entry naming its codec (clause 7.4): FlateDecode (zlib/DEFLATE) for text and vector content, DCTDecode (JPEG) and JPXDecode (JPEG 2000) for images, CCITTFaxDecode and JBIG2Decode for bilevel scans. A serious compressor works on several fronts at once:

  • Downsampling and re-encoding embedded images, where the real bytes usually live.
  • Subsetting fonts so only the glyphs you use are embedded, not the entire typeface.
  • Deduplicating identical objects (repeated logos, shared images) into a single referenced stream.
  • Stripping redundant metadata, unused objects, and orphaned page resources.

If a "compress" tool only re-zips already-Flate-compressed text, it saves almost nothing, because DEFLATE over already-compressed data is a near no-op. The win is in images and fonts.

That win is where the risk lives too. Image re-encoding is lossy by construction, and the most aggressive bilevel option has a documented failure mode that is worse than blur. JBIG2Decode in its lossy mode builds a dictionary of glyph shapes and replaces similar-looking regions with references to the same dictionary entry. In 2013 David Kriesel showed that Xerox WorkCentre scanners doing exactly this were substituting digits in scanned construction plans: a 6 rendered crisply as an 8, with no visual artefact to suggest anything was wrong. The output looked cleaner than a faithful scan. It was also incorrect. Any pipeline that scans numbers and compresses aggressively should know which mode it is using.

Linearization, the Structure Power Users Overlook

Linearized PDF (often labelled "fast web view", Annex F of the spec) reorganizes the file so the first page and its resources sit at the front, with a special linearization dictionary and a second xref. A viewer can then render page one before the whole file has arrived over a network. Tools that "optimize for web" are doing exactly this reordering, not extra compression.

Worth knowing: aggressive editors can silently de-linearize a file on save, because appending an incremental update puts new objects after the point linearization assumed was the end. If fast web view matters, verify it after every round trip rather than after the first one.

OCR, Fonts, and the Invisible Text Layer

A scanned PDF is just images. OCR adds a searchable text layer by running recognition, then writing the recognized words back as text drawn with text rendering mode 3, which Table 106 in clause 9.3.6 defines as "neither fill nor stroke text (invisible)", positioned to align with the underlying image glyphs. The image stays visually authoritative; the hidden text makes it selectable and searchable.

The consequence people miss is that nothing checks the two against each other. The image says one thing, the invisible layer says another, and every downstream consumer that extracts text (search indexes, data pipelines, anything automated) reads the layer, not the picture. A human proofreading the page cannot see the text being indexed. Quality depends on input resolution and the engine's language models, and 300 DPI is the widely used practical floor for document scanning rather than a value the PDF spec prescribes.

Fonts are the other place quality is won or lost. PDFs embed fonts as Type1 or TrueType (simple fonts, clause 9.6) or Type0 (composite, CID-keyed, for large character sets, clause 9.7), ideally subsetted. When text renders as boxes or extracts as garbage, the usual culprit is a font with no embedded program and a broken /ToUnicode CMap (clause 9.10.3), the table that maps glyph codes back to Unicode for copy and search. A good toolchain preserves or rebuilds that mapping. A subsetted font, meanwhile, is smaller and no longer editable in any application that needs a glyph you did not use.

Deleting Is Not Deleting

This is the one that ends up in the news. PDF supports incremental update: a change can be written by appending new objects and a new xref section, leaving the previous version of every object in the file. The viewer follows the latest cross-reference chain and shows you the current document. The earlier bytes are still there.

Two practical consequences:

  • Redaction by drawing a black rectangle does nothing. The rectangle is a new object painted on top. The text underneath is untouched and extracts perfectly. Redaction has to remove the content stream operators, and then the file has to be written without the history.
  • "Sanitised" files can carry their own drafts. Metadata, deleted pages and superseded object versions survive an incremental save. Rewriting the file completely, rather than appending, is what removes them.

If a tool cannot tell you whether it rewrote or appended, assume it appended.

A Practical Mental Checklist

When you reach for a PDF tool, ask what it touches:

OperationWhat it modifiesRisk
Merge, splitObject graph and xref onlyLow; broken bookmarks and named destinations
CompressStream codecs, font subsetsLossy image re-encoding; JBIG2 symbol substitution
LinearizeFile orderingSilently undone by a later incremental save
OCRAdds an invisible text layerLayer and image can disagree, and nothing checks
RedactShould rewrite the whole fileAppending leaves the original content in the bytes

When something breaks, inspect the structure instead of guessing. Open the file, read the xref, check the /Filter entries, and look at the font dictionaries. The format is legible by design, and the tools worth keeping are the ones that respect that design.

Sources

Checked 2026-08-20.

  • ISO 32000-1:2008 (PDF 1.7), published free by Adobe - clause 7.4 for the filter list including FlateDecode, CCITTFaxDecode, JBIG2Decode, DCTDecode and JPXDecode; clause 7.5.4 for the cross-reference table and trailer; clause 9.3.6 and Table 106 for text rendering mode 3 being invisible; clauses 9.6 and 9.7 for simple and composite fonts; clause 9.10.3 for ToUnicode CMaps; Annex F for linearized PDF.
  • ISO 32000-2:2020 - the current PDF 2.0 edition, for the note that none of the above structure changed.
  • David Kriesel, "Xerox scanners/photocopiers randomly alter numbers in scanned documents" - the JBIG2 pattern-substitution failure, including the scanned construction plans where digits changed.

Related Posts

9 min read
A walkthrough of QR internals against the spec: the module grid and its reserved regions, encoding modes, Reed-Solomon over GF(256), interleaved blocks, mask selection, and what the standard deliberately does not protect.
By NoWaterProgramming Team