PdfMergeImageText

Import

import { PdfMergeImageText } from '@stabrise/scaledp/pdf'

Group

Transform

Reads

document, document

Writes

document

Needs

nothing

PdfEmbeddedImages leaves a page spread across several rows, each holding one image's Document in that image's own pixels. This stage maps those boxes onto the page through the placement each row carries, merges them with the text layer, and emits the page once.

import { PdfMergeImageText } from '@stabrise/scaledp/pdf'

new PdfMergeImageText({ strategy: 'text-layer-wins' })

It is the one stage in the library that reduces the row count.

The searchable-scan problem

A page can describe the same words twice: a photograph of a document with an invisible OCR text layer laid over it. Concatenating both sources says everything twice, so one has to win.

• text-layer-wins (the default) keeps the PDF's own words and drops any OCR box the text layer already covers. The text layer is exact where it exists, so this is right for almost every corpus.
• ocr-wins is the mirror. Worth reaching for when a corpus extracts as mojibake — mis-encoded CID fonts are the usual cause — so the "exact" text layer is exactly wrong.
• union keeps both, for comparing the two readings.

The test is coverage, not IoU: a word read by OCR sits wholly inside the line-level box a text layer reports for the same words, and their IoU is small. An IoU test would keep both and the text would come out doubled. coverageThreshold is how much of a box the winner must cover before the loser is dropped.

A text layer that does not match the picture

Coverage alone trusts the winner blindly. On a searchable scan that is right, because the invisible layer is an OCR of the very picture beneath it. But a layer can disagree with what is printed: stale after the image was edited, a stamp laid over a scan, or planted to hide what is underneath. Under coverage alone, an invisible CONFIDENTIAL over a picture of Freya Yamamoto would erase the name from the result, and for redaction the visible words are the ones that leak.

So with matchText (on by default) the losing box is dropped only when the winner covering it reads the same words. The comparison ignores case, accents and punctuation, looks for the shorter text inside the longer (an OCR word inside the layer's line), and allows about one edit in three characters for OCR's misreads. Where the two disagree, both readings are kept.

matchText: false drops on coverage alone, which is how Python ScaleDP merges.

Pictures that overlap

A page can draw one picture over another, such as a framed photo with a second image laid on part of it. Each is read on its own, so the same name comes back twice: once whole, and once as a stray fragment from the other picture's crop. OCR readings are therefore de-duplicated against each other as well as against the text layer, by the same rule: a reading covered by a larger one that says the same words is dropped, and two readings that disagree are both kept.

Words OCR split apart

A text detector often cuts a word's first letter off as a region of its own (a larger initial, a letter with a gap after it), so the recognizer returns F and reya. Neither is the word, and line ordering can even put the letter after the rest. With joinFragments (on by default) such pieces are glued back into Freya before the merge. Two boxes count as pieces of one word when they sit on the same line and touch: a gap under 15% of the text height, where a word space is nearer 25–30%, or an outright overlap. One side must be a fragment of at most three characters unless the boxes truly overlap, and a box stacked on another is never fused into it.

It runs on each picture's own boxes before they are mapped onto the page, so boxes read from two different pictures are never joined.

Reading order, across both sources

text and bboxes come out of a single line grouping over the merged set, so the two agree — the boxes are in the order the text was built from, which is what lets NER character offsets map back to boxes.

Grouping the merged set rather than each source is the point. A vector-text header above an OCR'd table reads header-first; concatenating by source would put the header after the table whenever the image happens to sit at the top of the page.

Document.type is 'pdf+ocr' when an image contributed, 'pdf' when the text layer answered alone.

A failed image does not fail the page

One image's recognizer failure is recorded and dropped: the page keeps its text layer and every other image's reading, and exception stays empty. It is set only when nothing at all could be read, because hasUsableTextLayer and every downstream stage treat a non-empty exception as "upstream failed" — so poisoning the page over one bad picture would throw away the good text with it.

The limitation is the schema's: Document has one error slot, so a partial failure is not visible in the output.

Parameters