Porting from Python ScaleDP

The two libraries share a pipeline model, stage names, parameter names and schemas. What differs is the runtime.

Stage map

API differences

Options objects, not positional inputCol/outputCol. The column names remain valid options with the same defaults, so a stage can still be wired explicitly.

TesseractOcr(inputCol="image", outputCol="text", keepFormatting=True)

new PaddleTextRecognizer({ inputCol: 'image', outputCol: 'text', keepFormatting: true })

Rows, not a DataFrame. transform returns Row[] — plain objects. Page explosion produces several rows per input, exactly as posexplode does.

Everything is async. Model loading, decoding and inference are all promises.

No Estimators. There are none in Python ScaleDP either; every stage is a pure transform. There is no fit().

What carries over unchanged

• The non-throwing error contract: failures land in exception and the pipeline completes. propagateError opts into throwing.
• The Box convention: x/y is the top-left of the axis-aligned box of the same size centred on the rotated rect's centre; angle is degrees about that centre; width is the longer side.
• Parameter names and defaults, wherever a Python equivalent exists.
• Layout-preserving text reconstruction under keepFormatting, including the per-line indent and blank-line rules.

Both the Box geometry and the text reconstruction are verified against the real Python implementation: test/fixtures/*.py generate goldens by running ScaleDP itself, and the parity suites diff against those.

Deliberate divergences

Each is a fix, and each is commented at its site.

PdfToDocument emits pixels, not points. Text-layer and OCR boxes then share one coordinate space and can be compared directly.

TesseractRecognizer.onlyRotated defaults to false. In ScaleDP it is true, because there the stage refines an OCR pass that already ran and skipping upright boxes is the whole point. Standalone it is the primary recognizer, and that default would return an empty document for the ordinary detector-then-recognize pipeline.

TesseractRecognizer can return word boxes. boxLevel: 'word' is an addition, not a fix: Python always appends the region box it was handed, so its boxes are as coarse as the detector's — line-level, for every detector here. The words were always in the response (getTextBoxes('word')); only their geometry was being discarded. The default stays 'region', so nothing changes unless asked.

Rows carry their own timings. row_time is an addition: Python's execution_time is per stage across the whole run, identical on every row, and this library keeps that column with exactly that meaning. Per-row numbers are what answer "which page was slow", so they live beside it rather than changing it.

NER de-duplicates across chunks. Python's 500/480 sliding window has no cross-chunk dedup, so an entity in the 20-character overlap is reported twice.

The character-to-box map is built from the real text. Python derives it from len(box.text) + 1, assuming exactly one separator per box. That drifts as soon as keepFormatting inserts several spaces or a newline, shifting every entity's boxes after the first wide gap. Here each box's text is located in the document text instead.

DBNet runs the picture's own shape. ScaleDP pads every input to the full 1280×1280 square. Here it is padded only to the next multiple of 32 by default, which is up to 4× faster on a small or wide picture. The model was trained on squares, so some boxes differ; squareInput: true restores ScaleDP's input exactly.

PaddleOCR reads one line per inference. ScaleDP's PaddleRecognizer defaults to recBatchSize: 6. Here it is 1: see the padding bug below, and batching is not faster on WASM anyway.

A stage initialises lazily, inside the error contract. init() runs just before the first row that reaches the stage, so a stage no row reaches never downloads its model. A failed init() is recorded on that stage's rows like any other failure, where ScaleDP raises.

Box colours are a stable hash. colorForGroup hashes the group name, so a label is the same colour on every render. Python picks a random colour per run, which makes two renders of the same document impossible to compare.

Python bugs not reproduced

Found while porting; listed so nobody "fixes" the TypeScript side back toward them.

• DBNetOnnxDetector.scoreThreshold never reaches DBPostProcess; the effective threshold is the hardcoded 0.3. Here the parameter is wired through.
• use_gpu = params["model"] == Device.CUDA compares a model-name string to an enum and is always false.
• YoloOnnxTextDetector multiplies by the letterbox scale where DBNet divides.
• Ner.aggregate_ner_results builds new_ner_results and then discards it.
• PaddleRecognizer.read_crops pads each batched crop with np.pad(..., mode="edge"), repeating its last column (white paper on a document). PaddleOCR's own preprocessing, which the models were trained on, pads with zeros. On the sample scans a batch of 6 garbled 8 of 22 and 10 of 16 lines; zero padding garbled none. Python's default recBatchSize is 6.
• TesseractRecognizer sets b.conf but filters on b.score, so scoreThreshold is applied to the detector's score rather than the recognizer's.