Benchmarks

How fast a pipeline runs depends less on the library than on how it is spread over the machine. These numbers measure that spread, on one representative pipeline, so the defaults in resolveParallelism rest on measurements rather than intuition. How to act on them is in Performance and parallelism.

What was measured

• Pipeline. Each page is rendered at 144 DPI (PdfToImage). Text regions are found (DbnetOnnxDetector) and read (PaddleRecognizer), and faces are detected (FaceDetector). Every stage is a CPU-bound ONNX model on WASM.
• Input. 12 pages per run: copies of a one-page sample PDF, registered once with putContent and referenced by every row.
• Method. Each setting runs once as a warm-up, so model downloads and session creation are excluded, and is then timed. The figures are steady-state throughput.
• Harness. pnpm bench, in headless Chromium through createScaleDpWorkerPool. A run is rejected if any stage recorded a failure: failing stages return instantly, and would otherwise look fast.
• Environments. Not cross-origin isolated, where onnxruntime-web is single-threaded, as on GitHub Pages. And isolated with COOP/COEP, where WASM threads engage.

The baseline, one worker with one thread and one page at a time, is how the library ran before parallelism was configurable.

Without cross-origin isolation

Threads are unavailable, so the worker pool is the only CPU parallelism.

More pages in flight per worker, for comparison:

With cross-origin isolation

Threads engage, and combine with workers. Shown with one page in flight per worker, which was the fastest on this workload:

The same settings with more pages in flight per worker, on the same scale. Throughput holds level at best, and falls for every multi-threaded setting:

Reading the results

• Workers are the lever. Without isolation, four workers ran 3.2× the baseline. With isolation, four workers of four threads each ran 6.4×.
• More workers beat more threads per worker. At the same total thread count, four workers of one thread outran two of two, which outran one of four. Every worker owns its sessions, so nothing is shared to contend on.
• Threads shorten the wait for the first page. One worker of four threads returned its first page in under a second, against about two for one thread.
• More pages in flight cost throughput here. With every stage a CPU-bound model, a second page in one worker only competes for the same threads, and page latency roughly doubles. Pages in flight pay off where a stage waits on something else, such as a cloud NER call or a WebGPU queue, and this workload has neither.
• Memory is the price. Every worker holds its own copy of each model it loads. The auto preset caps workers by navigator.deviceMemory for that reason.

These are one machine and one pipeline. A GPU stage or a network-bound stage changes the picture, and WebGPU has not been benchmarked yet.

Text detection by provider

DbnetOnnxDetector on a 200 DPI page (1700×2200), measured with squareInput: true, so every call runs ScaleDP's full 1280×1280 input. Each configuration ran in its own worker, cross-origin isolated, in a visible Chromium window so WebGPU got the real GPU. Apple M5 Pro, 2026-10-09:

Every configuration finds the same boxes. For reference, ScaleDP in Python takes about 200 ms per call on CPU.

• Each call costs a full 1280×1280 inference, whatever the picture's size. A pipeline running DBNet over each embedded picture pays per picture: three small pictures on one page cost three calls.
• By default DBNet runs only the picture's own shape (squareInput: false), padded to a multiple of 32. A 1000×300 crop takes 115 ms instead of 450. The model was trained on padded squares, so some boxes differ from ScaleDP's; squareInput: true reproduces them exactly, at the full square's cost. The table above used the square.
• Headless Chromium only offers SwiftShader, a software WebGPU adapter. Timed there, WebGPU took 15.7 s per call, which measures the emulator, not the GPU. BENCH_HEADED=1 gets the real adapter.

BENCH_ISOLATED=1 BENCH_HEADED=1 npx vitest run --project bench bench/dbnet.bench.test.ts --reporter=verbose

Text recognition

PaddleRecognizer reading the 26 lines DBNet found in a scanned Russian page, line orientation on. Nearly all of its time is the recognition model itself: cropping, the orientation check and word splitting take 25–60 ms per page.

By provider (v6-small, each configuration in its own worker, isolated):

That WebGPU row was not the GPU. ppu catalogues its v6 models in ORT format (.ort), onnxruntime's pre-optimised format, which records only the kernels the model needed when it was converted, for the CPU. WebGPU rewrites every Conv into com.ms.internal.nhwc:Conv at session creation, finds no such kernel in the file, and the stage falls back to WASM. The same models are published as .onnx, and the library now fetches those. On WASM they give bit-identical output at the same speed, and on WebGPU recognition runs on the GPU:

By preset (single-threaded):

The model choice moves time and accuracy more than any setting. The default is the slowest of the three and cannot read Cyrillic, while the script's own model is both right and faster. Pick the model from the page's script (TesseractScriptDetector with presetCol) rather than leaving the default.

Batch size: one line per inference is faster and more accurate. A batch pads every line to the widest in it. ppu pads with the line's last column, which is white paper, but PP-OCR's models were trained on PaddleOCR's zero padding (mid-grey once normalised). They misread the white: on the sample scans a batch of 6 garbled 8 of 22 and 10 of 16 lines, and zero padding garbled none. On the Russian scan with v5-eslav-mobile:

The tables above were measured at the previous default of 6.

BENCH_ISOLATED=1 BENCH_HEADED=1 npx vitest run --project bench bench/paddle-recognizer.bench.test.ts --reporter=verbose

Run it yourself

pnpm bench                       # not isolated: workers only
BENCH_ISOLATED=1 pnpm bench      # isolated: workers and threads
VITE_BENCH_PAGES=24 pnpm bench   # more pages per run

Each run prints a table and writes bench/results/<date>-<cpu>-<mode>.json. This page reads those files, so a new result appears here without editing it.

All measurements