Release notes
What changed in each version of @stabrise/scaledp, and what it means for a pipeline you have already written.
What shipped, newest first. The full record — including the reasoning behind
each change — is CHANGELOG.md
in the repository.
The package is pre-1.0. Defaults still move when a better default is found, and every such move is called out below rather than left to be discovered on a Monday morning.
0.6.0 — 12 October 2026
One pipeline per document: streamed page by page, with stages switched per run, models loaded only for the pages that need them, and configurable parallelism.
Defaults that changed
Several, most of them in what a stage or the worker reports rather than in the text it reads:
DbnetOnnxDetectornow defaults tosquareInput: false: it pads the picture to the next multiple of 32 instead of to a 1280×1280 square. Up to 4× faster on a small or wide picture, about 1.3× on a page, but some boxes differ from ScaleDP's. SetsquareInput: truefor exact parity.- Paddle recognition reads one line per inference.
recBatchSizedefaults to 1, down from 6, inPaddleRecognizer, andPaddleTextRecognizergains the same parameter. A batch padded shorter lines with white paper, which the models misread; on the sample scans a batch of 6 garbled 8 of 22 and 10 of 16 lines. A batch of 1 has no padding and is about 20% faster. - A stage's
init()failure no longer throws out oftransform. It is recorded inexceptionon that stage's rows and the rest of the pipeline completes.propagateError: truestill throws. init()is called lazily, just before the first row that reaches the stage. A stage no row reaches never initialises.ensureModelFilesends withreadyafter a download as well as on a cache hit. Engine stages end their download oninitializingand reportreadyonce the session exists, so a model is no longer shown as ready while its session is still being built.- Concurrent loads of the same model files share one download. Aborting one caller's signal stops that caller waiting; the download still finishes into the cache.
- The worker host caches stage instances, not pipelines. Changing stages no
longer disposes the previous ones at once; up to
maxCachedStages(default 12) are kept and disposed least recently used first. - Worker results carry
contentRefback, rather than a copy of the file on every page row, for rows that were sent by reference. ScaleDpWorkerOptions.onProgressandonStagereceive a trailingrequestId.FitBoxesToInktakes at most half the gap to a neighbour, where it used to take all of it, and turned boxes now get neighbour limits too. Two boxes growing towards each other can touch but no longer overlap.
Streaming runs
Pipeline.run streams by page: once a document is split into pages, each page
walks the remaining stages on its own. It returns the same rows in the same
order as transform, which keeps Python's stage order. See
Streaming runs.
const rows = await pipeline.run(file, {
pagesInFlight: 2,
enabled: { faces: settings.detectFaces },
onEvent: (event) => status.update(event),
onRow: (update) => overlay.draw(update.unit.page, update.row),
})- Events (
onEvent, orpipeline.events()as an async iterable) cover the run, each stage's model loading and progress, each stage over each page, skips, row errors and barriers. Every page event carriesunit: { key, input, page, path }. - Row updates (
onRow) report each stage's rows per page, as a delta of what the stage wrote or in full, optionally with binary data stripped. - Per-run stage toggles. Give a descriptor an
idand switch it withRunOptions.enabled. No rebuild, no reload, and a disabled stage never touches its model. whenconditions, serialisable per-row tests (path/var,eq,in,gte,all/any/not, …). Rows that do not match pass through untouched.usableTextLayer()from/pdfishasUsableTextLayeras a condition, so text-layer-or-OCR is one pipeline.- Barriers. A stage with
scope: 'document', such as document-scopeNerConsistency, waits for every page;barrier: trueforces one. planPagesonPdfToDocument,PdfToImageandPdfEmbeddedImagessplits a document before the first PDF stage runs, not after it has rendered every page.
The whole shape is in the recipe One pipeline per document.
Parallelism and the worker
pagesInFlightruns several pages at once, with the oldest page getting a free stage first. Stages declaremaxConcurrencyand the shared sessions they hold, so PaddleOCR stages on one preset share a lock.StageDescriptor.runtime.executionProviderssets providers per stage.createScaleDpWorkerPool(/worker) spreads pages over several workers and looks like one worker from outside. On a page that is not cross-origin isolated it is the only CPU parallelism there is.resolveParallelism('auto' | 'conservative' | partial, probeEnvironment())picks workers, threads per worker, pages in flight and providers for the machine. See Performance and the measured numbers on Benchmarks.client.run()returns aRunHandle(async iterable,result,cancel()), and a cancel bypasses the queue.configureresolves with the worker's effective environment:numThreads,crossOriginIsolated,webgpu.createScaleDpWorker({ auth })andcreateScaleDpWorkerPool({ auth })answer the worker's token requests from the main thread, so a worker entry no longer needs its own copy of the token logic.
Models
- PaddleOCR runs on WebGPU. The
.ortfiles ppu catalogues had no WebGPU kernel for their convolutions, so every session fell back to WASM. The library now fetches the.onnxtwin of each model. On WebGPU, recognising 26 lines takes 275 ms instead of 1945 (v6-small) and detection 60 ms instead of 442; on WASM the output is bit-identical. The models download once more under their new URLs. - Model management in
/registry:pipelineModelslists what a pipeline will download,modelStatusreports what is cached and how large it is,prefetchModelsdownloads ahead of time with per-model progress, andevictModelsdeletes. See Managing models. - One download per origin. Each model file is fetched under a Web Lock named after its cache key, so when the page and its workers want the same file, one downloads it and the others read it from the cache, seeing the download's progress meanwhile.
New stages
NerMergermerges several NER readings of a page into one, rebasing every entity's offsets onto one document by its boxes. An entity two readings both found is kept once, at its best score, and one failed reading does not cost the page the others.DocumentDeltaemits the boxes a later reading of a page added — the text-layer page merged with its pictures' text — as aDocumentof its own, so a second NER pass reads only what is new. Gated withwhenondelta.bboxes.length, a page whose pictures added nothing costs one NER call.DbnetOnnxDetectorgains an opt-inpack, detecting several of a page's pictures in one 1280×1280 inference. A picture alone on its input comes out exactly as unpacked; pictures sharing one can read differently.
Fixes and performance
PdfMergeImageTextno longer lets a stray reading delete real lines. A photograph read as one letter over the whole picture used to count as a repeat of every line beneath it, and those lines were dropped.FitBoxesToInkno longer grows boxes into each other (see above). On the extraction-conditions benchmark page fitting used to create 17 overlapping pairs; it now creates none.FitBoxesToInkreads only the bands of the page its boxes can reach: about 25 ms on a 300 DPI page, down from 48 ms.
0.5.0 — 6 October 2026
Defaults that changed
These change output with no type error to warn you. Each has a switch that restores the previous behaviour, which is also Python ScaleDP's:
PdfMergeImageTextkeeps visible words a mismatched text layer covers. With the newmatchText(on by default) a covered box is dropped only when the box covering it reads the same words, so an invisibleCONFIDENTIALover a picture of a name no longer erases the name.matchText: falserestores coverage-only dropping.PdfMergeImageTextde-duplicates OCR readings against each other, where a page draws one picture over another.strategy: 'union'still keeps everything.PdfMergeImageTextjoins words OCR split apart (joinFragments, on by default):F+reyaback intoFreya.PaddleRecognizerandTesseractRecognizerread wider crops.endPadding: 0.3extends each region along its line, stopping at the next region, so a handwritten capital the detector left off is read;dropNested: trueskips a region lying mostly inside a larger one.endPadding: 0, dropNested: falserestores the previous crops.- "No embedded images" is a skip, not a failure. Downstream stages carry
This page has no embedded images to read.verbatim inexceptioninstead of anUpstream stage failedwrapper with a stack trace. Test for it with the newisSkipped(exception).
On the extraction-conditions benchmark the hybrid PDF pipeline now reads 53 of 54 names in full (46 before), with no duplicated boxes.
Pattern-based PII
RegexNer is an engine-free stage that
finds DATE, PHONE, EMAIL, URL, IBAN, CREDIT_CARD, PESEL, SSN,
ZIP_CODE and COUNTRY in OCR text. Patterns match loosely and candidates are
validated strictly — Luhn, the PESEL checksum, IBAN mod 97, the calendar, the
SSN allocation rules. A failed check lowers the score rather than dropping the
match.
- OCR lookalikes (
O→0,l→1,S→5…) are repaired inside numbers without changing the text's length, so offsets still land on their boxes. libphonenumber-jsandchrono-nodeare optional peers, loaded lazily, that add phone and date readings when installed.- Entities carry
source: 'pattern', whichNerConsistencynow preserves. FuzzyRegex, the approximate regular expressions behind it, is exported in plain TypeScript; the Nodefuzzy-regexpackage is a native addon and cannot run in a browser.
Boxes that cover their ink
FitBoxesToInk grows boxes,
pixel by pixel, until they cover the ink they belong to: a capital a detector
missed, a descender, a text-layer box sized from font metrics. It never grows
into a neighbouring box, and rotated boxes are fitted along their own lines.
Also
isSkipped,SkipError,upstreamErrorandNO_EMBEDDED_IMAGESare exported from the root, for telling a page with nothing to read from a failure. The merge helperstextsAgree,dedupeReadingsandjoinWordFragmentsare exported from/pdf, and the validators (ibanValid,luhnValid,peselValid,ssnValid,dateValid,ISO_COUNTRIES) from the root.showBoxesprints a rotated box's coordinates to one decimal.
0.4.0 — 22 September 2026
A worker can hold a file
A worker can now be given a document once and referenced from then on.
ScaleDpWorkerClient.putContent(key, bytes) registers it; a row carries
contentRef: key (CONTENT_REF_COL) in place of content; dropContent(key)
releases it, and dispose clears everything.
await client.putContent('doc', bytes)
const rows = await client.transform(stages, [{ contentRef: 'doc', path: file.name }])Structured clone copies every byte of every message, so several pipelines over
one file paid for the whole file per page, per pipeline — a 20 MB, 100-page
document moved roughly 6 GB through postMessage. It also defeated the
0.3.1 document cache, which keys on Uint8Array identity; a referenced buffer
is the same object each time, so the cache now hits. Rows carrying content
directly keep working unchanged. See Workers.
0.3.2 — 22 September 2026
Fixes
- PDF stages work inside a Worker. Every PDF stage threw
ReferenceError: document is not definedin a worker as soon asconfigure()supplied asset URLs — the normal configuration — so OCR, rendering and embedded-image extraction all failed there. pdf.js was reaching fordocument.baseURIto decideuseWorkerFetch; the library now passes it explicitly, and supplies anOffscreenCanvas-basedCanvasFactoryfor the scratch canvases rendering needs.
0.3.1 — 22 September 2026
Performance
- The worker host reuses its pipeline across transforms. It used to rebuild
the whole pipeline on every
transformrequest, so a host driven one page at a time re-read the weights from IndexedDB and built a newInferenceSessionfor every page. The pipeline is now kept while the incomingStageDescriptor[]is unchanged, and dropped after a run that throws. - PDF stages share a parsed document.
PdfToDocument,PdfToImageandPdfEmbeddedImageseach parsed the file on their own, which per page meant parsing a 100-page document hundreds of times. They now go throughwithPdfDocument, which keeps the last two parsed documents. The cache is keyed on the identity of theUint8Array, not its contents, so a caller that builds a fresh array per call simply gets the previous behaviour.resetPdfDocumentsdrops every cached document; a host that tears its engine down between documents should call it.
0.3.0 — 22 September 2026
Defaults that changed
showBoxesadds a text filter above its table whenever more than one row is shown. Pass{ search: false }as its third argument for the bare table. Rows keep theirdata-box-index, so click handlers keep working.PdfToDocumentreports rotated text with its realangleand a tight box, instead of an axis-aligned envelope atangle: 0. A vertical run now reports about 60×12 atangle90 where it used to report 12×60 atangle0. Upright text is unchanged.mergeBoxeskeeps the orientation of rotated boxes, takingminAreaRectover both boxes' corners. Axis-aligned pairs are unchanged. This departs from Python ScaleDP, which still resets the angle.
Fixes
- Word splitting in
PdfToDocumentwalks each run along its real reading direction, so right-to-left, bottom-to-top and rotated runs split correctly. - A blocked IndexedDB open now warns instead of hanging silently. Another tab holding the model cache across a version change used to make model loading hang with no signal; the warning names the database and says to close or reload the other tabs.
0.2.0 — 15 September 2026
Defaults that changed
Three, all of them in output you may already be parsing:
PaddleTextRecognizer'sboxLevelnow defaults to'word', not'region'. SetboxLevel: 'region'to keep the previous output.TesseractRecognizer'sboxLevelmoved the same way.ImageDrawBoxesnow draws labels attextSize: 24, up from 12.- Every NER model in the catalogue is now served at fp16 rather than 8-bit — roughly 1.7× the download, and the reason entities are found at all.
A PDF's text layer and its scanned images
A PDF's text layer covers its vector text and nothing else, so a scanned table
pasted into a typed page is invisible to
PdfToDocument — and OCR'ing the whole
raster reads the typed half back worse than the PDF already states it. Two new
stages read both and emit one Document per page.
PdfEmbeddedImagespulls the raster images out of the PDF, one row each, in the pixels the PDF stored rather than a crop of a render.placement.effectiveResolutionreports the DPI an image is actually placed at, andminEffectiveResolutionupscales anything below it before a recognizer sees a thumbnail. Bullets, rules, stencil masks and images the text layer already covers are skipped.PdfMergeImageTextfolds each image's reading back onto the page and reduces the rows to one per page.strategydecides who wins where both sources describe the same words —text-layer-winsby default,ocr-winsfor a corpus that extracts as mojibake,unionfor comparing the two.- The whole pipeline is in the recipe A PDF's text layer and its scanned images, and in the demo as PDF text layer + scanned images.
- Every PDF stage now honours the page its row already names. Two expanding PDF stages over one file used to square the row count — five pages became twenty-five. A chain now reads each page once.
OCR and recognition
- Paddle word boxes cost nothing in text quality. The line is read whole, at full context, and the words are split out of what came back — the reverse of the old order, which recognised each word crop on its own and dropped characters doing it. One inference per line instead of one per word.
PaddleTextRecognizerreturns word boxes, like every other recognizer here, viaboxLevelandwordGapRatio.PaddleRecognizerandTesseractRecognizerboth return one box per word by default. Word boxes are measured from the ink, not estimated from the character count.spaceRecoverystays off, and its help text now says why: it inserts a space wherever the space class clears0.001, which turns small or noisy text intos e n s i t i v e.
Multi-script documents
TesseractScriptDetector, a new stage answering "what is this page written in?" before you commit to a recognition model. It writes the script, Tesseract's OSD score and the page rotation, andpresetsForScript()names the PaddleOCR presets that can read the result. An unidentified script is an emptyscriptwith an emptyexception— a blank page is an answer, not a failure.presetCollets the model follow the page.presetis fixed when the pipeline is built, which is wrong for a document whose language you do not know in advance or that changes partway through. PointpresetColat a script column and each page is read by a model that can read it;presetstays as the fallback.- The tesseract.js asset paths are configurable.
osdWorkerPath,osdCorePath,osdLangPathandosdGzipundertesseractinconfigure()— OSD was the one place in the library still pulling assets from a public CDN.detectOsd()is exported for the raw reading.
NER and PII
NerConsistency, a new engine-free stage that re-tags every occurrence of an entity the model found somewhere. A model scores each mention on its own, so the same name is tagged in the body and missed in a heading set in caps — and for redaction the occurrence you missed leaks what the others hid.scope,minLength,minScore,resolveConflictsandoverrideModelLabelsbound what propagates;Entity.sourcerecords which is which.- Four more GLiNER models:
gliner-pii-edge(~181 MB, the smallest in the catalogue),gliner-medium,gliner-medium-newsandgliner-multi. - One colour per entity group, and colours that are actually different.
Hashing a label into a hue gave
phone_numberanddatethe same colour; every group the catalogue's models emit now has a hue assigned by hand, 18° from its neighbours.visualizeNerandImageDrawBoxesagree.
Looking at results
boxOverlay()draws box outlines over an image instead of burning them in, so a highlight can change as often as a reader clicks with nothing re-encoded.showBoxestags each row with its box index;showTexttakesmaxHeight: 'none'for a host with its own scrolling panel.boxCoverage(inner, outer)joinsboxIou— how much of one box lies inside another, which is the right question when two producers describe the same words at different granularities.- The demo's boxes table is clickable, its runtime strip exposes the execution provider and thread count, and its model menus mark what is already cached in this browser.
Fixes
- Entities found in the text but drawn nowhere on the page. The character-to-box map walked boxes in detection order with a forward-only cursor, while the text was laid out in reading order — so any box found out of order was skipped and its entity came back with no geometry.
GlinerNerfound nothing at all. 8-bit quantization wrecks these DeBERTa-backed models' score calibration rather than merely blunting it: spans decoded correctly and then scored below the default threshold. The catalogue is fp16 throughout, and a unit test keeps 8-bit weights out of it.- A failing model cache no longer takes the model down with it. Every IndexedDB read and write is best-effort; a refused write warns once and the run continues with the bytes in hand. See Models and cache.
- A downloaded model is cached as a Blob, not an ArrayBuffer, which avoided a second full copy of the weights in the renderer at commit time — enough to lose the tab on a several-hundred-megabyte model.
0.1.1 — 29 August 2026
Added
PaddleRecognizer: PP-OCR recognition over the boxes a separate detector found, the PaddleOCR counterpart toTesseractRecognizer. Crops are straightened before reading and batched onto one sheet canvas, so a forty-line page costs a handful of inferences rather than forty. Only the recognition half of the preset is downloaded, and its cache keys are shared with the other Paddle stages.getPaddleRecognizer()from@stabrise/scaledp/ocr, for a recognition-only service without the pipeline.createSession()gained an opt-infallbackToWasm— WebGPU fails at session creation on a graph it has no kernel for, PP-OCR's recognition model among them. Off by default, so a genuinely misconfigured provider still fails loudly.validatePreset()andisKnownPreset()are exported from theocrsubpath;rotate180()from the core image helpers.
Breaking
- The two PaddleOCR defaults constants were renamed to match their stage
classes:
PADDLE_DETECTOR_DEFAULTS→PADDLE_TEXT_DETECTOR_DEFAULTS, andPADDLE_RECOGNIZER_DEFAULTS→PADDLE_TEXT_RECOGNIZER_DEFAULTS.PADDLE_RECOGNIZER_DEFAULTSstill exists but now belongs to the newPaddleRecognizerand has a different shape, so the old import fails to type-check rather than misbehaving quietly. No stagetypestring changed — saved pipeline descriptors are unaffected.
0.1.0 — 29 August 2026
Initial release: the pipeline, schemas and geometry, PDF reading via pdf.js, PaddleOCR and Tesseract, DBNet and YOLO detection, GLiNER entity recognition, and the worker host.