Timings
execution_time is per run, row_time is per row, and the two deliberately do not add up.
Every row carries both.
execution_time — per run
rows[0].execution_time
// { stages: { PdfToImage: 412, PaddleTextRecognizer: 1830 }, total: 2244 }One number per stage, covering every row that stage processed, and the same
object on every row. This is what Python's ScaleDP records, and under
transform it keeps that meaning exactly. It cannot say how long page 7 took,
because each stage's number covers every page at once.
A stage's number includes its init(): the clock starts before the model is
loaded, so a first run's download shows up here.
Under Pipeline.run, which streams page by page, each stage's number is its
busy time summed across pages, and total is wall-clock time. With several
pages in flight the stages can add up to more than total. See
Streaming.
Two stages of the same class disambiguate by position:
// { PdfToImage: 412, ImageDrawBoxes: 22, 'ImageDrawBoxes#3': 19 }row_time — per row
for (const row of rows) console.log(row.page, row.row_time.total)
// 0 259
// 1 190
// 2 192Time spent on that row alone. This is an addition to the Python model, not a
change to it — the question "which page was slow" has no answer in
execution_time, so the answer lives beside it.
Why the row totals do not sum to execution_time.total
The difference is mostly stage.init(): downloading models and creating
sessions, once for the whole run rather than per page. It is charged to the
stage in execution_time but to no row, since row_time counts only the work
done on that row. A pipeline whose first run downloads 333 MB of GLiNER weights
will show a total far larger than the sum of its rows, and every subsequent
run on a warm cache will show them close.
Time a page spends waiting for a busy stage is in neither: stage:end reports
it separately as waitMs.
That is the number to watch when deciding whether to pre-warm a model.
Expanding stages split evenly
PdfToImage renders five pages in one call and returns five rows. Its elapsed
time is divided by five and charged to each. No finer attribution exists from
outside the stage, so a slow page 3 in a five-page render is invisible in
row_time — it shows up in execution_time as a large PdfToImage.
Stages that do not expand attribute exactly.
Live progress
transform also takes an onStage callback, which fires as each stage
completes rather than at the end:
await pipeline.transform(file, {
onStage: (name, ms, rows) => console.log(`${name}: ${ms}ms over ${rows} rows`),
})Under Pipeline.run stages interleave across pages, so onStage fires once
per stage when the run ends; per-page timing comes from the stage:end events
instead (ms, and waitMs).
The worker client exposes the same callback, so a pipeline running off the main thread reports the same shape. See Workers.
Model downloads are a separate channel — configure({ onProgress }), or the
run's model:progress events, attributed to a stage — because they happen
inside init() and are measured in bytes rather than stages.