Skip to content

perf(canon): Reconcile parses the documents seven times, not twelve - #2201

Draft
ako wants to merge 1 commit into
mainfrom
claude/trusting-wright-cqa3bb
Draft

ako wants to merge 1 commit into
mainfrom
claude/trusting-wright-cqa3bb

Conversation

@ako

@ako ako commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Refs #2160 — the reconcile, the last of the three parts of the settled page re-run.

This is in the identity code (ADR-0008), so the change is deliberately narrow and the old path is kept for comparison.

What was slow

canon.Reconcile was 23 s of a 77 s settled re-run of 600 pages, about 38 ms for each rebuilt page compared with its stored unit.

Reconcile is a row of steps from bytes to bytes, and each parsed both documents for itself: twelve whole-document parses for one write.

Step Parses before After
CarryTranslations 2 2
TransplantIDs: pairing 2 1, plus the shared stored parse
TransplantIDs: checking the substitution (ids before and after) 2 0
CarryPersistentIDs 2 0 for a document without the property
CarryUnexpressed 2 2
Equal 2 1, plus the shared stored parse

The change

  • One parse of the stored document for the steps that only read it: the transplant's pairing and the final comparison (parseStored, transplantIDs, equalToStored).
  • The substitution check reads ids off the raw bytes (elementIDSetOf) instead of parsing the document before and after it patches.
  • The PersistentId carry is skipped for a document that does not contain the bytes of a binary PersistentId element. Only a workflow's elements have one, and a pair needs it on the new side.

What was deliberately not done

Parsing each document once and handing the trees to every step. The translation carry and the unexpressed carry copy parts of the stored tree into the new one, so a shared stored tree could be changed under the steps after them, and the new document changes between steps. Those two carries keep a parse of their own.

Evidence

  • Byte for byte against the old Reconcile, kept verbatim as reconcileInPasses: every PedApp unit against itself, against a rebuild with new ids throughout, against rebuilds with properties left out, and the reverse, with and without ContentsOwnTranslations — 2,214 comparisons, same bytes and same verdict. The test asserts it covered unchanged and changed verdicts and reconciles that rewrote the contents.
  • The workflow path: the fixture has no workflow, so a workflow-shaped pair is compared separately and must come back unchanged with the stored PersistentId.
  • elementIDSetOf against parsing on every unit, as stored, rebuilt and thinned.
  • Controls, each run and seen to fail: not reading ids under arrays fails the id test (4 ids read where parsing reads 801); always skipping the PersistentId carry fails three tests. A first attempt at the second control did not compile and printed nothing, and was redone.
  • The existing canon tests pass unchanged, as do the executor's reconcile, idempotence, translation, GUID and workflow tests.
  • End to end: a script that changes all 600 pages, run on two copies with the previous binary and this one, leaves 1,719 units that are byte-identical between the two; 600 of them differ from before the run. Script output is identical.
600 pages Before After
Settled re-run 74.3 s 61.6 s
Run that changes every page 72.1 s 65.6 s

Same machine, interleaved, one run each. The microflow and entity re-runs are unchanged within noise (6.9 s / 7.0 s and 5.7 s / 5.5 s).

Not done

  • The remaining seven parses.
  • The new comparison test takes about 27 s.

Generated by Claude Code

Reconcile is a row of steps from bytes to bytes, and each parsed both
documents for itself: twelve whole-document parses for one write, 23 s
of a 77 s settled re-run of 600 pages. Five of them served steps that
only read the stored document or needed no tree at all.

- The transplant's pairing and the final comparison share one parse of
  the stored document. The carries between them copy parts of the
  stored tree into the new one, so they keep a parse of their own.
- The id substitution reads the document's ids off the raw bytes
  instead of parsing it before and after it patches.
- The PersistentId carry is skipped for a document that does not hold
  the bytes of a binary PersistentId element; only a workflow does.

The old Reconcile is kept as reconcileInPasses and compared byte for
byte, with the verdict, on every PedApp unit against itself, a rebuild
with new ids, rebuilds with properties left out, and the reverse:
2,214 comparisons. A workflow-shaped pair covers the PersistentId path
the fixture does not. Not reading ids under arrays, and always skipping
the carry, each fail the tests.

A script changing all 600 pages, run with the previous binary and this
one, leaves 1,719 byte-identical units. Settled page re-run: 74.3 s ->
61.6 s.

Refs #2160

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oK82u4HRCKfX2BBvhgsNm

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants