Skip to content

fix(streams): keep the batch that reveals a reorg, detect reorgs on resume, make InMemoryStateStore.commit idempotent - #35

Draft
benface wants to merge 2 commits into
mainfrom
benface/reorg-fixes
Draft

benface wants to merge 2 commits into
mainfrom
benface/reorg-fixes

Conversation

@benface

@benface benface commented Nov 2, 2026

Copy link
Copy Markdown

ProtocolStream lost the batch that reveals a reorg and missed reorgs that happened while it was stopped, and InMemoryStateStore.commit wasn't idempotent. These change ProtocolStream to match Amp's Rust client, and make the store skip IDs it already has.

  • The batch that reveals a reorg never reached the consumer. ProtocolStream now emits the Reorg, then the batch (as Data, or as Watermark when its ranges are complete and it has no rows).
  • Resuming after a reorg emitted the re-sent blocks as plain Data. The stream now starts from resumeWatermark's ranges, so it emits a Reorg first.
  • InMemoryStateStore.commit saved the same watermarks twice when applied twice. It now skips entries whose ID is already in the buffer.

You might want to look at:

  • Validation on resume. The first batch after a resume is now checked against the watermark (hash chain, gaps, networks), like in Rust. One that doesn't chain onto it now fails with ProtocolValidationError instead of passing silently.
  • PartialReorgError on resume. TransactionalStream now sees reorgs on resume, and fails with PartialReorgError when one starts inside the last watermark's range (watermark [35-37], 2-block reorg), because Amp re-sends from the first reorged block. Rust has the same check, so I left it alone.
  • Avoiding the double commit instead. You might prefer having the state actor not pass the same insert twice, on top of or instead of the store's check.

Checks:

  • pnpm lint, pnpm check:tsgo, pnpm test (237 passed, 4 new) and pnpm circular pass.
  • The new tests cover both ProtocolStream fixes with a mock ArrowFlight, a plain resume with no reorg, and the double commit. The reorg and double commit ones fail without the fix.
  • Against ampd v0.0.42 over Anvil, a 3-block reorg mid-stream gives Reorg, Data rows=4, Watermark (before: Reorg, Watermark).
  • Same setup, stopping, reorging 4 blocks and resuming gives Reorg, Data rows=5, Watermark (before: Data rows=5). TransactionalStream gives Undo, then Data after a 3-block reorg (before: Data).

Fixes #34

🤖 Generated with Claude Code

benface and others added 2 commits November 2, 2026 17:09
… reorgs on resume

ProtocolStream replaced the batch that revealed a reorg with the Reorg message,
so its rows (the replacement blocks) never reached the consumer. It now emits
the Reorg, then the batch as Data or Watermark, like Amp's Rust client.

It also started from empty ranges when given a resumeWatermark, so a reorg that
happened while the stream was stopped came through as plain Data. It now starts
from the resume watermark's ranges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StateStore.commit must be safe to call twice with the same data, but
InMemoryStateStore appended the inserted watermarks every time. It now skips
watermarks whose ID is already in the buffer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reorg bugs in ProtocolStream and the transactional stream

1 participant