Repository navigation
Conversation
… reorgs on resume ProtocolStream replaced the batch that revealed a reorg with the Reorg message, so its rows (the replacement blocks) never reached the consumer. It now emits the Reorg, then the batch as Data or Watermark, like Amp's Rust client. It also started from empty ranges when given a resumeWatermark, so a reorg that happened while the stream was stopped came through as plain Data. It now starts from the resume watermark's ranges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StateStore.commit must be safe to call twice with the same data, but InMemoryStateStore appended the inserted watermarks every time. It now skips watermarks whose ID is already in the buffer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ProtocolStreamlost the batch that reveals a reorg and missed reorgs that happened while it was stopped, andInMemoryStateStore.commitwasn't idempotent. These changeProtocolStreamto match Amp's Rust client, and make the store skip IDs it already has.ProtocolStreamnow emits theReorg, then the batch (asData, or asWatermarkwhen its ranges are complete and it has no rows).Data. The stream now starts fromresumeWatermark's ranges, so it emits aReorgfirst.InMemoryStateStore.commitsaved the same watermarks twice when applied twice. It now skips entries whose ID is already in the buffer.You might want to look at:
ProtocolValidationErrorinstead of passing silently.PartialReorgErroron resume.TransactionalStreamnow sees reorgs on resume, and fails withPartialReorgErrorwhen one starts inside the last watermark's range (watermark[35-37], 2-block reorg), because Amp re-sends from the first reorged block. Rust has the same check, so I left it alone.Checks:
pnpm lint,pnpm check:tsgo,pnpm test(237 passed, 4 new) andpnpm circularpass.ProtocolStreamfixes with a mockArrowFlight, a plain resume with no reorg, and the double commit. The reorg and double commit ones fail without the fix.Reorg,Data rows=4,Watermark(before:Reorg,Watermark).Reorg,Data rows=5,Watermark(before:Data rows=5).TransactionalStreamgivesUndo, thenDataafter a 3-block reorg (before:Data).Fixes #34
🤖 Generated with Claude Code