Skip to content

fix(connection): do not drop observer events when the sink registers late - #1797

Open
evan188199-tech wants to merge 1 commit into
cloudflare:masterfrom
evan188199-tech:fix/1796-graceful-shutdown-http2-event-sink
Open

evan188199-tech wants to merge 1 commit into
cloudflare:masterfrom
evan188199-tech:fix/1796-graceful-shutdown-http2-event-sink

Conversation

@evan188199-tech

Copy link
Copy Markdown

Summary

Fixes the flaky TestGracefulShutdownHTTP2 under -race described in #1796. There were two races:

  1. Late sink registration dropped events. Observer.RegisterSink only queued the sink on a buffered channel, and dispatchEvents selected over that channel and the event channel. When both were ready, the select could pick the event first and deliver it to an empty sinks slice, silently dropping it. Sinks are now kept in a sync.RWMutex-protected slice, so registration takes effect before RegisterSink returns. Delivery iterates over a snapshot taken under the lock, so a sink cannot deadlock by re-entering the observer.
  2. The test asserted asynchronous delivery once. TestGracefulShutdownHTTP2 now waits for the event with assert.Eventually instead of checking a single time.

The public API and sendEvent semantics are unchanged. Only the stdlib sync/slices packages are added; go.mod is untouched.

Tests

  • Added TestObserverRegisterSinkThenImmediatelySendEvents as a regression test for race 1.
  • gofmt clean, go vet ./connection/ ./supervisor/ passes.
  • go test -race -count=3 ./connection/ -run TestGracefulShutdownHTTP2 and the new regression test pass.

Fixes #1796

This change was prepared with AI assistance and reviewed before submission.

…late

RegisterSink only queued the sink on a buffered channel, and
dispatchEvents selected over that channel and the event channel. When
both were ready the select could deliver events first, dropping them on
an empty sinks slice. Track sinks in a mutex-protected slice instead so
RegisterSink takes effect before it returns, and have the test wait for
asynchronous delivery with assert.Eventually instead of asserting once.

Fixes cloudflare#1796

Signed-off-by: evan188199-tech <238371241+evan188199-tech@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TestGracefulShutdownHTTP2 is flaky: the event sink can miss every event (two races in the observer and the test)

1 participant