Skip to content

Add characterization tests for async and legacy network sync - #415

Open
bjester wants to merge 3 commits into
sandboxing-defensefrom
net-characterization-363
Open

bjester wants to merge 3 commits into
sandboxing-defensefrom
net-characterization-363

Conversation

@bjester

@bjester bjester commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Adds characterization tests that pin what a full network push and pull send and store, for the sync layer refactors in M2–M4. They cover both the async flow (ASYNC_OPERATIONS) and the legacy flow used with servers before 0.6.0
  • A spy on SessionWrapper.request records each request and passes the call through to the real method. Per request, it records the method, path, query params, body encoding (json/gzip), body keys, the values of the stable body fields (filter, push, records_total, records_transferred, transfer_stage), the record count of each pushed chunk, and is_retryable. Generated IDs are normalized to <id>
  • Asserts the final local transfer session fields, the Buffer, Store and app rows on both sides, and the byte counts the server stores
  • Fails a sync that sends more than 100 requests or runs longer than 120s, so a stalled sync fails instead of hanging the test run
  • For the legacy tests, the live server runs with MORANGO_TEST_LEGACY_SERVER, which turns off ASYNC_OPERATIONS and FSIC_V2_FORMAT in server2_settings.py. LiveServer and BasePushPullClientTestCase take extra server environment variables for that
  • Extracts the live server harness from test_syncsession.py into integration/helpers.py as BasePushPullClientTestCase, so the new tests reuse it without re-running PushPullClientTestCase's tests

Reviewer guidance

  • The recording goes beyond the plan's (method, path, body keys, transfer_stage) tuple. Mutation checks showed the narrower tuple let these through, and the expanded recording catches each: dropping is_retryable, sending buffer chunks as JSON instead of gzip, un-flipping the byte counts in the pull PATCH, and adding a query param to the buffer GET
  • The legacy tests emulate an older server, and the client keeps its full capabilities, as a current Kolibri does when it syncs with an older one. Tests that pin today's legacy network sync behavior #364 proposed reducing the client capabilities instead, but that fails today on the server for both push and pull, which is now tracked in Clients that disallow async operations cannot sync with servers that allow them #416
  • The legacy pull moves 11 records, not 10. The client creates the transfer session before it serializes, so it sends an empty client_fsic, and the legacy server queues the user from the initial pull again. This is today's behavior, pinned as-is
  • URL paths are literal strings on purpose, not reverse() or the api_urls constants. The client sends them to peers on other morango versions, so a renamed route must fail these tests
  • chunk_size is set in the test class rather than inherited, because the expected request lists depend on it and the harness is not frozen
  • Once this merges, test_network_characterization.py must not change until M4 is complete

Issues addressed

Closes #363
Closes #364

Related: #416

AI usage

Claude Code implemented the tests from the issues and the implementation plan. I had a separate Claude subagent review the async tests against the issue and spec. Its mutation checks showed regressions that the plan's narrower request tuple missed, so I had the recording strengthened, and directed extracting the shared harness into integration/helpers.py. While implementing the legacy tests, Claude found that the approach in #364 fails against the current code, so we switched to emulating a legacy server and filed #416. Claude ran the integration suite and the mutation checks to verify the changes.

🤖 Generated with Claude Code

bjester and others added 2 commits November 5, 2026 13:02
The second server, second_environment and the sync session setup move
from test_syncsession.py into integration/helpers.py, with the setup in
a new BasePushPullClientTestCase. PushPullClientTestCase extends it and
keeps its tests and the activity timestamp checks they use.

Other test modules can now reuse the harness without importing
PushPullClientTestCase, which would collect and run its tests again.

Refs #363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pins what a full push and a full pull with ASYNC_OPERATIONS send and
store, so the refactors of the sync layer can show they change neither.

A spy on SessionWrapper.request records each request, then passes the
call through to the real method. For each request it records the
method, path, query params, body encoding, body keys, the values of
the body fields that are stable between runs, the record count of each
pushed chunk, and the is_retryable flag. Generated IDs are normalized
to <id>. The tests compare the recording with fixed expected lists for
every transfer stage.

The tests also assert the final local transfer session, the Buffer,
Store and app rows on both sides, and the byte counts the server
stores. The spy fails a sync that sends more than 100 requests or runs
longer than 120 seconds, so a stalled sync fails instead of hanging.

Closes #363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bjester
bjester added this pull request to stack #412 November 5, 2026 21:41
@bjester bjester changed the title Net characterization 363 Add characterization tests for async network push and pull Nov 5, 2026
Pins what a full push and a full pull send and store when the server
lacks ASYNC_OPERATIONS, so the refactors of the LegacyNetwork*
operations can show they change neither.

The tests run against a live server that emulates a morango version
before 0.6.0: the MORANGO_TEST_LEGACY_SERVER variable turns off both
ASYNC_OPERATIONS and FSIC_V2_FORMAT on the server, and LiveServer now
takes extra environment variables for that. The client keeps its full
set of capabilities, as a current Kolibri does when it syncs with an
older one. Reducing only the client capabilities, as #364 first
proposed, fails today on the server for both push and pull (#416).

The expected lists show that the client creates the transfer session
and sends no stage updates for serializing, queuing, dequeuing, or
deserializing. The legacy pull moves 11 records instead of 10: the
client creates the transfer session before it serializes, so it sends
an empty client_fsic, and the server queues the user from the initial
pull again.

Closes #364

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bjester bjester changed the title Add characterization tests for async network push and pull Add characterization tests for async and legacy network sync Nov 5, 2026
@bjester
bjester marked this pull request as ready for review November 6, 2026 18:34
@bjester
bjester requested review from a team and leq-bot November 6, 2026 18:34
@bjester
bjester removed this pull request from stack #412 November 7, 2026 00:24
@bjester
bjester added this pull request to stack #418 November 7, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tests that pin today's legacy network sync behavior Tests that pin today's async network push and pull behavior

1 participant