Skip to content

Test: a vector search finds the vector committed in the call before, from the same task and from another, on both wheels (refs #306) - #373

Merged
tae898 merged 1 commit into
mainfrom
feat/306-asyncio-facade
Nov 12, 2026
Merged

tae898 merged 1 commit into
mainfrom
feat/306-asyncio-facade

Conversation

@tae898

@tae898 tae898 commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Item 2 of #306: a read-your-writes test for vector search, on both wheels. tests/test_vector_read_your_writes.py starts with 300 vectors and an index, then makes 200 committed inserts, each followed by a search for the vector just written. Test one does it in the writing task, with 4 tasks writing at once. Test two has one task write and 3 reader tasks search for each id after the writer announces it on a queue (after the commit returned). Every call runs through asyncio.to_thread, as the concurrency page says, so an insert and the next search of one task can run on different threads. The 200 inserts cross the rebuild threshold of 100 pending changes, so the searches meet the delta buffer first and the rebuilt graph later. ef_search covers the corpus, so a miss would mean the write was not seen. The test uses no Java object and runs on arcadedb and arcadedb-jvm.

The vectors page now says what an agent may rely on: a search finds every vector whose transaction committed before it started, from the same task or another one, also while the graph rebuilds. The concurrency page gets the asyncio anchor that sentence links to.

This is not the asyncio facade. That part of the issue waits for a decision on its names and on cancellation, see my comment on #306.

Related issues

Refs #306 (item 2 only, so no closing keyword).

Docs

how-to/vectors.adoc ("Agents that write as they go") and how-to/concurrency.adoc (the asyncio anchor).

Checklist

  • A regression test fails on the old code and passes on the new: this is a test that pins existing behavior, so it passes on main. It would fail if a committed vector were missed by the next search.
  • The test suite that the change can affect passes: CI runs it on both wheels.
  • uvx pre-commit run --files <changed files> passes (black reformatted the test, now committed).

🤖 Generated with Claude Code

@tae898

tae898 commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

The red "Test arcadedb" legs are not from this PR: only test_import_database_csv_documents_with_quoted_parallel_setting and test_import_database_csv_graph_vertices_and_edges_with_quoted_parallel fail, because arcadedb#11367 was fixed upstream and the two tests still expect the refusal. I filed it as #376 and will not merge this PR while the legs are red.

…in the same asyncio task and in another one, on both wheels (refs #306)

An agent that remembers things inserts a few vectors per turn and searches right after, from one task or several. The new test runs 200
committed inserts per test, each followed by a search, while the graph rebuilds in the background (the corpus starts at 300 vectors, so the
rebuild threshold of 100 pending changes is crossed): once in the task that wrote, once in readers that get the id from a queue after the commit.
Every call runs in a worker thread through asyncio.to_thread, as the concurrency page says, so an insert and the next search of one task can
run on different threads. ef_search covers the corpus, so a miss would mean the write was not seen and not an approximate walk. The file uses
no Java object and runs on the JVM build and the native build.

The vectors page says what the agent may rely on (committed writes are found by the next search, from any task), and the concurrency page
gets the anchor the new sentence links to. Item 2 of #306 only: the asyncio facade waits for a decision on its names, see the issue.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 force-pushed the feat/306-asyncio-facade branch from 14d1087 to 8ae3075 Compare November 12, 2026 01:53
@tae898
tae898 merged commit 7a8abae into main Nov 12, 2026
43 checks passed
@tae898
tae898 deleted the feat/306-asyncio-facade branch November 12, 2026 03:17
@github-actions github-actions Bot added this to the 26.12.1 milestone Nov 12, 2026
tae898 added a commit that referenced this pull request Nov 12, 2026
…and find_nearest from async def code (refs #306) (#379)

* arcadedb.aio.AsyncDatabase: await query, command, run_in_transaction and find_nearest from async def code (refs #306)

An asyncio facade over the blocking calls of an open Database, on a bounded thread pool of its own (max_workers, default 8). Decided on #306:
a transaction belongs to one thread and two awaits of one task can run on two workers, so there is no async with transaction;
run_in_transaction(fn) runs the plain function fn, begin to commit and every retry, on one worker. Cancellation is not exposed: cancelling
the awaiting task abandons the result, a call that started finishes, a call still waiting for a worker does not run, and close() waits for
the calls that were submitted (it does not close the Database). The docstring and the docs page say how AsyncDatabase differs from the
engine's AsyncExecutor.

tests/test_aio.py pins, on both wheels: rows and errors reach the awaiting task, run_in_transaction commits on one worker and rolls back on
an exception, 200 tasks write and read at once without a lost update, the pool runs max_workers calls at once and never more, close() waits
and refuses new calls, a cancelled await abandons the result. tests/test_docs_asyncio_page.py runs the blocks of the new page how-to/asyncio.adoc.
Items 3 and 4 of #306 still need measurements.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Read-your-writes tests also through AsyncDatabase; the concurrency and vectors pages point to the facade (refs #306)

The two read-your-writes tests of #373 (the writing task, and other tasks that search after the commit) run again through
arcadedb.aio.AsyncDatabase: an insert is run_in_transaction and a search is find_nearest, on both wheels. The asyncio section of the
concurrency page and the vectors page name the facade next to asyncio.to_thread.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Test helper: a nosec note for the fixed type name (Bandit B608)

The count helper builds its SQL from a fixed type name of the tests, backtick-quoted, with no values. CI's Bandit run flags it at low confidence; the repo's convention is a nosec with the reason.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant