Repository navigation
Test: a vector search finds the vector committed in the call before, from the same task and from another, on both wheels (refs #306) - #373
Merged
Conversation
Collaborator
Author
|
The red "Test arcadedb" legs are not from this PR: only |
2 of 3 tasks
…in the same asyncio task and in another one, on both wheels (refs #306) An agent that remembers things inserts a few vectors per turn and searches right after, from one task or several. The new test runs 200 committed inserts per test, each followed by a search, while the graph rebuilds in the background (the corpus starts at 300 vectors, so the rebuild threshold of 100 pending changes is crossed): once in the task that wrote, once in readers that get the id from a queue after the commit. Every call runs in a worker thread through asyncio.to_thread, as the concurrency page says, so an insert and the next search of one task can run on different threads. ef_search covers the corpus, so a miss would mean the write was not seen and not an approximate walk. The file uses no Java object and runs on the JVM build and the native build. The vectors page says what the agent may rely on (committed writes are found by the next search, from any task), and the concurrency page gets the anchor the new sentence links to. Item 2 of #306 only: the asyncio facade waits for a decision on its names, see the issue. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
force-pushed
the
feat/306-asyncio-facade
branch
from
November 12, 2026 01:53
14d1087 to
8ae3075
Compare
2 of 3 tasks
tae898
added a commit
that referenced
this pull request
Nov 12, 2026
…and find_nearest from async def code (refs #306) (#379) * arcadedb.aio.AsyncDatabase: await query, command, run_in_transaction and find_nearest from async def code (refs #306) An asyncio facade over the blocking calls of an open Database, on a bounded thread pool of its own (max_workers, default 8). Decided on #306: a transaction belongs to one thread and two awaits of one task can run on two workers, so there is no async with transaction; run_in_transaction(fn) runs the plain function fn, begin to commit and every retry, on one worker. Cancellation is not exposed: cancelling the awaiting task abandons the result, a call that started finishes, a call still waiting for a worker does not run, and close() waits for the calls that were submitted (it does not close the Database). The docstring and the docs page say how AsyncDatabase differs from the engine's AsyncExecutor. tests/test_aio.py pins, on both wheels: rows and errors reach the awaiting task, run_in_transaction commits on one worker and rolls back on an exception, 200 tasks write and read at once without a lost update, the pool runs max_workers calls at once and never more, close() waits and refuses new calls, a cancelled await abandons the result. tests/test_docs_asyncio_page.py runs the blocks of the new page how-to/asyncio.adoc. Items 3 and 4 of #306 still need measurements. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Read-your-writes tests also through AsyncDatabase; the concurrency and vectors pages point to the facade (refs #306) The two read-your-writes tests of #373 (the writing task, and other tasks that search after the commit) run again through arcadedb.aio.AsyncDatabase: an insert is run_in_transaction and a search is find_nearest, on both wheels. The asyncio section of the concurrency page and the vectors page name the facade next to asyncio.to_thread. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Test helper: a nosec note for the fixed type name (Bandit B608) The count helper builds its SQL from a fixed type name of the tests, backtick-quoted, with no values. CI's Bandit run flags it at low confidence; the repo's convention is a nosec with the reason. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Item 2 of #306: a read-your-writes test for vector search, on both wheels.
tests/test_vector_read_your_writes.pystarts with 300 vectors and an index, then makes 200 committed inserts, each followed by a search for the vector just written. Test one does it in the writing task, with 4 tasks writing at once. Test two has one task write and 3 reader tasks search for each id after the writer announces it on a queue (after the commit returned). Every call runs throughasyncio.to_thread, as the concurrency page says, so an insert and the next search of one task can run on different threads. The 200 inserts cross the rebuild threshold of 100 pending changes, so the searches meet the delta buffer first and the rebuilt graph later.ef_searchcovers the corpus, so a miss would mean the write was not seen. The test uses no Java object and runs onarcadedbandarcadedb-jvm.The vectors page now says what an agent may rely on: a search finds every vector whose transaction committed before it started, from the same task or another one, also while the graph rebuilds. The concurrency page gets the
asyncioanchor that sentence links to.This is not the asyncio facade. That part of the issue waits for a decision on its names and on cancellation, see my comment on #306.
Related issues
Refs #306 (item 2 only, so no closing keyword).
Docs
how-to/vectors.adoc("Agents that write as they go") andhow-to/concurrency.adoc(theasyncioanchor).Checklist
main. It would fail if a committed vector were missed by the next search.uvx pre-commit run --files <changed files>passes (black reformatted the test, now committed).🤖 Generated with Claude Code