Repository navigation
Hold queues by weak reference in SequentialOrderManager - #2426
Open
vlad-perevezentsev wants to merge 3 commits into
Open
vlad-perevezentsev wants to merge 3 commits into
vlad-perevezentsev wants to merge 3 commits into
Conversation
vlad-perevezentsev
requested review from
antonwolfy and
ndgrigorian
as code owners
November 11, 2026 18:15
Add a __weakref__ slot to SyclQueue. It follows the vtable pointer, so extensions built against an earlier dpctl find the existing fields and the vtable where they expect them; only the object grows, which they report as a size change at import. SyclQueue gets an empty __dealloc__, so that Cython gives it a deallocator of its own, which clears the weak references. Cython does not count __weakref__ as a field that needs one and would reuse the one of _SyclQueue, leaving weak references to freed memory.
An order manager now holds its sycl::queue in a SyclQueue object of its own, rather than the queue it was looked up by, so that the values of the order manager map do not keep its keys alive. _InOrderManager used to hold its key. The in-order test that counts waits on the queue now makes the manager itself, as the managers of the map wait on their copy.
The order manager map kept every queue it was asked for alive until clear(), so a program creating queues in a loop ran out of device or system memory (up to ~15 MB per live queue on Level Zero v2). Queues are now map keys by weak reference. A queue that is gone is only recorded by its weak reference callback; the owning thread drops its order manager on a later lookup once the work recorded in it is complete, without waiting, as the queue may be dropped on any thread, including inside a host task that the order manager would wait on. Until then, and while anything else holds it, the order manager is handed to an equal queue, so that its work stays ordered. Whether an in-order manager is idle is decided from the events recorded in it only: Level Zero v2 reports the last event of an in-order queue as submitted until it is waited on.
|
View rendered docs @ https://intelpython.github.io/dpctl/pulls/2426/index.html |
Collaborator
|
coverage: 75.419% (+0.2%) from 75.173% — order-manager-weak-keys into master |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SequentialOrderManagerkeeps an order manager per queue in a thread-local dict keyed by the queue. Onlyclear()removes entries, so every queue that ran adpnpoperation stays alive, along with its Level Zero command lists and event pools.On Level Zero v2 this costs up to ~15 MB per queue, and a program that keeps creating queues runs out of memory.
On Lunar Lake and Battlemage this often made
dpnpCI fail withUR_RESULT_ERROR_DEVICE_LOST.dpnpCI passes only because itsconftest.pycallsclear()after every test file.This PR makes the map hold queues by weak reference:
SyclQueuesupports weak references. The new slot follows the vtable pointer, so extensions built against an earlierdpctlkeep working.SyclQueuealso gets an empty__dealloc__. Without it, Cython reuses_SyclQueue's deallocator, which does not clear weak references.dpnphost_task, which would then wait for itself.sycl::queue) looked up later gets the same order manager while that manager still has work or is still referenced, so ordering is kept.A lookup costs ~30 ns more for a known queue and ~4 us more for a new one. Creating a queue and running its first operation takes ~1 ms.
Compatibility:
dpnpbuilt against an earlierdpctlworks, but warns at import thatSyclQueuesize changed and fails to import with-W error::RuntimeWarning.dpnpshould be rebuilt against this version.Testing: the
dpctltest suite passes (3870 tests, 10 of them new) on Max 1100 with Level Zero v1 and v2 and on OpenCL CPU, and also passes on free-threaded Python 3.14t. Peak memory of thedpnptest suites, with all tests passing in every run:master,clear()per file (CI)clear()dpnp/tests/tensordpnp/testsdpnp/tests/tensordpnp/testsWithout
clear(),masterreaches 17.6 GB host and 17.3 GB device memory indpnp/tests/tensoron Max 1100, and is killed at 20 GB on Lunar Lake.