Repository navigation
Conversation
jlarson4
left a comment
There was a problem hiding this comment.
Thanks for putting together this bug fix! I do have one concern about the .clone() we are using in _fire_post_hoc_hook. Please take a look at my comment below and let me know if you have any questions
| # Snapshot only active hooks: inference tensors have no mutation counter, | ||
| # and an in-place edit also changes every alias used for comparison. | ||
| original = ( | ||
| tensor.detach().clone() if hook._forward_hooks or hook._forward_pre_hooks else tensor |
There was a problem hiding this comment.
If we allowed this change, every hooked forward would copy and compare the full pattern, a plain run_with_cache would pay for it on every delegating attention layer. This would be an incredible loss of speed for the tool. We will need to find an alternative condition to check for solving this problem. Could the snapshot be limited to the cases that need it, so read-only hooks stay free?
There was a problem hiding this comment.
Agreed, the cache-only path should not pay for this. Internal caching callbacks are now marked through registration and batch-removal wrappers, so plain run_with_cache and composed caching hooks skip both the snapshot and tensor comparisons. Unknown user callbacks, pre-hooks and global PyTorch hooks still use the checked path, including when mixed with caching. The marker belongs to each callback, not the HookPoint, so removing cache hooks cannot leave subsequent user hooks classified as read-only.
| assert _discard_warnings(records) == [] | ||
|
|
||
|
|
||
| def test_no_hooks_do_not_clone_tensor(bridge, monkeypatch): |
There was a problem hiding this comment.
Removing the _forward_pre_hooks half of the gate, or the dtype guard on the NaN comparison, leaves these tests green. Can we add a test for each case or drop it if it isn't needed?
There was a problem hiding this comment.
Added targeted cases for mutating and read-only pre-hooks, including inference mode, and removed the redundant dtype guard. The cache fast-path cases also forbid pattern cloning/comparison, while mixed cache/user hooks and callback removal retain the discarded-edit warning.
koriyoshi2041
left a comment
There was a problem hiding this comment.
Validated changed head 91e2ccc locally on macOS arm64: the focused post-hoc hook file passes 47/47. I also traced the cache marker through get_caching_hooks, add_caching_hooks, and wrapper installation: the fast path is limited to library-owned cache callbacks, while pre-hooks, global hooks, and any unknown forward hook retain the snapshot comparison. Removing cache hooks also leaves later user hooks on the checked path. This addresses the earlier clone concern without weakening the discarded-edit warning.
jlarson4
left a comment
There was a problem hiding this comment.
Approved! Just re-running the failed CI task, assuming it passes I will merge
Description
Fix an omission in the existing discarded-edit warning for post-hoc attention pattern/scores hooks.
These hooks run after the wrapped HF module has computed its attention output. Previously,
_fire_post_hoc_hook()checked only replacement objects against the input tensor. An in-place edit could therefore change the observed weights without influencing the attention output or triggering the warning. Returning the same tensor, an alias,None, or a clone of the already-mutated input all bypassed the check.Compare the final values against a detached pre-hook snapshot when local forward/pre-forward hooks are attached. This covers inference tensors without mutation counters, keeps read-only and equal-valued hooks silent, and treats unchanged NaNs as unchanged. The model's attention computation and hook return behavior remain unchanged.
The unhooked path and internal-cache-only hooks do not snapshot or compare the pattern. Private read-only cache markers survive registration and batch-removal wrappers. Unknown user callbacks, pre-hooks and global PyTorch hooks still require a transient snapshot; no wall-clock performance benchmark is claimed. Native attention's writable pattern seam is unchanged.
Type of change
Validation
mypy .: 391 source files passed.git diff --checkpassed.The complete suite, pretrained end-to-end interventions, GPU/MPS and performance checks were not run. Existing SWIG deprecation warnings remain.
Checklist
This restores the existing warning for previously missed writes; the warning checkbox is intentionally unchecked. Only the affected unit-test surface was run locally. No public API or documentation change is needed.