Skip to content

Fix/explicit device applied - #2026

Open
jlarson4 wants to merge 4 commits into
devfrom
fix/explicit-device-applied
Open

jlarson4 wants to merge 4 commits into
devfrom
fix/explicit-device-applied

Conversation

@jlarson4

Copy link
Copy Markdown
Collaborator

Description

  • Explicit device= moves a pre-loaded hf_model – boot_transformers(hf_model=..., device=...) recorded the request in cfg.device and left the model where it was, so cfg.device named a device the weights were not on and string inputs, which are sent to cfg.device, failed at the first forward. The model is now moved to the device
  • build_bridge_from_module applies device= too – it recorded the request without moving the supplied module, with the same mismatch. It now moves it, and cfg.device reports where the parameters are.
  • Conflicts with a device_map or quantizer raise – a model carrying hf_device_map or a quantization config keeps the placement it was loaded with. A device= naming the device its input embedding is on is accepted, and any other raises ValueError where it used to be dropped or recorded.
  • Meta parameters are reported, not moved – a meta-device model has no values to move, so the request is logged as not applied and cfg.device reports meta.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

jlarson4 and others added 4 commits November 16, 2026 14:39
Brings in #2019 (explicit dtype on the from-module path), #2020 (dtype,
device and eval() on the no-weights path), #2021 and #2023.

Conflicts resolved in _bridge_builder.py (one multi_gpu import; the docstring
covers both the dtype cast and the device move), source.py (dev's
load_weights docstring) and interchange.py (dev's random_init_bridge, since
boot now applies dtype and device itself).

Follow-ons forced by #2020: boot's no-weights device read-back and its test
are dropped, because that path now moves the model and cfg.device follows the
move; the random-init interchange test returns to dev's version.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant