Skip to content

fix(bridge): route legacy TL-format repos from boot_transformers to boot_tl_legacy - #2029

Open
pablocs116 wants to merge 1 commit into
TransformerLensOrg:devfrom
pablocs116:fix/route-legacy-repos-to-boot-tl-legacy-2025
Open

pablocs116 wants to merge 1 commit into
TransformerLensOrg:devfrom
pablocs116:fix/route-legacy-repos-to-boot-tl-legacy-2025

Conversation

@pablocs116

Copy link
Copy Markdown
Contributor

Closes #2025

Problem

TransformerBridge.boot_transformers couldn't load legacy TransformerLens-format repos (NeelNanda/*, ArthurConmy/*, Baidicoot/*), even though they and their aliases (gelu-1l, solu-1l, attn-only-2l, ...) are in the model registry. The alias resolved to the repo ID, and then AutoConfig failed with Unrecognized model ... Should have a model_type key. boot_tl_legacy loads these repos, but nothing pointed to it.

Fix

In sources/transformers/source.py::boot, after alias resolution, a name under TL_LEGACY_PREFIXES is handed to boot_tl_legacy:

  • Warning on every auto-route, as requested in the issue. It names the exact TransformerBridge.boot_tl_legacy('<repo>') call to use instead.
  • Forwarded: device, dtype (None → float32, the same as a fresh boot_transformers load), tokenizer, checkpoint_index / checkpoint_value.
  • Unsupported options raise a ValueError naming boot_tl_legacy and the offending arguments, instead of being silently dropped: hf_config_overrides, load_weights=False, model_class, n_ctx, revision, code_revision, device_map, n_devices, max_memory, offload_folder. trust_remote_code is ignored, since .pth repos run no remote code.
  • hf_model= skips the routing. A pre-loaded model is HF-format whatever its repo name, so boot keeps reading its config as before.

Docs: the model_name docstring and migrating_to_v4.md now describe the routing and recommend calling boot_tl_legacy directly.

Tests

  • tests/unit/model_bridge/test_boot_tl_legacy_routing.py (new, mocked, no network, 15 tests): alias and repo ID both route, with the warning; supported options are forwarded; each unsupported option raises without loading; HF-format names and hf_model= aren't routed.
  • tests/integration/model_bridge/test_tl_legacy_loader.py: boot_transformers("solu-1l") gives logits identical to boot_tl_legacy("NeelNanda/SoLU_1L512W_C4_Code").
  • Without the fix, the new integration test fails with the original ValueError, and the unit tests error.

Ran locally (CPU, Linux/WSL2):

  • New unit tests + all of test_tl_legacy_loader.py: 24 passed
  • tests/unit/model_bridge/test_boot_*.py + tests/unit/model_bridge/sources: 144 passed, 2 skipped (optional inspect_ai not installed)
  • make format, uv run mypy .: clean

Downstream: SAELens works around this today (decoderesearch/SAELens#789) and can drop that once this ships.

🤖 Generated with Claude Code

…oot_tl_legacy

NeelNanda/*, ArthurConmy/* and Baidicoot/* repos (and aliases like
gelu-1l, solu-1l) have no HF model_type, so boot_transformers failed in
AutoConfig. Hand them to boot_tl_legacy with a warning pointing at the
direct call, forward device/dtype/tokenizer/checkpoint args, and raise on
HF loading options boot_tl_legacy can't honor. A pre-loaded hf_model
skips the routing.

Closes TransformerLensOrg#2025

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@jlarson4 jlarson4 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together @pablocs116, just a couple small notes, and a small bug I did not think about when reviewing your original proposal. Take a look and let me know what you think.

)
model_name = official_name
# A pre-loaded hf_model is HF-format by construction, whatever its repo name.
if hf_model is None and model_name.startswith(TL_LEGACY_PREFIXES):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something not previously taken into account: these namespaces also host regular HF-format models, e.g. Baidicoot/Llama-3-8B-Instruct-LAT. Those load through boot_transformers, but with this change would fail with KeyError: 'd_model'. Can we route on something narrower than the namespace?

device="cpu",
dtype=torch.float16,
tokenizer=tokenizer,
checkpoint_index=3,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every case here leaves checkpoint_value as None, so the tests still pass if the routing stops forwarding it. Could one case pass a real value?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants