Repository navigation
Conversation
jlarson4
left a comment
There was a problem hiding this comment.
Thanks for this @hojaeklee! Always appreciate little maintenance items like this. I have a couple of related areas you could also take a look at, if you're willing. See comments below.
|
|
||
| Args: | ||
| sequence: String being fed to the model. | ||
| tokens: Tokens being fed to the model. |
There was a problem hiding this comment.
demos/Head_Detector_Demo.ipynb carries a copy of these functions, so the same stale entries are still there. Would you be willing to fix that version as well?
There was a problem hiding this comment.
Yep, I edited the JSON entry directly so avoid messy output, hope that is okay! Done in f3896ae.
| @@ -213,7 +211,7 @@ def get_duplicate_token_head_detection_pattern( | |||
| """Outputs a detection score for [duplicate token heads](https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J#z=2UkvedzOnghL5UHUgVhROxeo). | |||
There was a problem hiding this comment.
The summary states that the function outputs a detection score, but what comes back is the pattern. Would you be willing to update this line as well?
| Args: | ||
| activation_name: | ||
| The name of the activation to be stacked | ||
| layer: |
There was a problem hiding this comment.
This layer entry is also out of date, would you be willing to update it as well?
Description
Docstring-only fixes in
head_detector.pyandActivationCache.pywhere the documented parameters had drifted from the actual signatures, plus two typos. No code changes.transformer_lens/head_detector.pydetect_head: remove thehead_nameentry; the parameter no longer exists (a head name is now passed viadetection_pattern).get_duplicate_token_head_detection_pattern,get_induction_head_detection_pattern: the Args documentedsequence: String …, but the parameter istokens: torch.Tensor. Nowtokens: Tokens being fed to the model., matchingget_previous_token_head_detection_pattern.compute_head_attention_similarity_score:exclude_bcurrent_token→exclude_current_token.transformer_lens/ActivationCache.pystack_activation: remove theincl_remainderentry (copied fromstack_head_results; not in this signature).get_neuron_results: "results of for neurons" → "results for neurons".Type of change
Checklist: