Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Support early stopping in huggingface LLM wrapper #429

Open
1 task done
chanind opened this issue Feb 16, 2025 · 0 comments · May be fixed by #430
Open
1 task done

[Proposal] Support early stopping in huggingface LLM wrapper #429

chanind opened this issue Feb 16, 2025 · 0 comments · May be fixed by #430

Comments

@chanind
Copy link
Collaborator

chanind commented Feb 16, 2025

Proposal

We currently have a TODO in load_model.py#L98 to support the stop_at_layer param in HookedProxyLM for huggingface models. Adding support for this will save compute when we're just trying to extract LLM activations at a specific layer, since there's no need to extract activations at later layers.

An example of how to do this is in baukit/nethook.py . We just need to throw an exception after we process the layer we care about to stop processing, and then catch that exception before returning to the user.

Checklist

  • I have checked that there is no similar issue in the repo (required)
@chanind chanind linked a pull request Feb 17, 2025 that will close this issue
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant