-
Notifications
You must be signed in to change notification settings - Fork 547
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ability to retrieve the protocol response headers in InferenceClient
#2281
Comments
Hi @fxmarty, thanks for the feature request. Any suggestion on how this information should/could be returned in the current InferenceClient framework? Open to suggestions on that. |
I'm closing this issue since no new details have been provided. @fxmarty Happy to reopen it if you want. Just let me know what would be your use case for such a feature so that we can figure out what's the best way of supporting it. |
@Wauplin the feature enabled by this is tracking of the response time from TGI from the client. With stats like
For example, in https://huggingface.co/spaces/fxmarty/tgi-mi300-demo-chat/blob/main/app.py, I wanted to use |
@fxmarty thanks for the explanation! Any suggestion on how you would like this information to be returned in the current InferenceClient framework? |
As per title, it would be helpful to be able to retrieve the header as is possible with
curl --include
.Example of a useful response header:
The text was updated successfully, but these errors were encountered: