-
Notifications
You must be signed in to change notification settings - Fork 978
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python SDK] REST endpoints mishandle headers #3022
Labels
Comments
9 tasks
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
/kind bug
What steps did you take and what happened:
Headers are not handled and returned properly in v1/v2 endpoints in the recent master branch:
kserve/python/kserve/kserve/protocol/rest/v1_endpoints.py
Lines 74 to 80 in f9e9a07
if the response is a
dict
, then the headers will be completely discarded.in v2 endpoints, the headers are handled properly:
kserve/python/kserve/kserve/protocol/rest/v2_endpoints.py
Lines 146 to 147 in 97bada5
dataplane.encode
completely discarded the response headers from the modelkserve/python/kserve/kserve/protocol/dataplane.py
Lines 259 to 260 in 97bada5
So any custom headers returned back from the model will be completely lost for both v1/v2 endpoints.
kserve/python/kserve/kserve/protocol/rest/v1_endpoints.py
Lines 76 to 85 in 97bada5
What did you expect to happen:
Custom headers should be passed through and returned properly, so things won't break if users upgrade their Python SDK to 0.11.x
What's the InferenceService yaml:
[To help us debug please run
kubectl get isvc $name -n $namespace -oyaml
and paste the output]Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
maybe related issue: #3012
Environment:
kubectl version
):/etc/os-release
):The text was updated successfully, but these errors were encountered: