[Q] How to call predictor backend independently from trasformer #3330
-
I'm following this docs example on using triton as the predictor with a transformer. I tried a couple of things but I just can't figure out the right call address/port api signature. It always seems to route to the transformer. |
Beta Was this translation helpful? Give feedback.
Answered by
sivanantha321
Dec 2, 2023
Replies: 1 comment
-
@cceyda Transformer and Predictor both have separate URLs. You can bypass the transformer by using the predictor URL. You can find it in the isvc status. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
cceyda
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@cceyda Transformer and Predictor both have separate URLs. You can bypass the transformer by using the predictor URL. You can find it in the isvc status.