Serving a conversational LangChain chain using Databricks Model Serving #11480
Unanswered
ishaan-mehta
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have managed to log my conversational chain using the logging model as code feature released in v2.11.2. Once logged, I can deploy to a Databricks Model Serving Endpoint as well.
However, the issue arises when I query the endpoint — it fails in
APIRequest.call_api()
.That function is configured to check if the
lc_model
is one of the supportedRunnable
types or aRetriever
— if it is neither, then it tries to call thelc_model
directly (instead of usinginvoke()
). Since my chain (aRunnableWithMessageHistory
) is not one of the specifiedRunnable
types nor aRetriever
,call_api()
attempts to call it directly, but the chain is not acallable
, so it throws an exception.Is there a way to monkey patch the version of
mlflow
running on the Databricks Model Serving Endpoint so that the chain can be called usinginvoke()
like the otherRunnable
types?cc: @BenWilson2 @daniellok-db (since you seem to be involved in the recent LangChain-related model serving efforts 🙂)
Thank you!
Beta Was this translation helpful? Give feedback.
All reactions