You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Tools don't properly pass pydantic schema for the objects in their functions.
For example, my tool takes the following as its input parameter
class ReportType(Enum):
REPORT_1 = "report_1"
REPORT_2 = "report_2"
@classmethod
def values(cls) -> list[str]:
return [type.value for type in cls]
class ReportConfig(BaseModel):
sheet_name: str = Field(description="Name of the sheet")
requested_start_date: date = Field(description="Start date")
requested_end_date: date = Field(description="End date")
report_type: str = Field(
description="Type of the report",
enum=ReportType.values()
)
If I call ReportConfig.model_json_schema() to be passed into, say, OpenAI,
tools=[
{
"type": "function",
"function": {
"name": "report_work",
"description": "Adds a report. The configuration must follow the ReportConfig schema.",
"parameters": ReportConfig.model_json_schema(),
},
}
]
I get a correct call ReportConfig(sheet_name='ABC', requested_start_date=datetime.date(2024, 1, 1), requested_end_date=datetime.date(2024, 1, 31), report_type='report_1')
with the correct tool call {'properties': {'sheet_name': {'description': 'Name of the sheet', 'title': 'Sheet Name', 'type': 'string'}, 'requested_start_date': {'description': 'Start date', 'format': 'date', 'title': 'Requested Start Date', 'type': 'string'}, 'requested_end_date': {'description': 'End date', 'format': 'date', 'title': 'Requested End Date', 'type': 'string'}, 'report_type': {'description': 'Type of the report', 'enum': ['report_1', 'report_2'], 'title': 'Report Type', 'type': 'string'}}, 'required': ['sheet_name', 'requested_start_date', 'requested_end_date', 'report_type'], 'title': 'ReportConfig', 'type': 'object'}
However, when I use LiteLLMModel with a tool with the following inputs:
inputs = {
"report_config": {
"type": "object",
"description": "Configuration for the new report.",
"properties": ReportConfig.model_json_schema()['properties'],
},
}
I get the call with {'report_config': {'sheet_name': 'ABC', 'requested_start_date': '2024-01-01', 'requested_end_date': '2024-01-31'}}
So first, required fields are not passed, which results in an incomplete request. Moreover, the enum is not properly passed as get_json_schema function in ..._hint_utils.py actually uses strange choice way of defining enums
Describe the solution you'd like
I want to be able to specify ReportConfig.model_json_schema() in my tool input as a parameter, for a complex nested model. This will ensure much more consistent tool calling
Is this not possible with the current options.
At least I am not aware how to pass the schema differently
Describe alternatives you've considered
The only alternative I see is to write my own Tool class and overwrite most of the functions there, but it's possible this won't be enough as the LiteLLMModel might work with tools in a very specific way.
Additional context
Basically, it would be good to have at least parity with openai/anthropic tool calls capabilities in tool arguments
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Tools don't properly pass pydantic schema for the objects in their functions.
For example, my tool takes the following as its input parameter
If I call
ReportConfig.model_json_schema()
to be passed into, say, OpenAI,I get a correct call
ReportConfig(sheet_name='ABC', requested_start_date=datetime.date(2024, 1, 1), requested_end_date=datetime.date(2024, 1, 31), report_type='report_1')
with the correct tool call
{'properties': {'sheet_name': {'description': 'Name of the sheet', 'title': 'Sheet Name', 'type': 'string'}, 'requested_start_date': {'description': 'Start date', 'format': 'date', 'title': 'Requested Start Date', 'type': 'string'}, 'requested_end_date': {'description': 'End date', 'format': 'date', 'title': 'Requested End Date', 'type': 'string'}, 'report_type': {'description': 'Type of the report', 'enum': ['report_1', 'report_2'], 'title': 'Report Type', 'type': 'string'}}, 'required': ['sheet_name', 'requested_start_date', 'requested_end_date', 'report_type'], 'title': 'ReportConfig', 'type': 'object'}
However, when I use
LiteLLMModel
with a tool with the following inputs:inputs = {
"report_config": {
"type": "object",
"description": "Configuration for the new report.",
"properties": ReportConfig.model_json_schema()['properties'],
},
}
I get the call with
{'report_config': {'sheet_name': 'ABC', 'requested_start_date': '2024-01-01', 'requested_end_date': '2024-01-31'}}
So first,
required
fields are not passed, which results in an incomplete request. Moreover, theenum
is not properly passed asget_json_schema
function in..._hint_utils.py
actually uses strangechoice
way of defining enumsDescribe the solution you'd like
I want to be able to specify
ReportConfig.model_json_schema()
in my tool input as a parameter, for a complex nested model. This will ensure much more consistent tool callingIs this not possible with the current options.
At least I am not aware how to pass the schema differently
Describe alternatives you've considered
The only alternative I see is to write my own Tool class and overwrite most of the functions there, but it's possible this won't be enough as the LiteLLMModel might work with tools in a very specific way.
Additional context
Basically, it would be good to have at least parity with openai/anthropic tool calls capabilities in tool arguments
The text was updated successfully, but these errors were encountered: