Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] Enable Tool arguments to process pydantic schemas properly #699

Open
fedorzh opened this issue Feb 18, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@fedorzh
Copy link

fedorzh commented Feb 18, 2025

Is your feature request related to a problem? Please describe.
Tools don't properly pass pydantic schema for the objects in their functions.
For example, my tool takes the following as its input parameter

class ReportType(Enum):
    REPORT_1 = "report_1"
    REPORT_2 = "report_2"

    @classmethod
    def values(cls) -> list[str]:
        return [type.value for type in cls]

class ReportConfig(BaseModel):
    sheet_name: str = Field(description="Name of the sheet")
    requested_start_date: date = Field(description="Start date")
    requested_end_date: date = Field(description="End date")
    report_type: str = Field(
        description="Type of the report",
        enum=ReportType.values()
    )

If I call
ReportConfig.model_json_schema() to be passed into, say, OpenAI,

tools=[
        {
            "type": "function",
            "function": {
                "name": "report_work",
                "description": "Adds a report. The configuration must follow the ReportConfig schema.",
                "parameters": ReportConfig.model_json_schema(),
            },
        }
]

I get a correct call
ReportConfig(sheet_name='ABC', requested_start_date=datetime.date(2024, 1, 1), requested_end_date=datetime.date(2024, 1, 31), report_type='report_1')
with the correct tool call
{'properties': {'sheet_name': {'description': 'Name of the sheet', 'title': 'Sheet Name', 'type': 'string'}, 'requested_start_date': {'description': 'Start date', 'format': 'date', 'title': 'Requested Start Date', 'type': 'string'}, 'requested_end_date': {'description': 'End date', 'format': 'date', 'title': 'Requested End Date', 'type': 'string'}, 'report_type': {'description': 'Type of the report', 'enum': ['report_1', 'report_2'], 'title': 'Report Type', 'type': 'string'}}, 'required': ['sheet_name', 'requested_start_date', 'requested_end_date', 'report_type'], 'title': 'ReportConfig', 'type': 'object'}

However, when I use LiteLLMModel with a tool with the following inputs:
inputs = {
"report_config": {
"type": "object",
"description": "Configuration for the new report.",
"properties": ReportConfig.model_json_schema()['properties'],
},
}

I get the call with
{'report_config': {'sheet_name': 'ABC', 'requested_start_date': '2024-01-01', 'requested_end_date': '2024-01-31'}}

So first, required fields are not passed, which results in an incomplete request. Moreover, the enum is not properly passed as get_json_schema function in ..._hint_utils.py actually uses strange choice way of defining enums

Describe the solution you'd like
I want to be able to specify ReportConfig.model_json_schema() in my tool input as a parameter, for a complex nested model. This will ensure much more consistent tool calling

Is this not possible with the current options.
At least I am not aware how to pass the schema differently

Describe alternatives you've considered
The only alternative I see is to write my own Tool class and overwrite most of the functions there, but it's possible this won't be enough as the LiteLLMModel might work with tools in a very specific way.

Additional context
Basically, it would be good to have at least parity with openai/anthropic tool calls capabilities in tool arguments

@fedorzh fedorzh added the enhancement New feature or request label Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant