[BUG]Agentic Search <-> Ollama serialization error

[blevine]

Originally from opensearch-project/OpenSearch#20402 (comment)

I’ve followed all the steps to configure agentic search (conversational agent) on my local OpenSearch server to use a local LLM running under Ollama. However, my query produces the error below. See the HTTP traces showing that a property is incorrectly serialized as a float instead of an int.

This is the query I issued using curl (sorry for the formatting):

curl -k -u admin:admin -X GET “``http://localhost:9200/able_chunks_v1/_search?search_pipeline=agentic-pipeline``”   -H “Content-Type: application/json”   -d ‘{
  “query”: {
      “agentic”: {
        “query_text”: “How many documents are there in the index”
      }
  }
}’

And this is the error:
”json: cannot unmarshal number 0.0 into Go struct field ToolCall.messages.tool_calls.index of type int”

Full error:

{“error”:{“root_cause”:[{“type”:“illegal_argument_exception”,“reason”:“Agentic search failed - Agent execution error - Agent ID: [_Nh7rZsBMCptIK-aGFFT], Error: [Error from remote service: {"error":{"message":"json: cannot unmarshal number 0.0 into Go struct field ToolCall.messages.tool_calls.index of type int","type":"invalid_request_error","param":null,"code":null}}]”}],“type”:“illegal_argument_exception”,“reason”:“Agentic search failed - Agent execution error - Agent ID: [_Nh7rZsBMCptIK-aGFFT], Error: [Error from remote service: {"error":{"message":"j**son: cannot unmarshal number 0.0 into Go struct field ToolCall.messages.tool_calls.index of type int",**"type":"invalid_request_error","param":null,"code":null}}]”,“caused_by”:{“type”:“status_exception”,“reason”:“Error from remote service: {"error":{"message":"json: cannot unmarshal number 0.0 into Go struct field ToolCall.messages.tool_calls.index of type int","type":"invalid_request_error","param":null,"code":null}}”}},“status”:400

Configuration:

OS/Hardware: MacOS: MacBook Pro M3 Max
OpenSearch: OpenSearch 3.4 running under Docker. Image: opensearchproject/opensearch:3.4.0
LLM: A Qwen model running under Ollama. Ollama is running on host. Ollam version: 0.13.5

** Plugins **

opensearch-node1 opensearch-alerting 3.4.0.0
opensearch-node1 opensearch-anomaly-detection 3.4.0.0
opensearch-node1 opensearch-asynchronous-search 3.4.0.0
opensearch-node1 opensearch-cross-cluster-replication 3.4.0.0
opensearch-node1 opensearch-custom-codecs 3.4.0.0
opensearch-node1 opensearch-flow-framework 3.4.0.0
opensearch-node1 opensearch-geospatial 3.4.0.0
opensearch-node1 opensearch-index-management 3.4.0.0
opensearch-node1 opensearch-job-scheduler 3.4.0.0
opensearch-node1 opensearch-knn 3.4.0.0
opensearch-node1 opensearch-ltr 3.4.0.0
opensearch-node1 opensearch-ml 3.4.0.0
opensearch-node1 opensearch-neural-search 3.4.0.0
opensearch-node1 opensearch-notifications 3.4.0.0
opensearch-node1 opensearch-notifications-core 3.4.0.0
opensearch-node1 opensearch-observability 3.4.0.0
opensearch-node1 opensearch-performance-analyzer 3.4.0.0
opensearch-node1 opensearch-reports-scheduler 3.4.0.0
opensearch-node1 opensearch-search-relevance 3.4.0.0
opensearch-node1 opensearch-security 3.4.0.0
opensearch-node1 opensearch-security-analytics 3.4.0.0
opensearch-node1 opensearch-skills 3.4.0.0
opensearch-node1 opensearch-sql 3.4.0.0
opensearch-node1 opensearch-system-templates 3.4.0.0
opensearch-node1 opensearch-ubi 3.4.0.0
opensearch-node1 query-insights 3.4.0.0

HTTP Traffic:

I sniffed the HTTP traffic between OpenSearch and Ollama and found that OpenSearch is indeed sending an index property with a float (0.0) instead of an integer (0).

OpenSearch sends 2 POSTs to the v1/chat/completions endpoint. The response to the first POST looks like the below (note the index property in choices[0].tool_calls[0].index):

{
  "id": "chatcmpl-170",
  "object": "chat.completion",
  "created": 1768222804,
  "model": "qwen3-a3b-16k",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_2l0j5wr2",
            "index": 0,  <---- Note!
            "type": "function",
            "function": {
              "name": "ListIndexTool",
              "arguments": "{\"indices\":[\"able_chunks_v1\"]}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 1643,
    "completion_tokens": 25,
    "total_tokens": 1668
  }
}

The second POST contains the following snippet (I've elided the rest of the request to save space, but can provide it if needed). Note that index property under tool_calls[0]:

{
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "call_2l0j5wr2",
          "index": 0.0, <---- Note!
          "type": "function",
          "function": {
            "name": "ListIndexTool",
            "arguments": "{\"indices\":[\"able_chunks_v1\"]}"
          }
        }
      ]
    },

So it looks like OpenSearch is formatting that value as a float instead of an integer which is causing the problem

[blevine]

Additional information. Some Googling came up with the statement below. So perhaps a fix for this is not to include the index property at all.

In the OpenAI API protocol, the index property is primarily a feature of streaming responses (via the ChatCompletionChunk object).
In a standard non-streaming request, the assistant message contains a tool_calls array where order is implicit by the list index. However, when you send a message back to the API to maintain conversation history, or if you are manually constructing a request that reflects a previous tool-calling turn, you do not typically include the index property in the messages array; the API expects id, type, and function