Python: Bug: Mistral-Large-2411 does not work as exepcted

[ManojBableshwar]

Describe the bug
The https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/chat-history?pivots=programming-language-python

I'm trying out the simulating-function-calls section from the above docs. It works with gpt-4o-mini but does not work with Mistral-large-2411

To Reproduce

from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion

chat_completion_service = AzureChatCompletion(
    deployment_name="Mistral-Large-2411",  
)

from semantic_kernel import Kernel

# Initialize the kernel
kernel = Kernel()

# Add the chat completion service created above to the kernel
kernel.add_service(chat_completion_service)

from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase

# Retrieve the chat completion service by type
chat_completion_service = kernel.get_service(type=ChatCompletionClientBase)
print (chat_completion_service)

# Retrieve the chat completion service by id
chat_completion_service = kernel.get_service(service_id=chat_completion_service.service_id)

# Retrieve the default inference settings
execution_settings = kernel.get_prompt_execution_settings_from_service_id(chat_completion_service.service_id)
print (execution_settings)

from semantic_kernel.connectors.ai.open_ai import OpenAIChatPromptExecutionSettings

execution_settings = OpenAIChatPromptExecutionSettings()


from semantic_kernel.contents.chat_history import ChatHistory
from semantic_kernel.contents import ChatMessageContent, TextContent, ImageContent
from semantic_kernel.contents.utils.author_role import AuthorRole
# Removed the import statement for ChatMessage as it does not exist

chat_history = ChatHistory()
from semantic_kernel.contents import ChatMessageContent, FunctionCallContent, FunctionResultContent

# Add a simulated function call from the assistant
chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.ASSISTANT,
        items=[
            FunctionCallContent(
                name="get_user_allergies-User",
                id="0001",
                arguments=str({"username": "laimonisdumins"})
            ),
            FunctionCallContent(
                name="get_user_allergies-User",
                id="0002",
                arguments=str({"username": "emavargova"})
            )
        ]
    )
)

# Add a simulated function results from the tool role
chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.TOOL,
        items=[
            FunctionResultContent(
                name="get_user_allergies-User",
                id="0001",
                result="{ \"allergies\": [\"peanuts\", \"gluten\", \"eggs\"] }"
            )
        ]
    )
)
chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.TOOL,
        items=[
            FunctionResultContent(
                name="get_user_allergies-User",
                id="0002",
                result="{ \"allergies\": [\"dairy\", \"gluten\"] }"
            )
        ]
    )
)

chat_history.add_user_message("list unique allergens")

response = await chat_completion_service.get_chat_message_content(
    chat_history=chat_history,
    settings=execution_settings,
)

print(response)

Expected behavior
If i use gpt-4o-mini i correctly get the following response:

The unique allergens from the provided lists are:

1. Peanuts
2. Gluten
3. Eggs
4. Dairy

So the complete list of unique allergens is:

• Peanuts
• Gluten
• Eggs
• Dairy

However, if i use Mistral-Large-2411, i get the following error:

HttpResponseError: (Bad Request) {"object":"Error","message":"Unexpected role 'user' after role 'tool'","type":"invalid_request_error","code":3230}
Code: Bad Request
Message: {"object":"Error","message":"Unexpected role 'user' after role 'tool'","type":"invalid_request_error","code":3230}

Platform

• Language: [Python]
• Source: [pip install]
• AI model: [Mistral-Large-2411]
• IDE: [VS Code]
• OS: [WSL]

Additional context
Add any other context about the problem here.

[moonbox3]

Hi @ManojBableshwar, the AzureChatCompletion class exists as a means to interface with OpenAI models through Azure. It isn't meant to be a catch-all as an AI connector. With that said, given that you can deploy other models on Azure, and possibly reference them by changing endpoints, it may not always be supported right out of the box.

Can you have a look at using our AzureAIInferenceChatCompletion AI connector? This is meant to provide connections to more models through Azure: https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/?tabs=csharp-AzureOpenAI%2Cpython-AzureAIInference%2Cjava-AzureOpenAI&pivots=programming-language-python

Given our AI connector, as linked above, it looks like they provide support to call Mistral models as shown here: https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-mistral?tabs=mistral-large&pivots=programming-language-python#create-a-client-to-consume-the-model

If needed, please help provide more context about where your model is hosted -- for example, it's hosted through Azure AI Foundry?

[ManojBableshwar]

Tried this:

from semantic_kernel.connectors.ai.azure_ai_inference import AzureAIInferenceChatCompletion
chat_completion_service = AzureAIInferenceChatCompletion(
    ai_model_id="Mistral-Large-2411",
)

from semantic_kernel import Kernel
kernel = Kernel()
kernel.add_service(chat_completion_service)
from semantic_kernel.connectors.ai.azure_ai_inference import AzureAIInferenceChatPromptExecutionSettings
execution_settings = AzureAIInferenceChatPromptExecutionSettings()

but i get the same error...

[moonbox3]

@TaoChenOSU thoughts here?

[TaoChenOSU]

Hi @ManojBableshwar, thank you for reaching out!

TL;DR: Try inserting an assistant message after the tool messages.
 
I think there are two things I'd like point out here:

1. We don't know your underlying endpoint of the service you are using so there could be some ambiguity in terms of the behavior. Since you were able to talk to both models, I am assuming the APIs are compatible. However, it'd still help if you can provide more information.
2. You mentioned that your code worked for GPT-4o-mini but not Mistral-Large. I suspect the error you got is more likely due to model limitations. From my experience, OpenAI models (or services) are less restrictive on the sequence of messages users provide. In your case when using Mistral-Large, I would see if inserting an assistant message right after the tool messages can help resolve the issue. This is because the correct message sequence should be 'user query' -> 'assistant tool call -> 'tool execution results' -> 'assistant final answer', and some models expect the correct sequence or exceptions will be thrown.

[ManojBableshwar]

good catch, this worked...

from semantic_kernel.connectors.ai.azure_ai_inference import AzureAIInferenceChatCompletion
chat_completion_service = AzureAIInferenceChatCompletion(
    ai_model_id="Mistral-Large-2411",
)

from semantic_kernel import Kernel
kernel = Kernel()
kernel.add_service(chat_completion_service)

from semantic_kernel.connectors.ai.azure_ai_inference import AzureAIInferenceChatPromptExecutionSettings
execution_settings = AzureAIInferenceChatPromptExecutionSettings()

from semantic_kernel.contents.chat_history import ChatHistory
from semantic_kernel.contents import ChatMessageContent, TextContent, ImageContent
from semantic_kernel.contents.utils.author_role import AuthorRole

chat_history = ChatHistory()
from semantic_kernel.contents import ChatMessageContent, FunctionCallContent, FunctionResultContent

# Add a simulated function call from the assistant
chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.USER,
        items=[
            TextContent(text="list all unique allergens"),
        ]
    )
)

chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.ASSISTANT,
        items=[
            FunctionCallContent(
                name="get_user_allergies-User",
                id="123456789",
                arguments=str({"username": "laimonisdumins"})
            ),
            FunctionCallContent(
                name="get_user_allergies-User",
                id="223456789",
                arguments=str({"username": "emavargova"})
            )
        ]
    )
)

# Add a simulated function results from the tool role
chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.TOOL,
        items=[
            FunctionResultContent(
                name="get_user_allergies-User",
                id="123456789",
                result="{ \"allergies\": [\"peanuts\", \"gluten\", \"eggs\"] }"
            )
        ]
    )
)
chat_history.add_message(
    ChatMessageContent(
        role=AuthorRole.TOOL,
        items=[
            FunctionResultContent(
                name="get_user_allergies-User",
                id="223456789",
                result="{ \"allergies\": [\"dairy\", \"gluten\"] }"
            )
        ]
    )
)

#chat_history.add_user_message("list unqiue allergens")

response = await chat_completion_service.get_chat_message_content(
    chat_history=chat_history,
    settings=execution_settings,
)

print(response)