KernelArguments never passed to a Plugin

[tojuasp]

When using an agent with FunctionChoiceBehavior.Auto(), when chatting with the agent like so:

  var kernelArgs = new KernelArguments()
  {
      ["Testing"] = "Testing"
  };
  
  var response = _agent.InvokeAsync(chatHistory, kernelArgs, cancellationToken: cancellationToken);

And if the agent makes a decision to invoke a plugin,

public class TimePlugin
{
    [KernelFunction("get_time")]
    [Description("Get current date and time in utc timezone")]
    public DateTime GetTime(KernelArguments kernelArgs)
    {
        // Demonstrating purposes
        var testing = kernelArgs["Testing"]?.ToString();
        return DateTime.UtcNow;
    }
}

Should it pass the kernelArguments to that plugin? Currently it doesn't. Am I doing something wrong or what. Only thing I can use to pass custom data to the plugin is to use kernel.Data, because that does get passed down.

[sphenry]

@markwallace-microsoft can you take a look?

[markwallace-microsoft]

@tojuasp What type of Agent are you using?

[tojuasp]

I am using ChatCompletionAgent with AzureOpenAICompletion service.

[markwallace-microsoft]

@crickman what do you think of this request?

[crickman]

Doesn't the model specifiy all parameter names and values when invoking a plug-in function?

In this example, the KernelArguments provided to GetMenu is empty:

    Kernel kernel = this.CreateKernelWithChatCompletion();
    KernelFunction promptFunction = kernel.CreateFunctionFromPrompt("What is on the menu?");
    KernelPlugin menuPlugin = kernel.CreatePluginFromType<MenuPlugin>();
    kernel.Plugins.Add(menuPlugin);
    KernelArguments arguments = new(new OpenAIPromptExecutionSettings() { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() }) { { "now", DateTime.UtcNow } };
    FunctionResult result = await promptFunction.InvokeAsync(kernel, arguments);
    Console.WriteLine(result.GetValue<string>());

    private sealed class MenuPlugin
    {
        [KernelFunction]
        public string GetMenu(KernelArguments arguments)
        {
            Console.WriteLine(arguments.Count);
            return
                """
                Special Soup: Clam Chowder
                Special Salad: Cobb Salad
                Special Drink: Chai Tea
                """;
        }
    }

@markwallace-microsoft , is your intent to support this pattern generally? (I seem to remember this being a point of contention early on and work being done to eliminate argument propegation between function invocations.) W.R.T. Chat-Completion, I don't believe there is an option to pass KernelArguments on invocation.

[tojuasp]

I think I misunderstood a bit how KernelArguments work when trying this out. We ended up using kernel.Data anyway to pass that kind of information around that the LLM doesn't need. At first we were intrigued by the KernelArguments because it seemed like a way to pass information without having to bind it to kernel's lifecycle but then found a fix to our initial bot lifecycle issue that made us look into this in the first place.

So for now, this "issue" is not that relevant for us, only in terms of understanding how KernelArguments work. Is there any scenario even that a plugin like this would receive KernelArguments? Are those just something that if the prompt given to the LLM includes some parameters, the LLM would then also provide those back with values as KernelArguments? So in our case, nothing gets injected because our Plugin, when converted to LLM prompt, doesn't actually include any parameters.

[crickman]

Thanks for the follow-up @tojuasp. Generally, the intent of the KernelArguments passed to the agent is for it be used for parameterized instructions.