[Agentic Memory] Allow other LLMs for Fact Extraction

[rithin-pullela-aws]

Description

• Make it flexible to use any LLM models for fact extraction.
• Adds basic ITs for fact extraction in Add Memory API
• Refactor prompts for better readability and improved output format generation

With this PR we expect the users to pass response_filter in parameters of the LLM connectors and thus allowing more flexible LLM options for fact extraction in Agentic Memory. This also means we can have a more flexible connector request body without tying to only claude models.

To use a Claude model:

{
  "name": "Bedrock infer model",
  "function_name": "remote",
  "description": "LLM model for memory processing",
  "connector": {
    "name": "Amazon Bedrock Connector: LLM",
    "description": "The connector to bedrock Claude 3.7 sonnet model",
    "version": 1,
    "protocol": "<AWS_REGION>",
    "parameters": {
      "region": "us-east-1",
      "service_name": "bedrock",
      "max_tokens": 8000,
      "temperature": 1,
      "anthropic_version": "bedrock-2023-05-31",
      "model": "us.anthropic.claude-3-5-sonnet-20240620-v1:0",
      "response_filter": "$.content[0].text"
    },
    "credential": {
        "access_key": <AWS_ACCESS_KEY>,
        "secret_key": <AWS_SECRET_KEY>,
        "session_token": <AWS_SESSION_TOKEN>
    },
    "actions": [{
      "action_type": "predict",
      "method": "POST",
      "headers": {"content-type": "application/json"},
      "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/invoke",
      "request_body": "{ \"system\": \"${parameters.system_prompt}\", \"anthropic_version\": \"${parameters.anthropic_version}\", \"max_tokens\": ${parameters.max_tokens}, \"temperature\": ${parameters.temperature}, \"messages\": [{\"role\":\"user\",\"content\":[{ \"type\": \"text\", \"text\":\"${parameters.messages}\"}]}]}"
    }]
  }
}

To use Open AI models:

{
    "name": "My OpenAI model: gpt-4o",
    "function_name": "remote",
    "description": "test model",
    "connector": {
        "name": "My openai connector: gpt-4o mini",
        "description": "The connector to openai chat model",
        "version": 1,
        "protocol": "http",
        "parameters": {
            "model": "gpt-4o-mini",
            "response_filter": "$.choices[0].message.content"
        },
        "credential": {
            "openAI_key": <OPENAI_API_KEY>
        },
        "actions": [
            {
                "action_type": "predict",
                "method": "POST",
                "url": "https://api.openai.com/v1/chat/completions",
                "headers": {
                    "Authorization": "Bearer ${credential.openAI_key}"
                },
                "request_body": "{ \"model\": \"${parameters.model}\", \"messages\": [{\"role\":\"system\",\"content\":\"${parameters.system_prompt}\"},{\"role\":\"user\",\"content\":\"${parameters.messages}\"}] }"
            }
        ]
    }
}

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[rithin-pullela-aws]

The failing ITs are because of usage of deprecated Bedrock models.

{"status":404,"error":{"type":"OpenSearchStatusException","reason":"System Error","details":"Error from remote service: {\"message\":\"This model version has reached the end of its life. Please refer to the AWS documentation for more details.\"}"}}

Will raise another PR to fix

[dhrubo-os]

The failing ITs are because of usage of deprecated Bedrock models.

{"status":404,"error":{"type":"OpenSearchStatusException","reason":"System Error","details":"Error from remote service: {\"message\":\"This model version has reached the end of its life. Please refer to the AWS documentation for more details.\"}"}}

Will raise another PR to fix

Cool. Let's raise the PR to fix the integ suite. Then we can upgrade the branch with the change to run the integ test again. Thanks.

[dhrubo-os]

Here we are catching a generic exception and then throwing a 400 level exception. Can we try to catch specific exceptions to provide better error context?