Add the Vertex AI API support for Anthropic backend

[dodjiseketeli]

Google Cloud Platform's Vertex AI provides access to Anthropic's Claude models through a wrapper API that differs slightly from Anthropic's direct API. The key differences are:

1. The model identifier is embedded in the endpoint URL rather than in the request body
2. Vertex AI rejects requests that include a "model" field in the JSON payload with an "Extra inputs are not permitted" error
3. Authentication uses Google Cloud OAuth tokens instead of Anthropic API keys

This patch adds a uses-vertex-ai flag to the gptel-anthropic backend structure that, when enabled, prevents the "model" field from being added to API requests. This allows gptel to work seamlessly with Claude models accessed through Vertex AI.

An example configuration for the Vertex AI API in a user's Emacs configuration would be:

(setq gptel-backend
(gptel-make-anthropic "VertexAI"
:uses-vertex-ai t
:header (lambda ()
(let ((token (string-trim
(shell-command-to-string
"gcloud auth print-access-token"))))
`(("Authorization" . ,(concat "Bearer " token)))))
:stream t
:host "-aiplatform.googleapis.com"
:request-params '(:anthropic_version "vertex-2023-10-16")
:endpoint (format "/v1/projects/%s/locations/%s/publishers/anthropic/models/%s:streamRawPredict"
"YOUR-PROJECT-ID"
""
gptel-model)))

The uses-vertex-ai parameter defaults to nil, maintaining backward compatibility with existing Anthropic API configurations.

    * gptel-anthropic.el (gptel-anthropic): Add `uses-vertex-ai` slot
    to the structure.
    (gptel--request-data): Conditionally omit the `:model` field from the
    request plist when `uses-vertex-ai` is non-nil.
    (gptel-make-anthropic): Add `uses-vertex-ai` keyword parameter and
    pass it to the backend constructor.

[dodjiseketeli]

hello, I just wanted to add that I have been testing this patch and it does work both when using the GCP/Vertex AI API and when using the direct anthropic API. Thank you for such a great project!
Cheers,

[karthink]

@dodjiseketeli Thanks for the PR. I'm quite confused about what's happening here, though:

You are accessing Anthropic's models via Vertex and want to add support to gptel for doing that. Is that correct?

If it is, I think it will be better to add a gptel-vertex feature to gptel adding explicit support for Vertex via gptel. You can then use gptel-make-vertex to access the Vertex API.

Since you're familiar with the Vertex API, do you know which API format it uses? Is it OpenAI-Completions, Anthropic or Gemini, or is it more than one of these, or is the API used model-dependent?

[dodjiseketeli]

@dodjiseketeli Thanks for the PR.

You are welcome.

[...]

You are accessing Anthropic's models via Vertex and want to add support to gptel for doing that. Is that correct?

That is correct.

If it is, I think it will be better to add a gptel-vertex feature to gptel adding explicit support for Vertex via gptel.

You mean, something like what this issue suggests? #220.

I have seen that issue after I solved my issue by trial and error.

You can then use gptel-make-vertex to access the Vertex API.

Right.

In the grand scheme of things, it's arguable that having a vertex backend for gptel would be the way to go. However, we need to keep in mind that we would then need specific support for each kind of model vendor supported by Vertex AI; that is anthropic support, openai support, gemini support, etc.

Alternatively, we could also think that each existing gptel backend could have a "parameter" to be configured to run either natively (as it is now) or, on the Vertex AI platform.

The later is the approach I have taken for the gptel-anthropic backend, just because it was more direct to do that my own personal need, without having to write an entire vertex AI backend with anthropic support.

I believe the former approach is the one taken by https://github.com/ashlineldridge/gptel-vertex, where it has support for anthropic and gemini.

Since you're familiar with the Vertex API, do you know which API format it uses? Is it OpenAI-Completions, Anthropic or > Gemini, or is it more than one of these, or is the API used model-dependent?

From my limited exposure of this thing, I would say that Vertex API generic format is specific to Vertex, especially for things like authentication and the selection of the model to run. But then, it uses a model-dependent format that re-uses the format of the model that is selected.

You can see an example of the format used for anthropic models at https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/generative_ai/anthropic_claude_3_intro.ipynb.

Maybe I should play with https://github.com/ashlineldridge/gptel-vertex and report back. This looks like the approach you would prefer, right?