Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Provide information about the different models #315

Closed
benthamite opened this issue May 15, 2024 · 9 comments
Closed

[enhancement] Provide information about the different models #315

benthamite opened this issue May 15, 2024 · 9 comments
Labels
enhancement New feature or request

Comments

@benthamite
Copy link
Contributor

benthamite commented May 15, 2024

In my personal config I have some extensions for gptel, including a function and associated variable that displays information about the available models:

image

You can find it here. Would you be interested in a pull request? (I would adapt some of the descriptions to make them less subjective and personal.)

@karthink
Copy link
Owner

karthink commented May 15, 2024

Wow this looks great! Please give me a few days to take a look at your code, it's been a rough week. This ties in with some refactoring of the backend definitions I've been meaning to do that should make attaching metadata to the models easier.

@karthink karthink added the enhancement New feature or request label May 17, 2024
@karthink
Copy link
Owner

@benthamite As of gptel 0.9.5 (published today),

  • models are now symbols, and
  • each model can now be specified with a description.

Now would be a good time to incorporate your suggested changes. The only change from your version is that instead of defining a separate gptel-extras-ai-models variable to hold this information, fields like :description can be included in the model's symbol-plist.

@benthamite
Copy link
Contributor Author

Thanks! Let me take a closer look and I’ll try to prepare a pull request.

@benthamite
Copy link
Contributor Author

@karthink, a couple of questions:

  1. I see that, for some backends, you have already provided model descriptions (examples: OpenAI, Claude, Gemini). Would you like me to add descriptions for the remaining backends?

  2. Under "Other LLM backends", the Readme file mentions, for each backend, an optional configuration where the user can specify a particular model as the default. Some of the examples are outdated, as are the comments indicating alternative models. Maybe a more robust alternative is to just instruct the user to inspect the value of the :models property of the appropriate function (e.g. gptel-make-anthropic), so that the readme does not have to be constantly updated to reflect model changes?

  3. I have developed, for my own use, a modeline integration that displays the cost of making a gptel request at point with the selected model. I am now sure if this is worth it, but one could add an extra property (besides :description) that lists the cost per million tokens. Would you be interested in this?

@karthink
Copy link
Owner

  1. Those descriptions are placeholders. I think your descriptions are more helpful, you can replace the current ones with your versions.

  2. There are two kinds of backends: ones that we manage and update ourselves (like the OpenAI, Anthropic and Gemini backends) and ones managed by the user, where all models have to be specified by the user when creating the backend struct. I think we can update the examples in the README for the former kind, in addition to mentioning that available models can be accessed via (gptel-backend-models gptel-backend).

  3. Modeline integration: I'd like to avoid adding this, at least for now. Modeline elements are hard to manage in the long run, it's very easy to slow down Emacs accidentally that way. Adding a :cost field to the model properties is a good idea! It will have to be in USD for now, I guess -- I don't know if it's worth adding code to fetch costs in the local currency. You can also add as properties the other fields you have in your screenshot above -- the :last-update (which can just be :updated) and :tokens fields.

The last step will be updating the annotation function for the model selector in gptel-transient to show these details when selecting a model.

@karthink
Copy link
Owner

Considering the amount of information that will be shoved into the definition of gptel-make-openai (and others), I think it would be good to create defconsts like you do -- one per backend type in each backend file -- and just use it as the default value of models:

(defconst gptel--anthropic-models
  ;; model details here
)
(cl-defun gptel-make-anthropic (name &key (models gptel--anthropic-models) ...) ...)

@benthamite
Copy link
Contributor Author

Makes sense, thanks.

Concerning (3), yeah, sorry: I meant to suggest that I could add the :cost property so that this information is then available for integration with the modeline, rather than adding the integration itself to your package. Once the :cost property is available, I can submit a pull request to doom-modeline (which is the modeline I use and for which I have developed the extension). I’ll add :updated and :token properties as well.

@benthamite
Copy link
Contributor Author

Considering the amount of information that will be shoved into the definition of gptel-make-openai (and others), I think it would be good to create defconsts like you do

Noted, thanks.

@karthink
Copy link
Owner

Closed by #420.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants