Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Litellm open ai compatible protocol #99

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

ngoiyaeric
Copy link

—>
feat: A new feature
optimize: A new optimization
fix: A bug fix
perf: A code change that improves performance
-->

Check the PR title.

  • This PR title match the format: <type>(optional scope): <description>
  • The description of this PR title is user-oriented and clear enough for others to understand.
  • Attach the PR updating the user documentation if the current PR requires user awareness at the usage level. [User docs repo](https://github.com/

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:
zh(optional):

(Optional) Which issue(s) this PR fixes:

Add LitellmModel for OpenAI-compatible API integration.

Define LitellmModel struct with a client field for OpenAI-compatible APIs.
Implement NewLitellmModel function to create a new instance of LitellmModel.
Add GenerateText method to generate text based on a given prompt using the OpenAI-compatible API.
Include a comment explaining Litellm's compatibility with the OpenAI API protocol.

Add LitellmModel for OpenAI-compatible API integration.

* Define `LitellmModel` struct with a client field for OpenAI-compatible APIs.
* Implement `NewLitellmModel` function to create a new instance of `LitellmModel`.
* Add `GenerateText` method to generate text based on a given prompt using the OpenAI-compatible API.
* Include a comment explaining Litellm's compatibility with the OpenAI API protocol.
Add Litellm integration with OpenAI-compatible APIs
@CLAassistant
Copy link

CLAassistant commented Mar 8, 2025

CLA assistant check
All committers have signed the CLA.

@meguminnnnnnnnn
Copy link
Contributor

Hi, thanks for contribution.

  • This repository is used to store component abstractions, orchestration capabilities, Agent paradigms, and other implementation-independent code. Component implementations can be submitted in the eino-ext.
  • Components in eino-ext must implement interface defined in eino/component, for example,
    type ChatModel interface {
  • Why do you need this pr? I noticed that this protocol is exactly the same as OpenAI's, so maybe you can directly use the OpenAI package?

@ngoiyaeric
Copy link
Author

Do you have any code specific improvements should I be calling https://github.com/openai/openai-go from instead? Should this be implemented on eino-ext in this package eino-ext/libs/acl as a litellm folder with open ai protocol

@meguminnnnnnnnn
Copy link
Contributor

It seems that this PR simply wraps the OpenAI protocol, which has already been done in https://github.com/cloudwego/eino-ext/tree/main/components/model/openai.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants