Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use free Llama 3 on Groq instead of OpenAI #16

Open
jeffxtang opened this issue Jun 16, 2024 · 2 comments
Open

Use free Llama 3 on Groq instead of OpenAI #16

jeffxtang opened this issue Jun 16, 2024 · 2 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@jeffxtang
Copy link

If using OpenAI is too expensive for you to try out examples with long text, you can create a free API key in seconds at https://console.groq.com/keys using your gmail or github then make the changes below:

  1. run:
    pip install groq

  2. Add a new line in your .env file (renamed from .env.sample, originally with a line starting with "OPEN_API_KEY"):
    GROQ_API_KEY="your_groq_api_key"

  3. In src/translation_agent/utils.py, make 3 changes:

  • change import openai to from groq import Groq
  • change client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY")) to
    client = Groq(api_key=os.getenv("GROQ_API_KEY"))
  • change model: str = "gpt-4-turbo" to model: str = "llama3-8b-8192" or model: str = "llama3-70b-8192"

Now running python examples/example_script.py will use Llama 3 on Groq for free.

@icecreamzhao
Copy link

Add one more line: change model_name="gpt-4" to model_name="llama3"

@methanet
Copy link
Collaborator

@jeffxtang Thanks for the suggestions!
Adding support for Llama would be a great extension to the codebase. Would you like to submit a PR with the proposed extension incorporating the Groq APIs? Much appreciated.

@icecreamzhao good catch!

@methanet methanet added the enhancement New feature or request label Jun 28, 2024
@methanet methanet added the help wanted Extra attention is needed label Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants