Ollama - Good Translator Offline (Gemini-Gemma2, many models free)

[snakeotakon]

With Gemma 2
A family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models. INFO: https://ai.google.dev/gemma

INSTALLATION

1. Download the installer and install the program ollama Compatible clients ChatGPT
2. Run prompt cmd "ollama run gemma2"
   C:\Users\myuser> ollama run gemma2 (Automatic Download Model Gemma2 5GB)
3. Configure your Translator ChatGPT
   • Override Model: gemma2
   • 3rd party api url: http://localhost:11434/v1/

Works with the CPU, but is highly recommended with nvidia cards. RTX-3060

Ollama, a platform that makes local development with open-source large language models a breeze. With Ollama, everything you need to run an LLM—model weights and all of the config—is packaged into a single Modelfile

There are a large number of models that can be tried
Meta Llama 3: The most capable openly available LLM to date
deepseek-coder-v2 An open-source Mixture-of-Experts code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks.

https://ollama.com/library

[aqssxlzc]

 AstraZeroZak , I am using your setting and it works. You could change the api-key(which should be unnecessary) to some key looks real. AstraZeroZak ***@***.***> 于2024年9月3日周二 04:29写道：

…

I don't quite understand, I do everything as in the picture, but it doesn't work, can you tell me what the reason is? -1.png (view on web) <https://github.com/user-attachments/assets/6f2056c6-4184-433b-873a-01706374d065> — Reply to this email directly, view it on GitHub <#515 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIPT7IBZSNJKM4P2ALUUOLZUTDBZAVCNFSM6AAAAABKPXKRHOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJSGQ4DGNA> . You are receiving this because you are subscribed to this thread.Message ID: <dmMaze/BallonsTranslator/repo-discussions/515/comments/10524834@ github.com>

[AstraZeroZak]

Thanks for the reply, I solved the problem and deleted my message yesterday.
The problem was in my VPN settings.

[kocy33]

I just wanted to add my two cents to this topic of using LLMs for translation.
I have been very successful using the model "aya23:35b" (also available in the OLLAMA model library: "https://ollama.com/library/aya").
You need a very powerful graphics card for this (such as a 3090 or 4090) due to the VRAM requirements. It's perfectly possible to run the model on a 3090 or 4090, but the model barely fits.
However, the translations, even for very tricky cases, are outstanding compared to any other offline model in my application/tests (on par with GPT-4).
You can also run the model on a CPU, but you'll need a lot of RAM, and it will be much slower.

The best thing about "aya" is that it does not adhere to strict ethics compared to other LLMs like GPT-4, for example.
It won't teach you how to steal a car, but you can throw "NSFW" content at "aya," and it will gladly translate it without any questions asked.

I adjusted the system prompt in "Ballon Translator" slightly with: "Never talk to me directly, and just give me the translation."
This way, I don't get any "Of course, I am happy to help you with that..." responses in my translations.

Hope this helps someone. :)

P.S.:
I run AYA on a separate computer and use the API (I needed to set my environment variable to OLLAMA_HOST - 0.0.0.0) link in "Ballon Translator."
I also run a Web UI on this machine to prompt Aya directly for specific translations and to get explanations (for example: "Please break down this sentence and explain each part of it: ...").
It's a very good tool for learning. :)