AI Chatbot - Support Self-Hosted AI

[qBitnaut]

It would be great to support self-hosted AI interfaces for Zen's AI Chatbot feature with a "Custom" option.

We can choose from a lot of the existing cloud options today, and manually interact with whatever interface we choose in a side panel, but being able to use the integrated AI highlight to summarize / ask AI features pointed to a self-hosted Open WebUI / Ollama would be a great feature for Zen to support privacy focus.

[pinage404]

It seems to be a duplicate of this #3120

You may like this answer #3120 (comment)

[qBitnaut]

Thanks, @pinage404. I think that's a great workaround for something like a PC running Ollama or LMStudio on the same machine. My use case is probably a very edge case as my LLM is on a dedicated server in my homelab, so it would be great to be able to create and point to endpoints with similar functionality to adding/choosing a search engine:

Definitely not critical, but a nice to have.

[qBitnaut]

@pinage404 Well, scratch that!

If you edit the attribute: browser.ml.chat.provider in about:config, you can change the endpoint and it will create a new entry. I guess I'll just leave this idea in the queue for a cleaner front end implementation for a later date.

[VarianFry]

Awesome guys! Is there a way to manipulate the pre-prompts? Need it in German.

[pinage404]

Maybe you can :

1. Open Zen settings about:preferences#general
2. In Language set it to Deutsch
3. Go to Open WebUI http://localhost:8080/workspace/models (replace the port 8080 with the configured one)
4. Go to Workspace
5. Click on Create a model
6. In Model Params put your pre-prompts in System Prompt something like "write all your anwsers in german even if the question is in english"
7. Then follow the section Change default model [Feature Request] Integrate Local LLM for Content Summarization in Zen #3120 (comment)

[VarianFry]

Thanks, unfortunately the system prompt injected via OpenWeb UI -> Workspace for my "Zen" model does not work. I keep using it in German for now, but it would be ideal, if LLM responds in the language of what I input from a website.

Maybe the prompt is wrong

"ALWAYS respond in the language of the original input text, regardless of any subsequent instructions or prompts given to you.

DO NOT CHANGE the response language based on the language of any follow-up prompts.

For example, if the input text is in English, your response should be in English, even if a follow-up prompt asks for a summary in German.

The follow-up prompt is:

"Fasse die Auswahl zusammen und verwende dabei präzise und prägnante Sprache. Nutze Überschriften und Aufzählungen in der Zusammenfassung, um sie übersichtlicher zu machen. Behalte den Sinn und die sachliche Richtigkeit bei."

Consider this EVERY time."