It is so difficult to setup the Ai Provider

[realcarlos]

I think we need a setting page to test if the LLM and Embedder works easily, now it is so wired, always show "failed to create ask"

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[wwwy3y3]

@paopa , @cyyeh we might need to think about how to make AI provider integration easier. Seems a lot barrier for users.

[ravenizzed]

Yes please, i have been trying to integrate Deepseek with ollama, for the past 4 days, and still it says Failed to create task, unable to deploy. Even tho I am not a tech sound person, I am still trying yet failing miserably.

[paopa]

Hey @ravenizzed, @realcarlos, could you guys share your thoughts with us? What kind of option would work better for you—setting up an independent page on the UI, a more dedicated document, or something else? Also, could you share which part of the configuration setup is the most challenging for you? Your feedback will really help us improve the setup flow. Thanks a lot!

[realcarlos]

Hi @paopa , thank you for your prompt reply. Usually we choose "custom" when init the setup, and there is no visual UI for us to test if the LLM and Embedder config works. I think if there is a Ai setting page in the "Setting" section will help. Just replace the config.yaml with a visual web page , and add a "Test connection" button.

If you want to make it more perfect, you can refer to the model configuration module of Dify and add all the existing AI inference services.

WrenAi looks good ,I hope I can experience it locally soon. Thank you very much.

[ravenizzed]

Well, @paopa Let me share my recent experience with WrenAI.
A quick background: I am not technical and can read okayish.
When you access it through Openai, the process: OpenAI -> Key -> Model -> Data selection -> All good ( I did not have to look into any config files, or docker files)
FYI: I made Gemini custom model run after 1 day of tweaks, which is an achievement for me. ^_^
But Custom -> read config files, change config files, replace config files, more changing config files.
The option I would like to have:

1. Custom -> which custom model am I using? as you already have config files for the custom models WrenAI supports.
2. When in custom feature -> change the selected config right then and there with the user, Testing and editing and finalizing before moving ahead.
   2.1 - I took me 3 days just to find out my local host was the problem and my deepseek-coder llm had to have the word Latest or it would not work. and the errors information is limited.
   2.2 Custom -> Gemini -> Config file (you already have on github) -> make required changes, config, docker, env -> Test connection -> this step would have updated me to check my connection -> I worked eventually.
   2.3 Things got interesting when I tried to use Ollama service with deepseek-coder and I have been stuck since 3 days (I actually had the connection problem here using ollama not gemini)

The point is, the GUI we have is fair enough for a starter piece, the user needs to be updated along while connecting, because if the connection works, then its just the data we have to worry.

i hope this minor step would change so much, because you already have this with OpenAI setup.

and thankyou for the quick response.
Edit: i will share or open up a discussion with my current problems. (some json issue)

[realcarlos]

@paopa @qdrddr
Let me explain more about why it is confused:

in .env ,there are some key strings:

LLM_OPENAI_API_KEY=
EMBEDDER_OPENAI_API_KEY=
OPENAI_API_KEY=

in config.yaml:

type: embedder
provider: litellm_embedder
models:

define OPENAI_API_KEY=<api_key> in ~/.wrenai/.env if you are using openai embedding model

please refer to LiteLLM documentation for more details: https://docs.litellm.ai/docs/providers

I wonder which key is the right one.

[paopa]

Hi @ravenizzed and @realcarlos, thanks so much for your feedback! We’ll chat in the team about how to make the config step even easier. If you have any more thoughts on the config setup, I think we can keep commenting on this issue. It’ll be a great option for us to make it easier!

[ravenizzed]

@Nikita23526 i can share my env, config, docker file if you want

[Nikita23526]

@ravenizzed yes please

[ravenizzed]

@Nikita23526
Well, let's just discuss what is happening with you. I remembered there is a lot of updated information since I used Gemini and might share an irrelevant file that is already outdated, the config file in the example of GitHub is pretty self-explanatory but it is confusing nonetheless.

why don't you share your files, env, config, and docker so the devs can help you out as well.

I will resume with gemini in a couple of days, trying to setup dB via postgresSql to Redshift. So I can guide as best I can with the limited knowledge I have.

[Nikita23526]

@ravenizzed I have used ollama mistral but i was unable to understand that do i need to put mistral api key ....2025-03-19 11:27:32 wren-ai-service-1 | File "/src/globals.py", line 49, in create_service_container
2025-03-19 11:27:32 wren-ai-service-1 | **pipe_components["semantics_description"],
2025-03-19 11:27:32 wren-ai-service-1 | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-19 11:27:32 wren-ai-service-1 | KeyError: 'semantics_description'
2025-03-19 11:27:32 wren-ai-service-1 |
2025-03-19 11:27:32 wren-ai-service-1 | ERROR: Application startup failed. Exiting. and i am getting this error although i have ollama.mistral in semantics_description inside pipeline
My Config file

LLM Configuration (Ollama Mistral)

type: llm
provider: ollama
timeout: 120
models:

model: mistral
api_base: http://host.docker.internal:11434/ # Use http://localhost:11434/ if running outside Docker
kwargs:
temperature: 0
n: 1
max_tokens: 4096
seed: 0

Embedding Model Configuration

type: embedder
provider: ollama
models:

model: mxbai-embed-large
api_base: http://host.docker.internal:11434/
timeout: 120

Wren Engine Configuration

type: engine
provider: wren_ui
endpoint: http://wren-ui:3000/

---

Document Store Configuration

type: document_store
provider: qdrant
location: http://qdrant:6333/
embedding_model_dim: 3072
timeout: 120
recreate_index: true

---

AI Pipeline Configuration

type: pipeline
pipes:

name: db_schema_indexing
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: historical_question_indexing
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: table_description_indexing
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: db_schema_retrieval
llm: ollama.mistral
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: historical_question_retrieval
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: sql_generation
llm: ollama.mistral
engine: wren_ui
name: sql_correction
llm: ollama.mistral
engine: wren_ui
name: followup_sql_generation
llm: ollama.mistral
engine: wren_ui
name: sql_summary
llm: ollama.mistral
name: sql_answer
llm: ollama.mistral
name: sql_breakdown
llm: ollama.mistral
engine: wren_ui
name: sql_expansion
llm: ollama.mistral
engine: wren_ui
name: semantics_description
llm: ollama.mistral
name: relationship_recommendation
llm: ollama.mistral
engine: wren_ui
name: question_recommendation
llm: ollama.mistral
name: question_recommendation_db_schema_retrieval
llm: ollama.mistral
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: question_recommendation_sql_generation
llm: ollama.mistral
engine: wren_ui
name: intent_classification
llm: ollama.mistral
embedder: ollama_embedder.text-embedding-3-large
document_store: qdrant
name: data_assistance
llm: ollama.mistral
name: sql_pairs_indexing
document_store: qdrant
embedder: ollama_embedder.text-embedding-3-large
name: sql_pairs_retrieval
document_store: qdrant
embedder: ollama_embedder.text-embedding-3-large
llm: ollama.mistral
name: preprocess_sql_data
llm: ollama.mistral
name: sql_executor
engine: wren_ui
name: chart_generation
llm: ollama.mistral
name: chart_adjustment
llm: ollama.mistral
name: sql_question_generation
llm: ollama.mistral
name: sql_generation_reasoning
llm: ollama.mistral
name: sql_regeneration
llm: ollama.mistral
engine: wren_ui

General Settings

settings:
engine_timeout: 30
column_indexing_batch_size: 50
table_retrieval_size: 10
table_column_retrieval_size: 100
allow_using_db_schemas_without_pruning: false
query_cache_maxsize: 1000
query_cache_ttl: 3600
langfuse_host: https://cloud.langfuse.com/
langfuse_enable: true
logging_level: DEBUG
development: false

[ravenizzed]

@Nikita23526, well the first thing i see is the embeddings, try using phi4 as stated in the config logs, check if that works.
secondly, let me try mistrial myself to check the issue. Did you download the most updated config file for ollama.

try using nomic embeder.
and the chat model phi4, if it works let me know, if it doesn't share the logs on discord, you can get quick responses from the devs
logs for wren-ai-server, engine, ibis-server, ui

[Nikita23526]

@ravenizzed As mistral is supported so i think we can use it and yes i have used most updated config file yet facing the issue

[ravenizzed]

@Nikita23526 hey, sorry was a bit occupied, I still yet to test mistrial.
Do you have any update if Gemini is working for you?
From the updated config files you have to change a few things which I will share

The link of Google /v1/... Is not supported by litellm, if you can check litellms website to verify the link.

Second picture check the dim numbers for the embedding model you have chosen.

Type the exact name of the model from Google ai studio