Local Docker Deployment & Instructions #1193

davidamacey · 2024-01-16T12:59:16Z

Docker Deployment

In order to serve this application on my system I prefer Docker Instances for ease of deployment. In an order to setup easily, I created a Makefile, Scripts, and commands to run from the console to increase deployment flexibility.

This includes a docker build file for the chatbot-ui, as well as a docker-compose file for deployment of the UI and ollama.

I did my best to model your setup from the legacy branch.

Makefile

Using a Makefile, we can easily run commands to deploy, stop, and remove all or parts of the deployment.

Scripts

New scripts folder includes automatic ollama install and Supabase cli for ease of use. These may need to be updated in the future for versions and their repo updates. In the meantime, they will do the job.

Readme Updated

Updated instructions for using the Makefile with notes necessary for deployment.

Issue #1088 Bug

In order to get the correct output I added the streaming: false parameter per Issue post. The output is correct, we just don't see the streaming output.

Notes

I did my best to streamline the local deployment. I am open to suggestions and corrections if someone sees a better process or method.

I tested the docker deployment on Ubuntu 22.04 with GPUs.

Thank you!

Your hardworking on this repo is amazing and very much appreciated!

emsi · 2024-01-16T17:41:32Z

Having SUPABASE_SERVICE_ROLE_KEY or any other secrets in the repo code is never a good thing.
You should think how to work around it.

davidamacey · 2024-01-17T02:54:37Z

@emsi Thank you for the comment. I understand your concern. I am using the standard example provided by Supabase's repo Supabase env.example

This simplifies the basic setup. The user/developer can adjust their security settings as they see fit once deployed.

Otherwise we would use a customized .env file.

Thank you for the feedback.

Bortus-AI · 2024-01-17T17:40:33Z

@davidamacey
I am getting this error on a fresh checkout of the repo and this pull request. on my Debian 12 server.

docker compose up -d
  [chatbot-build-ui 7/7] RUN npm run build:
......
22.14 
**22.14 info  - Need to disable some ESLint rules? Learn more here: https://nextjs.org/docs/basic-features/eslint#disabling-rules
23.27 Failed to compile.
23.27 
23.27 ./components/sharing/add-to-workspace.tsx:155:20
23.27 Type error: Type '"tools"' does not satisfy the constraint '"assistants" | "workspaces" | "folders" | "chats" | "files" | "collections" | "file_items" | "messages" | "presets" | "prompts" | { schema: keyof Database; } | "assistant_workspaces" | ... 7 more ... | "prompt_workspaces"'.**
23.27 
23.27   153 |     },
23.27   154 |     tools: async (
23.27 > 155 |       item: Tables<"tools">,
23.27       |                    ^
23.27   156 |       workspaceId: string,
23.27   157 |       userId: string
23.27   158 |     ) => {
------
failed to solve: executor failed running [/bin/sh -c npm run build]: exit code: 1
make: *** [Makefile:41: start-docker] Error 17

davidamacey · 2024-01-17T22:53:00Z

@Bortus-AI The error you see is from the types.ts file within the supabase folder. The types.ts file is autogenerate from the gen-datatypes command.

I suggest you re-run make generate-db-types from the repo, then re-run the make start-chatbot-docker

FYI I am working on updates to the docker compose to be more efficient in connecting to the Supabase containers.

Thank you for trying out my PR. Let me know if you have more questions.

Bortus-AI · 2024-01-17T23:22:03Z

make generate-db-type

Happy to help and look forward to the next version. Here is what happened with those commands

bortus@ny:~/chatbot-ui$ make generate-db-types
Generating database types...
supabase gen types typescript --local > supabase/types.ts
(node:1) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
.................................................
 > [chatbot-build-ui 7/7] RUN npm run build:                                                                                                                 
0.376                                                                                                                                                        
0.376 > [email protected] build                                                                                                                               
0.376 > next build       
.................................................
23.16 ./components/sharing/add-to-workspace.tsx:155:20
23.16 Type error: Type '"tools"' does not satisfy the constraint '"assistants" | "workspaces" | "folders" | "chats" | "files" | "collections" | "file_items" | "messages" | "presets" | "prompts" | { schema: keyof Database; } | "assistant_workspaces" | ... 7 more ... | "prompt_workspaces"'.
23.16 
23.16   153 |     },
23.16   154 |     tools: async (
23.16 > 155 |       item: Tables<"tools">,
23.16       |                    ^
23.16   156 |       workspaceId: string,
23.16   157 |       userId: string
23.16   158 |     ) => {
------
failed to solve: executor failed running [/bin/sh -c npm run build]: exit code: 1
make: *** [Makefile:41: start-docker] Error 17```

davidamacey · 2024-01-18T00:27:45Z

I attempted to reorganize the docker-compose to leverage the Supabase containers and docker network. Unfortunately, the front-end container struggles to connect. I must use the 'network_mode: host' for the UI container. This is not advised, but otherwise it is not able to see the supabase_kong_chatbotui container. Additionally the front-end throughs exceptions and errors stating it cannot reach the container. The Supabase setup is automated through the cli, so it is difficult to incorporate into one docker-compose/network.

Unfortunately, I have hit my knowledge limit of Docker for this particular task.

This pull request allows you to build a fully Docker application accessible on local machine and accessible on other machines on the network via port forward.

Best Regards, David

davidamacey · 2024-01-18T01:02:38Z

make generate-db-type

Happy to help and look forward to the next version. Here is what happened with those commands

bortus@ny:~/chatbot-ui$ make generate-db-types
Generating database types...
supabase gen types typescript --local > supabase/types.ts
(node:1) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
.................................................
 > [chatbot-build-ui 7/7] RUN npm run build:                                                                                                                 
0.376                                                                                                                                                        
0.376 > [email protected] build                                                                                                                               
0.376 > next build       
.................................................
23.16 ./components/sharing/add-to-workspace.tsx:155:20
23.16 Type error: Type '"tools"' does not satisfy the constraint '"assistants" | "workspaces" | "folders" | "chats" | "files" | "collections" | "file_items" | "messages" | "presets" | "prompts" | { schema: keyof Database; } | "assistant_workspaces" | ... 7 more ... | "prompt_workspaces"'.
23.16 
23.16   153 |     },
23.16   154 |     tools: async (
23.16 > 155 |       item: Tables<"tools">,
23.16       |                    ^
23.16   156 |       workspaceId: string,
23.16   157 |       userId: string
23.16   158 |     ) => {
------
failed to solve: executor failed running [/bin/sh -c npm run build]: exit code: 1
make: *** [Makefile:41: start-docker] Error 17```

Sorry I looked over your response.

I have this similar issue once before. Try renaming the types.ts file to something else. Be sure you are in the repo directory, supabase running, then run the gem-types command. This should generate a new types.ts file.

Note, I had troubles similar as I pulled before a large commit was done, but I thought I fixed the types file issue.

jmtatsch · 2024-01-18T08:32:08Z

I attempted to reorganize the docker-compose to leverage the Supabase containers and docker network. Unfortunately, the front-end container struggles to connect. I must use the 'network_mode: host' for the UI container.

Did you base your attempt on this compose file https://github.com/supabase/supabase/blob/master/docker/docker-compose.yml ?
Could you publish your attempt maybe on another branch?

The Supabase setup is automated through the cli, so it is difficult to incorporate into one docker-compose/network.

That also stopped my first attempt. But without its not really a dockerized solution.

davidamacey · 2024-01-18T13:29:22Z

@jmtatsch Thank you for taking a look, maybe with a second set of eyes we can figured it out.

In my attempt I still utilize the Supabase CLI to start the backend, but leverage the docker network to connect to the Supabase containers.

Docker Attempt Notes

I use the docker-compose from this PR to build the UI and ollama containers. I commented out the with 'ATTEMPT DOCKER COMOSE' sections

I edit the .env file to use the container names in the URL, see below for .env issues.

If the chatbot-ui is not in 'network_mode: 'host'' then the app will not render in the UI. There seems to be a weird connection issue with the frontend and the kong container. Because within the chatbot-ui container I can curl to Ollama and kong containers.

I suspect there are some variables or settings in the chatbot-ui frontend that references specific connection types, but I am not a UI dev, so I do not know how to troubleshoot.

Verified with the network setup that all containers are on the same network, therefore they should all be referenced by their container names instead of localhost to 127...

From supabase

Yes I attempted to create the docker-compose from the Supabase repo. I used the .toml from the chatbot-ui and created a docker-compose file. This creates an issue of starting the Supabase containers and initializing with the chatbotbui migration scripts. I could not find documentation to start a docker setup from scratch. From the docs, you would need to have a local supabase running, then migrate that to a 'cloud' version (or running docker setup). This attempt was unsuccessful.

Weird ENV Issue

In an attempt to centralize the .env file and read into the UI container via docker env, it will not run properly unless the .env file is built into the UI container. I am not sure where/how the front end is referencing the specific file. So it must be rebuilt if there is an .env change.

Changes

Pushed these updates to my fork.

.env has a commented section
docker-compose for chatbot-ui has a commented section
docker-compose-supabase file is configured from the supabase repo
added a few make commands to speed up testing

I have attempt many permutations of these changes above and the only way for the frontend to connect to the supabase is through network_mode = host. I even tried an NGINX proxy in front of the localhost version. That presents it own issues from the frontend, as it references localhost urls and there is a header mismatch; see env issue above.

Screenshot

Attached is a UI screenshot from browser when using the docker-compose setup using none network_mode host and URLs of container names. The frontend cannot connect to the kong containers.

I hope this explanation was useful or helpful, but I don't see a way forward at the moment.

Bortus-AI · 2024-01-18T18:21:52Z

@davidamacey

I was able to fix it. I had to update nodejs and then all worked

Bortus-AI · 2024-01-18T18:35:56Z

@davidamacey
Any flags to have it skip trying to run/install ollama?

davidamacey · 2024-01-18T23:55:40Z

@davidamacey Any flags to have it skip trying to run/install ollama?

in the make file there is a command make start-ui

This assumes supabase and ollama are already running. You can use an any of the commands in the Makefile in any order to customize your needs; or make new ones. :)

Glad you were able to get your issue fixed!

Bortus-AI · 2024-01-19T15:58:43Z

@davidamacey Thanks for that command. Whether I use it or make start-chatbot-docker the container chatbot-ui-chatbot-build-ui-1 starts but then exits with an error code. Logs below

> [email protected] start
> next start
   ▲ Next.js 14.0.2
   - Local:        http://localhost:3000
Error: ENOENT: no such file or directory, open '/usr/src/app/.next/BUILD_ID'
    at async open (node:internal/fs/promises:633:25)
    at async Object.readFile (node:internal/fs/promises:1242:14)
    at async setupFsCheck (/usr/src/app/node_modules/next/dist/server/lib/router-utils/filesystem.js:145:19)
    at async initialize (/usr/src/app/node_modules/next/dist/server/lib/router-server.js:54:23)
    at async Server.<anonymous> (/usr/src/app/node_modules/next/dist/server/lib/start-server.js:236:36) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/usr/src/app/.next/BUILD_ID'
}

h3xcat · 2024-01-29T07:37:30Z

I am wondering if it's possible to fully encapsulate both Supabase and the app within a single container. This approach would aim to somewhat replicate the simplicity of V1 deployment. In its current form, V2 requires users to dedicate a significant amount of time to understanding how Supabase functions and to implementing it properly within their local environments.

It might be worth exploring the s6 overlay for this purpose.

henrywithu · 2024-01-29T09:52:58Z

I am wondering if it's possible to fully encapsulate both Supabase and the app within a single container. This approach would aim to somewhat replicate the simplicity of V1 deployment. In its current form, V2 requires users to dedicate a significant amount of time to understanding how Supabase functions and to implementing it properly within their local environments.

It might be worth exploring the s6 overlay for this purpose.

Totally agree

jollytoad · 2024-02-02T14:04:08Z

I've also built this to run in a docker container, and hit some of the same issues. Anyway I hope this may help anyone else going down the same path. I'm not a next.js expert btw, so what I'm suggesting may not be the true next.js way!

Here's my Dockerfile for a start... https://github.com/jollytoad/chatbot-ui/blob/org/Dockerfile

I also had the problem with env vars, which it seems that next.js wants to compile inline, which I really don't like the idea of one bit. So I inject the vars at runtime into the html doc, and read them from there...

This allows the built container to be independent of the supabase instance at least.

Also, I had a problem with the fact that requests appear to come from localhost:3000, which can cause redirection responses and other URL constructions to fail. So I created a publicUrl fn that recreates the original URL from x-forwarded headers, and use that where necessary, although I guess this could also be done in the middleware for all requests.

https://github.com/jollytoad/chatbot-ui/blob/org/lib/public-url.ts

I'd be interested if anyone knows a better way to deal with these issues.

PaulWoitaschek · 2024-02-08T15:33:05Z

Linking this as it's related:

#1376 (comment)

There's a version with sqlite and simplified auth in the works that'll ship with this.

Expect that within 2-3 weeks. A bit swamped but it's in progress.

mckaywrigley · 2024-02-18T13:14:28Z

Linking this as it's related:

#1376 (comment)

There's a version with sqlite and simplified auth in the works that'll ship with this.
Expect that within 2-3 weeks. A bit swamped but it's in progress.

There's been a massive influx of enterprise demand, so lots going on to try and get feedback from teams on how to best go about this.

Trying to strike a balance between simplicity and power.

guenhter · 2024-02-22T15:20:02Z

.env.local

I guess the .env.local file should not be committed to the repo

or a .env.local.example

the .env.local

As far as I understood, this file should be a .gitignored file which contains the API_KEYS and the like for the local environment without always having it popping up in the git changes.

guenhter · 2024-02-22T15:23:02Z

Makefile

Wouldn't it make sense to add all those commands simply to the package.json so that npm is the only tool (instead of introducing make)?

PathToLife · 2024-02-27T03:16:44Z

Dockerfile

@@ -0,0 +1,38 @@
+# Use the Node.js official image as a parent image
+# FROM node:20
+FROM node:21.5.0-bookworm


Should this be node:20-bookworm?

Node 20 is the current LTS version

kpi88 · 2024-02-29T00:04:48Z

Hi everyone, after a couple weeks of on and off work, I've managed to shrink this whole thing to just one docker-compose with 8 extra files. The only catch is that it requires some kind of DNS. I made it a separate project here as I think it's currently too unwieldy to merge, and I imagine too different from what will eventually appear here officially. Please try it out if it serves your needs and if you can, deliver feedback, I would appreciate it.

MAS-CreativeLabs · 2024-03-01T05:31:10Z

Thanks @kpi88 you did great! I'm going to be able to try chatbot-ui V2 finally 🙏

Aj7Ay · 2024-03-10T05:48:18Z

Anyone tried this, i got this error
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy' nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown make: *** [Makefile:44: start-docker] Error 1

Aj7Ay · 2024-03-14T14:18:00Z

Hello Guys I tried to deploy the chatbot UI on Server
Here are steps i followed

Installed supabase cli https://github.com/supabase/cli/releases
wget https://github.com/supabase/cli/releases/download/v1.149.4/supabase_1.149.4_linux_amd64.deb

sudo dpkg -i supabase_1.149.4_linux_amd64.deb

Download this repository

cd chatbot-ui
cd supabase

update config.toml with your server IP address at three places and update migrations file with same server Ip

cd .. (in supabase Directory )

Run this command
supabase start

it will start all the containers that are required

Come back to chatbot UI

Create a Dockerfile
Before that update .env file with server IP as url (Not localhost)

FROM node:21.5.0-bookworm

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm cache clean -f
RUN npm ci

COPY . .

RUN npm run build

ENV NODE_ENV production

EXPOSE 3000

CMD ["npm", "start"]

docker build -t chatbot .

docker run -d --name bot -p 3000:3000 chatbot:latest

Its wokring with Ip
when we add Dns name getting cors issue. Help needed for this

ruizcrp · 2024-04-08T13:05:59Z

Hi, just as info, I tried @kpi88 solution. You mention under https://github.com/kpi88/chatbot-docker#known-issues issues with podman. Actually I get the same issues with docker: The unhealthy containers; the ulimits-attribute but also the name-attribute at the beginning and the tags attribute. When removing the unknown attributes, I get the unhealthy containers. Thus, it might not be specifically linked to podman.

Aj7Ay · 2024-04-08T13:09:23Z

Hi, just as info, I tried @kpi88 solution. You mention under https://github.com/kpi88/chatbot-docker#known-issues issues with podman. Actually I get the same issues with docker: The unhealthy containers; the ulimits-attribute but also the name-attribute at the beginning and the tags attribute. When removing the unknown attributes, I get the unhealthy containers. Thus, it might not be specifically linked to podman.

Okay
but we deployed the Chatbot 2.0 with docker
using self hosted supabase

ruizcrp · 2024-04-08T15:44:27Z

@Aj7Ay with your solution, I follow it literally an then get:

supabase start
supabase start is already running.
Run supabase status to show status of local Supabase containers.

supabase status
supabase_db_chatbotui container is not running: exited

davidamacey added 3 commits January 16, 2024 07:41

setup docker deployment, makefile, scripts, streaming bug

d662621

merge with packages

8e628e5

spelling error

91477f7

docker compose attempts

829291a

arduilex mentioned this pull request Jan 20, 2024

Docker deployment #1060

Closed

GlennSvanberg approved these changes Jan 25, 2024

View reviewed changes

wrapss mentioned this pull request Jan 30, 2024

Using Supabase Docker already deployed #1087

Closed

guenhter reviewed Feb 22, 2024

View reviewed changes

Digital39999 mentioned this pull request Feb 22, 2024

Docker container for chatbot-ui v2 self-hosted deployment #1458

Open

guenhter mentioned this pull request Feb 23, 2024

Add simple dockerfile #1476

Closed

PathToLife reviewed Feb 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Local Docker Deployment & Instructions #1193

Local Docker Deployment & Instructions #1193

davidamacey commented Jan 16, 2024

emsi commented Jan 16, 2024

davidamacey commented Jan 17, 2024

Bortus-AI commented Jan 17, 2024

davidamacey commented Jan 17, 2024 •

edited

Bortus-AI commented Jan 17, 2024

davidamacey commented Jan 18, 2024

davidamacey commented Jan 18, 2024

jmtatsch commented Jan 18, 2024

davidamacey commented Jan 18, 2024 •

edited

Bortus-AI commented Jan 18, 2024

Bortus-AI commented Jan 18, 2024

davidamacey commented Jan 18, 2024 •

edited

Bortus-AI commented Jan 19, 2024

h3xcat commented Jan 29, 2024

henrywithu commented Jan 29, 2024

jollytoad commented Feb 2, 2024

PaulWoitaschek commented Feb 8, 2024

mckaywrigley commented Feb 18, 2024

guenhter Feb 22, 2024

heralight Feb 22, 2024

guenhter Feb 22, 2024

guenhter Feb 22, 2024

PathToLife Feb 27, 2024

kpi88 commented Feb 29, 2024

MAS-CreativeLabs commented Mar 1, 2024

Aj7Ay commented Mar 10, 2024

Aj7Ay commented Mar 14, 2024 •

edited

ruizcrp commented Apr 8, 2024

Aj7Ay commented Apr 8, 2024

ruizcrp commented Apr 8, 2024

Local Docker Deployment & Instructions #1193

Are you sure you want to change the base?

Local Docker Deployment & Instructions #1193

Conversation

davidamacey commented Jan 16, 2024

Docker Deployment

Makefile

Scripts

Readme Updated

Issue #1088 Bug

Notes

Thank you!

emsi commented Jan 16, 2024

davidamacey commented Jan 17, 2024

Bortus-AI commented Jan 17, 2024

davidamacey commented Jan 17, 2024 • edited

Bortus-AI commented Jan 17, 2024

davidamacey commented Jan 18, 2024

davidamacey commented Jan 18, 2024

jmtatsch commented Jan 18, 2024

davidamacey commented Jan 18, 2024 • edited

Docker Attempt Notes

From supabase

Weird ENV Issue

Changes

Screenshot

Bortus-AI commented Jan 18, 2024

Bortus-AI commented Jan 18, 2024

davidamacey commented Jan 18, 2024 • edited

Bortus-AI commented Jan 19, 2024

h3xcat commented Jan 29, 2024

henrywithu commented Jan 29, 2024

jollytoad commented Feb 2, 2024

PaulWoitaschek commented Feb 8, 2024

mckaywrigley commented Feb 18, 2024

guenhter Feb 22, 2024

Choose a reason for hiding this comment

heralight Feb 22, 2024

Choose a reason for hiding this comment

guenhter Feb 22, 2024

Choose a reason for hiding this comment

guenhter Feb 22, 2024

Choose a reason for hiding this comment

PathToLife Feb 27, 2024

Choose a reason for hiding this comment

kpi88 commented Feb 29, 2024

MAS-CreativeLabs commented Mar 1, 2024

Aj7Ay commented Mar 10, 2024

Aj7Ay commented Mar 14, 2024 • edited

ruizcrp commented Apr 8, 2024

Aj7Ay commented Apr 8, 2024

ruizcrp commented Apr 8, 2024

davidamacey commented Jan 17, 2024 •

edited

davidamacey commented Jan 18, 2024 •

edited

davidamacey commented Jan 18, 2024 •

edited

Aj7Ay commented Mar 14, 2024 •

edited