Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Infra/restructure docker compose docs #102

Merged
merged 2 commits into from Mar 5, 2024
Merged

Infra/restructure docker compose docs #102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4e0475c
Restructure docker-compose docs
langchain-infra Mar 5, 2024
4ef6829
Restructure docker-compose docs
langchain-infra Mar 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
141 changes: 86 additions & 55 deletions docs/self_hosting/docker.mdx
View file
Open in desktop
Expand Up @@ -18,135 +18,166 @@ This guide provides instructions for installing and setting up your environment
```bash
docker info
```

If you don't see any server information in the output, make sure Docker is installed correctly
and launch the Docker daemon.
If you don't see any server information in the output, make sure Docker is installed correctly and launch the Docker daemon.
2. LangSmith License Key
1. You can get this from your Langchain representative. Contact us at support@langchain.dev for more information.
1. You can get this from your Langchain representative. Contact us at sales@langchain.dev for more information.
3. LangSmith Version
1. We default to the latest version of LangSmith. In a production environment, you should specify a specific version. Contact us at <a href="mailto:[email protected]">[email protected]</a> for more information.
1. Our Docker Compose files are pegged to the latest self-hosted release of LangSmith. If you want to use a different version, you can specify it in the `.env` file or in the `docker-compose.yml` file.
4. OpenAI API Key(optional).
1. Used for natural language search feature. Can specify OpenAI key in browser as well for the playground feature.
5. Oauth Configuration(optional).
1. You can configure oauth using the `.env` file. You will need to provide a `client_id` and `client_issuer_url` for your oauth provider.
2. Note, we do rely on the OIDC Authorization Code with PKCE flow. We currently support almost anything that is OIDC compliant however Google does not support this flow.
6. External Postgres(optional).
1. You can configure external postgres using the `.env` file. You will need to set the `POSTGRES_DATABASE_URI` environment variable to the connection string for your postgres instance.
2. If using a schema other than public, ensure that you do not have any other schemas with the pgcrypto extension enabled or you must include that in your search path.
3. Note: We do only officially support Postgres versions >= 14.
7. External Redis(optional).
1. You can configure external redis using the `.env` file. You will need to set the `REDIS_DATABASE_URI` environment variable to the connection string for your redis instance.
2. Currently, we do not support using Redis with TLS. We will be supporting this shortly.
3. We only official support Redis versions >= 6.

## Running via the LangSmith SDK

The following explains how to run the LangSmith using the LangSmith SDK.
## Running via Docker Compose

### 1. Install the LangSmith SDK
The following explains how to run the LangSmith using Docker Compose. This is the most flexible way to run LangSmith without Kubernetes. In production, we highly recommend using Kubernetes.

Both the Python and TypeScript LangSmith SDKs expose a `langsmith` command line tool.
### 1. Fetch the LangSmith `docker-compose.yml` file

First, install a recent version of the `langsmith` package:
You can find the `docker-compose.yml` file and related files in the LangSmith SDK repository here: [<u>LangSmith Docker Compose File</u>](https://github.com/langchain-ai/langsmith-sdk/blob/main/python/langsmith/cli/docker-compose.yaml)

- Python/Pip: `pip install -U langsmith`
- TypeScript/yarn: `yarn add langsmith`
- TypeScript/npm: `npm install -S langsmith`
- TypeScript/pnpm: `pnpm add langsmith`
Copy the `docker-compose.yml` file from the LangSmith SDK to your project directory. Make sure to copy the other files in the same directory as well.

This will install the LangSmith Client and the bundled command line tool.
### 2. Configure environment variables

Copy the `.env.example` file from the LangSmith SDK to your project directory and rename it to `.env`. Then, set the following environment variables in the `.env` file:

```bash
# Don't change this file. Instead, copy it to .env and change the values there. The default values will work out of the box as long as you provide your license key.
_LANGSMITH_VERSION=0.1.12 # Change to your desired LangSmith version. Required. Typically you should use the latest version defined in the .env file.
LANGSMITH_LICENSE_KEY=your-license-key # Change to your Langsmith license key. Required
OPENAI_API_KEY=your-openai-api-key # Needed for Online Evals and Magic Query features
AUTH_TYPE=none # Set to "oauth" if you want to use OAuth2.0
OAUTH_CLIENT_ID=your-client-id # Required if AUTH_TYPE=oauth
OAUTH_ISSUER_URL=https://your-issuer-url # Required if AUTH_TYPE=oauth
API_KEY_SALT=super # Change to your desired API key salt. Can be any random value. Must be set if AUTH_TYPE=oauth
POSTGRES_DATABASE_URI=postgres:postgres@langchain-db:5432/postgres # Change to your database URI if using external postgres. Otherwise, leave it as is
REDIS_DATABASE_URI=redis://langchain-redis:6379 # Change to your Redis URI if using external Redis. Otherwise, leave it as is
LOG_LEVEL=warning # Change to your desired log level
MAX_ASYNC_JOBS_PER_WORKER=10 # Change to your desired maximum async jobs per worker. We recommend 10/suggest spinning up more replicas of the queue worker if you need more throughput.
```

You can also set these environment variables in the `docker-compose.yml` file directly or export them in your terminal. We recommend setting them in the `.env` file.

### 2. Start server

Start the LangSmith tracing server by executing the following command in your terminal:
Start the LangSmith application by executing the following command in your terminal:

```bash
langsmith start --langsmith-license-key=<YOUR_LANGSMITH_LICENSE_KEY> --version=<LANGSMITH_VERSION> --openai-api-key=<YOUR_OPENAI_API_KEY>
docker-compose up
```

If you installed the JS SDK locally, the binary won't be automatically added to your path. In this case, you can start the server from the node_modules directory:
You can also run the server in the background by running:

```bash
node_modules/.bin/langchain
docker-compose up -d
```

If the server starts correctly, it will open up the UI in your browser at [http://localhost](http://localhost).
If the server starts correctly, it will open up the UI in your browser at [http://localhost](http://localhost/).

The LangSmith UI should be visible/operational and look like this:

![./static/langsmith_ui.png](./static/langsmith_ui.png)

### Stopping the server
To stop the server, run the following command:

```bash
langsmith stop
docker-compose down
```

### Checking the logs
If, at any point, you want to check if the server is running and see the logs, run

```bash
langsmith logs
docker-compose logs
```

## Running via Docker Compose

The following explains how to run the LangSmith using the LangSmith SDK.
## Running via the LangSmith SDK

### 1. Copy the LangSmith `docker-compose.yml` file
The following explains how to run the LangSmith using the LangSmith SDK. This is a convenient wrapper around the `docker-compose` command.
We recommend only using this in a local setting as it is not as flexible as using `docker-compose` directly.

Copy the `docker-compose.yml` file from the LangSmith SDK to your project directory.
### 1. Install the LangSmith SDK

You can find the `docker-compose.yml` file in the LangSmith SDK here: [<u>LangSmith Docker Compose File</u>](https://github.com/langchain-ai/langsmith-sdk/blob/main/python/langsmith/cli/docker-compose.yaml)
Both the Python and TypeScript LangSmith SDKs expose a `langsmith` command line tool.

### 2. Export or set the required environment variables
First, install a recent version of the `langsmith` package:

```bash
export LANGSMITH_LICENSE_KEY=<YOUR_LANGSMITH_LICENSE_KEY>
export _LANGSMITH_IMAGE_VERSION=<LANGSMITH_VERSION>
export OPENAI_API_KEY=<YOUR_OPEN_OPENAI_API_KEY> # Optional, used for natural language search feature
```
- Python/Pip: `pip install -U langsmith`
- TypeScript/yarn: `yarn add langsmith`
- TypeScript/npm: `npm install -S langsmith`
- TypeScript/pnpm: `pnpm add langsmith`

You can also set these environment variables in the `docker-compose.yml` file directly.
This will install the LangSmith Client and the bundled command line tool.

### 2. Start server

Start the LangSmith application by executing the following command in your terminal:
Start the LangSmith tracing server by executing the following command in your terminal:

```bash
docker-compose up
langsmith start --langsmith-license-key=<YOUR_LANGSMITH_LICENSE_KEY> --version=<LANGSMITH_VERSION> --openai-api-key=<YOUR_OPENAI_API_KEY>
```

You can also run the server in the background by running:
If you installed the JS SDK locally, the binary won't be automatically added to your path. In this case, you can start the server from the node_modules directory:

```bash
docker-compose up -d
node_modules/.bin/langchain
```

If the server starts correctly, it will open up the UI in your browser at [http://localhost](http://localhost/).
If the server starts correctly, it will open up the UI in your browser at [http://localhost](http://localhost).

The LangSmith UI should be visible/operational and look like this:

![./static/langsmith_ui.png](./static/langsmith_ui.png)

### Stopping the server

To stop the server, run the following command:
```bash
docker-compose down
langsmith stop
```

### Checking the logs
If, at any point, you want to check if the server is running and see the logs, run

```bash
docker-compose logs
langsmith logs
```

## Using LangSmith
Now that LangSmith is running, you can start using it to trace your code. You can find more information on how to use self-hosted LangSmith in the [Self-Hosted Usage Guide](/self_hosting/usage).

## FAQ:

### How can we upgrade our application?
We plan to release new minor versions of the LangSmith application every 6 weeks. This will include release notes and all changes should be backwards compatible. To upgrade, you will need to restart your LangSmith instance specifying the new version.
### How can we back up our application?
The docker solution uses docker volumes to store data. You can back up the data by backing up the volumes.
### How can we authenticate to the application?
Currently, our docker solution does not include any authentication. We recommend setting up a proxy to handle authentication.
### What networking configuration is needed for the application?
### Frequently Asked Questions

#### How can we upgrade our application?
- We plan to release new minor versions of the LangSmith application every 6 weeks. This will include release notes and all changes should be backwards compatible. To upgrade, you will need to restart your LangSmith instance specifying the new version.
#### How can we back up our application?
- The docker solution uses docker volumes to store data. You can back up the data by backing up the volumes.
#### How can we authenticate to the application?
- Currently, our self-hosted solution supports oauth as an authn solution.
- Note, we do offer a no-auth solution but highly recommend setting up oauth before moving into production.
#### How can I use External `Postgres` or `Redis`?
- You can configure external postgres or redis using the external sections in the `.env` file. You will need to provide the connection url/params for the database/redis instance. Look at the `.env.example` file for more information.
#### What networking configuration is needed for the application?
- Our deployment only needs egress for a few things:
- Fetching images (If mirroring your images, this may not be needed)
- Talking to any LLMs
- Your VPC can set up rules to limit any other access.
- Note: We require the X-Tenant-Id to be allowed to be passed through to the backend service. This is used to determine which tenant the request is for.
### What resources should we allocate to the application?
We recommend at least 4 vCPUs and 16GB of memory for our application. This is a rough estimate and can vary based on the number of users and the size of the data you are working with.
#### What resources should we allocate to the application?
- We recommend at least 4 vCPUs and 16GB of memory for our application. This is a rough estimate and can vary based on the number of users and the size of the data you are working with.
#### Increasing the number of replicas
- If you need more throughput, you can increase the number of replicas of the queue worker by running the following command:
```bash
docker-compose up --scale langchain-queue=5
```
This will start 5 replicas of the queue worker. You can change the number to whatever you need. Note that these containers are fairly CPU intensive, and you should ensure you have enough resources to support the number of replicas you are starting.
30 changes: 15 additions & 15 deletions docs/self_hosting/kubernetes.mdx
View file
Open in desktop
Expand Up @@ -39,7 +39,7 @@ Ensure you have the following tools/items ready. Some items are marked optional
2. Helm
1. `brew install helm`
3. LangSmith License Key
1. You can get this from your Langchain representative. Contact us at support@langchain.dev for more information.
1. You can get this from your Langchain representative. Contact us at sales@langchain.dev for more information.
4. SSL(optional)
1. This should be attachable to a load balancer that will be provisioned by your cloud provider. This will be used for the frontend service.
5. OpenAI API Key(optional).
Expand Down Expand Up @@ -150,26 +150,26 @@ config:
## Using LangSmith
Now that LangSmith is running, you can start using it to trace your code. You can find more information on how to use self-hosted LangSmith in the [Self-Hosted Usage Guide](/self_hosting/usage).

## FAQ:

### How can we upgrade our application?
We plan to release new minor versions of the LangSmith application every 6 weeks. This will include release notes and all changes should be backwards compatible. To upgrade, you will need to follow the upgrade instructions in the Helm README and run a `helm upgrade langsmith --values <values file>`
### How can we back up our application?
Currently, we rely on PVCs/PV to power storage for our application. We strongly encourage setting up `Persistent Volume` backups or moving to a managed service for `Postgres` to support disaster recovery
### How does load balancing work/ingress work?
Currently, our application spins up one load balancer using a k8s service of type `LoadBalancer` for our frontend. If you do not want to set up a load balancer you can simply port-forward the frontend and use that as your external ip for the application. We also have an option for the chart to provision an ingress resource for the application.
### How can we authenticate to the application?
Currently, our self-hosted solution supports oauth as an authn solution. Note, we do offer a no-auth solution but highly recommend setting up oauth before moving into production.
### How can I use External `Postgres` or `Redis`?
You can configure external postgres or redis using the external sections in the `values.yaml` file. You will need to provide the connection url/params for the database/redis instance. Look at the configuration above example for more information.
### What networking configuration is needed for the application?
### Frequently Asked Questions:

#### How can we upgrade our application?
- We plan to release new minor versions of the LangSmith application every 6 weeks. This will include release notes and all changes should be backwards compatible. To upgrade, you will need to follow the upgrade instructions in the Helm README and run a `helm upgrade langsmith --values <values file>`
#### How can we back up our application?
- Currently, we rely on PVCs/PV to power storage for our application. We strongly encourage setting up `Persistent Volume` backups or moving to a managed service for `Postgres` to support disaster recovery
#### How does load balancing work/ingress work?
- Currently, our application spins up one load balancer using a k8s service of type `LoadBalancer` for our frontend. If you do not want to set up a load balancer you can simply port-forward the frontend and use that as your external ip for the application. We also have an option for the chart to provision an ingress resource for the application.
#### How can we authenticate to the application?
- Currently, our self-hosted solution supports oauth as an authn solution. Note, we do offer a no-auth solution but highly recommend setting up oauth before moving into production.
#### How can I use External `Postgres` or `Redis`?
- You can configure external postgres or redis using the external sections in the `values.yaml` file. You will need to provide the connection url/params for the database/redis instance. Look at the configuration above example for more information.
#### What networking configuration is needed for the application?
Our deployment only needs egress for a few things:
- Fetching images (If mirroring your images, this may not be needed)
- Talking to any LLMs
- Talking to any external services you may have configured
- Fetching OAuth information
Your VPC can set up rules to limit any other access. Note: We require the X-Tenant-Id to be allowed to be passed through to the backend service. This is used to determine which tenant the request is for.
### What resources should we allocate to the application?
#### What resources should we allocate to the application?
- We recommend at least 4 vCPUs and 16GB of memory for our application.
- We have some default resources set in our `values.yaml` file. You can override these values to tune resource usage for your organization.
- If the metrics server is enabled in your cluster, we also recommend enabling autoscaling on all deployments.