Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address more feedback from docs rollout #27670

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from
Draft

Address more feedback from docs rollout #27670

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8a9d0d9
quick fix for branch deployments doc
neverett Feb 7, 2025
5a153dd
Merge branch 'master' into nikki/docs/more-feedback
neverett Feb 23, 2025
0b08fde
remove reference to github/gitlab integration and remove example code…
neverett Feb 23, 2025
02e680d
add missing alert policies
neverett Feb 23, 2025
2b3fb79
small edit
neverett Feb 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 54 additions & 21 deletions docs/docs/dagster-plus/features/alerts/creating-alerts.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ title: Creating alerts in Dagster+
sidebar_position: 200
---

You can create alerts in the Dagster+ UI or using the [`dagster-cloud` CLI](/dagster-plus/deployment/management/dagster-cloud-cli).

{/* TODO link to dagster-cloud CLI tool doc */}
You can create alerts in the Dagster+ UI or with the [`dagster-cloud` CLI](/dagster-plus/deployment/management/dagster-cloud-cli).

:::note

Before you create alerts, you must [configure an alert notification service](configuring-an-alert-notification-service).

:::

## Alerting when a run fails

You can set up alerts to notify you when a run fails.

By default, these alerts will target all runs in the deployment, but they can be scoped to runs with a specific tag.
Expand All @@ -25,9 +26,6 @@ By default, these alerts will target all runs in the deployment, but they can be
6. Choose a notification channel for your Run alert and click **Continue**.
7. Review and save your Run alert and click **Save alert**.


5. Select **Job failure**.

If desired, add **tags** in the format `{key}:{value}` to filter the runs that will be considered.

</TabItem>
Expand Down Expand Up @@ -56,9 +54,11 @@ If desired, add **tags** in the format `{key}:{value}` to filter the runs that w
</Tabs>

## Alerting when a run is taking too long to complete

You can set up alerts to notify you whenever a run takes more than some threshold amount of time.

By default, these alerts will target all runs in the deployment, but they can be scoped to runs with a specific tag.
By default, these alerts will target all runs in the deployment, but they can be scoped to runs with a specific tag.

<Tabs groupId="ui_or_cli">
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
Expand All @@ -69,8 +69,6 @@ You can set up alerts to notify you whenever a run takes more than some threshol
6. Choose a notification channel for your Run alert and click **Continue**.
7. Review and save your Run alert and click **Save alert**.

5. Select **Job running over** and how many hours to alert after.

If desired, add **tags** in the format `{key}:{value}` to filter the runs that will be considered.

</TabItem>
Expand Down Expand Up @@ -99,6 +97,7 @@ If desired, add **tags** in the format `{key}:{value}` to filter the runs that w
</Tabs>

## Alerting when an asset fails to materialize

You can set up alerts to notify you when an asset materialization attempt fails.

By default, these alerts will target all assets in the deployment, but they can be scoped to a specific asset or group of assets.
Expand All @@ -110,8 +109,8 @@ If using a RetryPolicy, an alert will only be sent after all retries complete.
:::

<Tabs groupId="ui_or_cli">
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
2. Click the **Alert policies** tab.
3. Click **Create alert policy**.
4. Select **Asset** from the menu and click **Continue**.
Expand Down Expand Up @@ -146,12 +145,13 @@ If using a RetryPolicy, an alert will only be sent after all retries complete.
</Tabs>

## Alerting when an asset check fails

You can set up alerts to notify you when an asset check on an asset fails.

By default, these alerts will target all assets in the deployment, but they can be scoped to checks on a specific asset or group of assets.
<Tabs groupId="ui_or_cli">
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
2. Click the **Alert policies** tab.
3. Click **Create alert policy**.
4. Select **Asset** from the menu and click **Continue**.
Expand Down Expand Up @@ -185,12 +185,13 @@ By default, these alerts will target all assets in the deployment, but they can
</Tabs>

## Alerting when a schedule or sensor tick fails

You can set up alerts to fire when any schedule or sensor tick across your entire deployment fails.

Alerts are sent only when a schedule/sensor transitions from **success** to **failure**, so only the initial failure will trigger the alert.
<Tabs groupId="ui_or_cli">
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
2. Click the **Alert policies** tab.
3. Click **Create alert policy**.
4. Select **Automation** from the menu and click **Continue**.
Expand Down Expand Up @@ -223,10 +224,11 @@ Alerts are sent only when a schedule/sensor transitions from **success** to **fa
</Tabs>

## Alerting when a code location fails to load

You can set up alerts to fire when any code location fails to load due to an error.
<Tabs groupId="ui_or_cli">
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
2. Click the **Alert policies** tab.
3. Click **Create alert policy**.
4. Select **Code Location** from the menu and click **Continue**.
Expand Down Expand Up @@ -259,14 +261,17 @@ You can set up alerts to fire when any code location fails to load due to an err
</Tabs>

## Alerting when a Hybrid agent becomes unavailable

:::note
This is only available for [Hybrid](/dagster-plus/deployment/deployment-types/hybrid/) deployments.

Alerting when a Hybrid agent becomes unavailable is only available for [Hybrid deployments]((/dagster-plus/deployment/deployment-types/hybrid/)).

:::

You can set up alerts to fire if your Hybrid agent hasn't sent a heartbeat in the last 5 minutes.
<Tabs groupId="ui_or_cli">
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
<TabItem value='ui' label='In the UI'>
1. In the Dagster UI, click **Deployment**.
2. Click the **Alert policies** tab.
3. Click **Create alert policy**.
4. Select **Code Location** from the menu and click **Continue**.
Expand Down Expand Up @@ -296,4 +301,32 @@ You can set up alerts to fire if your Hybrid agent hasn't sent a heartbeat in th
</Tabs>

</TabItem>
</Tabs>
</Tabs>

## Alerting when a Dagster+ Insights metric crosses a specified threshhold (Experimental)

Sends a notification when a [Dagster+ Insights](/dagster-plus/features/insights/) metric exceeds or
falls below a specified threshold over a specified time window. This can be used to alert on:

- Dagster credit usage across a deployment or for a specific job
- Performance regressions on asset or job runtime
- Spend on external tools such as Snowflake or BigQuery credits

Alerts can be scoped to the sum of any metric across an entire deployment, or for a specific job, asset group, or asset key.

:::note

Alerts are sent only when the threshold is first crossed, and will not be sent again until the value returns to expected levels. Insights data may become available up to 24 hours after run completion.

:::

## Alerting when your organization has reached its monthly credit limit (Experimental)

:::info Availability

This alert is only available on [Serverless deployments](/dagster-plus/deployment/deployment-types/serverless/).

:::

Each newly created organization on Dagster+ Serverless starts with an alert policy of this type, configured to notify the email address used to create the organization.

64 changes: 31 additions & 33 deletions ...dagster-plus/features/ci-cd/branch-deployments/setting-up-branch-deployments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,8 @@ Using this approach to branch deployments may be a good fit if:
- You use **GitHub** for version control
- You want Dagster to fully automate Branch Deployments

This approach is simplified if you use the [GitHub integration](/integrations/libraries/github) to import your project into Dagster+.

</TabItem>
<TabItem value="gitlab" label="GitLab">
</TabItem>
<TabItem value="gitlab" label="GitLab">

You can set up GitLab to automatically create branch deployments for new PRs, using GitLab's CI/CD workflow.

Expand All @@ -41,10 +39,8 @@ Using this approach to branch deployments may be a good fit if:
- You use **GitLab** for version control
- You want Dagster to fully automate Branch Deployments

This approach is simplified if you use the [GitLab integration](/todo) to import your project into Dagster+.

</TabItem>
<TabItem value="cli" label="dagster-cloud CLI">
</TabItem>
<TabItem value="cli" label="dagster-cloud CLI">

You can manually execute dagster-cloud CLI commands to deploy and manage branch deployments.

Expand All @@ -71,10 +67,12 @@ In this step, you'll generate a token for the Dagster+ agent. The Dagster+ agent

Keep the token somewhere handy - you'll need it to complete the setup.

## Step 3: Create and configure an agent
## Step 3: Create and configure a branch deployment agent

:::note
If using [Serverless deployment](/dagster-plus/deployment/deployment-types/serverless), this step can be skipped.

If you are using [Serverless deployment](/dagster-plus/deployment/deployment-types/serverless), you can skip this step.

:::

While you can use your existing production agent, we recommend creating a dedicated branch deployment agent. This ensures that your production instance isn't negatively impacted by the workload associated with branch deployments.
Expand Down Expand Up @@ -133,7 +131,7 @@ While you can use your existing production agent, we recommend creating a dedica
<Tabs groupId="method">
<TabItem value="github" label="GitHub">

### Step 4.1: Add GitHub CI/CD script to your project
**Step 4.1: Add GitHub CI/CD script to your project**
:::note
If you used the GitHub app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-github-action-runs)
:::
Expand All @@ -146,7 +144,7 @@ Copy the following files to your project, and **replace** all references to `qui

In the next step, you'll modify these files to work with your Dagster+ setup.

### Step 4.2: Add the agent registry to dagster_cloud.yaml
**Step 4.2: Add the agent registry to dagster_cloud.yaml**

:::note
If you used the GitHub app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-github-action-runs)
Expand All @@ -158,7 +156,7 @@ For example:

<CodeExample path="docs_beta_snippets/docs_beta_snippets/dagster-plus/deployment/branch-deployments/dagster_cloud.yaml" language="yaml" />

### Step 4.3: Configure GitHub Action secrets
**Step 4.3: Configure GitHub Action secrets**

:::note
If you used the GitHub app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-github-action-runs)
Expand All @@ -177,7 +175,7 @@ Repeat steps 3-6 for each of the secrets required for the registry used by the a

<TabItem value="docker" label="Docker">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)
- `DOCKERHUB_USERNAME` - Your DockerHub username
- `DOCKERHUB_TOKEN` - A DockerHub [access token](https://docs.docker.com/docker-hub/access-tokens/#create-an-access-token)
Expand All @@ -186,25 +184,25 @@ Repeat steps 3-6 for each of the secrets required for the registry used by the a

<TabItem value="ecr" label="Amazon ECR">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)
- `AWS_ACCESS_KEY` - The **Access key ID** of the AWS IAM user you created in [Step 3](#step-3-create-and-configure-an-agent)
- `AWS_SECRET_ACCESS_KEY` - The **Secret access key** of the AWS IAM user you created in [Step 3](#step-3-create-and-configure-an-agent)
- `AWS_ACCESS_KEY` - The **Access key ID** of the AWS IAM user you created in step 3
- `AWS_SECRET_ACCESS_KEY` - The **Secret access key** of the AWS IAM user you created in step 3
- `AWS_REGION` - The AWS region where your ECR registry is located

</TabItem>

<TabItem value="gcr" label="Google Container Registry (GCR)">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)
- `GCR_JSON_KEY` - Your GCR JSON credentials

</TabItem>

</Tabs>

### Step 4.4: Configure GitHub Action
**Step 4.4: Configure GitHub Action**

:::note
If you used the GitHub app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-github-action-runs)
Expand All @@ -228,7 +226,7 @@ jobs:

Save and commit the file to your repository.

### Step 4.5: Verify GitHub action runs
**Step 4.5: Verify GitHub action runs**

The last step is to verify that the GitHub Action runs successfully.

Expand All @@ -240,7 +238,7 @@ The last step is to verify that the GitHub Action runs successfully.
</TabItem>
<TabItem value="gitlab" label="GitLab">

### Step 4.1: add GitLab CI/CD script to your project
**Step 4.1: add GitLab CI/CD script to your project**

:::note
If you used the GitLab app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-gitlab-pipeline-runs)
Expand All @@ -254,7 +252,7 @@ Copy the following files to your project, and **replace** all references to `qui

In the next step, you'll modify these files to work with your Dagster+ setup.

### Step 4.2: add the agent registry to dagster_cloud.yaml
**Step 4.2: add the agent registry to dagster_cloud.yaml**

:::note
If you used the GitLab app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-gitlab-pipeline-runs)
Expand All @@ -267,7 +265,7 @@ For example:

<CodeExample path="docs_beta_snippets/docs_beta_snippets/dagster-plus/deployment/branch-deployments/dagster_cloud.yaml" language="yaml" />

### Step 4.3: configure GitLab CI/CD variables
**Step 4.3: configure GitLab CI/CD variables**

:::note
If you used the GitLab app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-gitlab-pipeline-runs)
Expand All @@ -287,14 +285,14 @@ Repeat steps 3-6 for each of the secrets required for your registry type:

<TabItem value="gitlab" label="GitLab Container Registry">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)

</TabItem>

<TabItem value="docker" label="Docker">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)
- `DOCKERHUB_USERNAME` - Your DockerHub username
- `DOCKERHUB_TOKEN` - A DockerHub [access token](https://docs.docker.com/docker-hub/access-tokens/#create-an-access-token)
Expand All @@ -303,25 +301,25 @@ Repeat steps 3-6 for each of the secrets required for your registry type:

<TabItem value="ecr" label="Amazon ECR">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)
- `AWS_ACCESS_KEY` - The **Access key ID** of the AWS IAM user you created in [Step 3](#step-3-create-and-configure-an-agent)
- `AWS_SECRET_ACCESS_KEY` - The **Secret access key** of the AWS IAM user you created in [Step 3](#step-3-create-and-configure-an-agent)
- `AWS_ACCESS_KEY` - The **Access key ID** of the AWS IAM user you created in step 3
- `AWS_SECRET_ACCESS_KEY` - The **Secret access key** of the AWS IAM user you created in step 3
- `AWS_REGION` - The AWS region where your ECR registry is located

</TabItem>

<TabItem value="gcr" label="Google Container Registry (GCR)">

- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in [Step 2](#step-2-generate-a-dagster-agent-token)
- `DAGSTER_CLOUD_API_TOKEN` - The Dagster+ agent token you created in step 2
- `DAGSTER_CLOUD_URL` - Your Dagster+ base URL (`https://my_org.dagster.cloud`)
- `GCR_JSON_KEY` - Your GCR JSON credentials

</TabItem>

</Tabs>

### Step 4.4: configure GitLab CI/CD script
**Step 4.4: configure GitLab CI/CD script**

:::note
If you used the GitLab app to configure you're repository, this step isn't required. [Skip ahead to Step 4.5](#step-45-verify-gitlab-pipeline-runs)
Expand All @@ -341,7 +339,7 @@ build-image:

Save and commit the files to the project.

### Step 4.5: verify GitLab pipeline runs
**Step 4.5: verify GitLab pipeline runs**

The last step is to verify that the GitLab pipeline runs successfully.

Expand All @@ -367,7 +365,7 @@ The following examples assume the registry URL and image tag are stored in the `

:::

### Step 4.1 Create a branch deployment associated with the branch
**Step 4.1 Create a branch deployment associated with the branch**

Execute the following command within your CI/CD process:

Expand Down Expand Up @@ -421,7 +419,7 @@ BRANCH_DEPLOYMENT_NAME=$(
)
```

### Step 4.2 Deploy your code to the branch deployment
**Step 4.2 Deploy your code to the branch deployment**

Execute the following command within your CI/CD process:

Expand Down
Loading