Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up slack notifications for failing AI e2e tests #5784

Open
Irev-Dev opened this issue Mar 13, 2025 · 3 comments
Open

Set up slack notifications for failing AI e2e tests #5784

Irev-Dev opened this issue Mar 13, 2025 · 3 comments
Assignees
Labels
tests Pull requests that update or improve our test suite

Comments

@Irev-Dev
Copy link
Collaborator

The ML team wants notifications in slack for when there are failures in

e2e/playwright/prompt-to-edit.spec.ts
e2e/playwright/text-to-cad-tests.spec.ts

They run early enough that it can help indicate a problem with their endpoint.

See these slack threads too:
https://kittycadworkspace.slack.com/archives/C030D6PR135/p1741645854199789
https://kittycadworkspace.slack.com/archives/C04KFV6NKL0/p1741381658820369

@Irev-Dev
Copy link
Collaborator Author

CC @greg-kcio

@greg-kcio
Copy link
Contributor

Thank you! You can send them to #ml-notifications

@jtran
Copy link
Collaborator

jtran commented Mar 13, 2025

I know this is expanding scope, and any notifications are better than none. But whenever you make something that notifies on failure, you basically want to record the heartbeat on success. And then have another monitor that notifies you if there hasn't been a successful heartbeat in a while. It's a common pattern for infrastructure.

Always remember: who watches the watcher? 🙂

@lf94 lf94 added the tests Pull requests that update or improve our test suite label Mar 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Pull requests that update or improve our test suite
Projects
None yet
Development

No branches or pull requests

5 participants