Ability to trigger an Alert every single time the Alert's conditions are met

[mcassels]

Problem Statement

I would like to receive a Slack message whenever an event with a particular tag is captured by Sentry.

I have an Alert configured to send the Slack message whenever an event occurs with the matching tag. However, the minimum action interval is 5 minutes. I do not want to set an action interval, since I would like to receive a Slack message every time this alert is triggered, even if it is triggered twice in a 5 minute period.

Other than the tag that triggers the alert, the events will also have other tags, which are shown in the slack message and will be different across the different events. I don't want to get a single slack message with only the version of the tags from the first event that occurred within that 5 minutes interval.

Solution Brainstorm

No response

Product Area

Alerts

[getsantry]

Assigning to @getsentry/support for routing ⏲️

[getsantry]

Routing to @getsentry/product-owners-alerts for triage ⏲️

[scefali]

The 5 minute interval refers to how often you get notified on a single issue. Are you saying you want to be notified on a single issue more than once every 5 minutes?

[mcassels]

The 5 minute interval refers to how often you get notified on a single issue. Are you saying you want to be notified on a single issue more than once every 5 minutes?

Yes. I am using captureEvent to report on an event that occurs in the application. It will not occur very often, but when it does I want to receive a slack message. And I want to receive that slack message every single time it occurs, regardless of whether it occurs twice within 5 minutes.

I have an Alert set up as follows:

• When an event is captured
• If the event's tags match a particular tag
• Then send a notification to the slack channel and show certain tags in the notification

Maybe Alerts aren't intended for my use case. Does sentry offer any other way of triggering a slack notification every time an event with a given tag is captured?

[malwilley]

Unfortunately, we have no built-in solution for this @mcassels. Alerts are what you want, but we don't support alerts without an action interval. I'll mark this as a feature request and put it on the backlog. Although, it isn't on our roadmap right now. If anyone else reads this and also wants a feature like this, please leave a comment to help get it prioritized!

[GodFan1995]

I need this feature, perhaps to manually suppress alarms like alertmanager

[lobsterkatie]

I need this feature, perhaps to manually suppress alarms like alertmanager

@GodFan1995 - Can you please clarify? This is about sending more alerts, but you're talking about suppressing them.

[suriya-zenefits]

We need this feature as well.

Whats currently happening is that Sentry sends group alerts instead of individual alerts.

Min interval span is 5 mins. All the events that occurred within 5 mins interval are grouped together (based on stack trace) and alerts are sent out accordingly.

For example: If 100 error events occurred within 5 mins, we won’t be getting 100 individual alerts; instead, sentry categorise them and send us a grouped alert (may be 5 or 10 depends on the error classification).

[scefali]

@mcassels @GodFan1995 @suriya-zenefits What I'm not understanding is why someone would want to be pinged for a single issue more than one time per 5 minutes on a single channel? I'd think it would be overly repetitive being notified about the same issue over and over. Not to mention Slack has rate limiting we'd start to hit more frequently if we notified more often. Maybe you can go into more detail about why you want this feature?

[jwarder]

I also require this feature as well, seems like a very basic requirement especially if you're using captureMessage for notifying of user behaviour. I'd like to know every time this happens without a delay.

[rachrwang]

Hi @jwarder - we don't have any plans to change this configuration right now, though thank you for letting us know that this is an important use case for you. We'll update this issue if we do decide to make any changes here

[tgdva]

+1 on that, we need this feature as well. We don't want to wait for 5 minutes to receive a new alert. At least having the 1 minute option in the action interval would be nice.

Currently, we would need to set something up on our own, and considering that we already use Sentry for monitoring, it would be great not to need to do any workaround.

[rachrwang]

@tgdva - can you share more about your use case and the impact that a 1 minute option would have vs. 5 minutes?

[camwardy]

+1 we'd also like to send a slack message for every event received

[leedongwei]

Hi folks, is there a compelling reason for 1min granularity on the alert? Going from 5min to 1min would 5x the queries to our OLAP database, so there's quite a bit of scalability/cost concerns on our end.

[camwardy]

@leedongwei personally we wouldn't want 1 min granularity. We're in a very similar position to @mcassels where we'd like to trigger a new Slack notification for each event (that does not occur regularly). Although in the rare case 2 events occurred within 5 mins of each other, we'd still like 2 notifications.

[tgdva]

@tgdva - can you share more about your use case and the impact that a 1 minute option would have vs. 5 minutes?

Sorry for the delay. There are specific critical events where I want to receive Slack alerts without waiting 5 minutes to receive the next event.

In the ideal world, I'd like to receive a Slack alert after any occasion that event was caught by Sentry, like the OP suggestion, but if that's not possible, a short window of 1 minute won't solve my use case but at least we will be alerted more often when that critical event occurs than needing to wait 5 minutes, which is currently the shorter interval.

I could configure manually other solution to do that for me but since I advocated Sentry to do this monitorization role where I work I have some colleagues bothered due to the fact that we can't have real-time alerts when that's the case.

[leedongwei]

Yep, I understand better now. It is non-trivial change with our current architecture, so you'll need to find a workaround for now.

We will keep it in mind. The Alerts product will have some changes coming in the next 3-4 months and we'll try our best to factor it in.