Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

edr monitor --teams-webhook giving error #1644

Open
JyotsnaMalode opened this issue Jul 22, 2024 · 17 comments
Open

edr monitor --teams-webhook giving error #1644

JyotsnaMalode opened this issue Jul 22, 2024 · 17 comments
Labels
Bug Something isn't working Triage 👀

Comments

@JyotsnaMalode
Copy link

JyotsnaMalode commented Jul 22, 2024

Describe the bug
When I execute below command:
edr monitor --teams-webhook giving error

Getting below error in Power Automate workflow
Oops, we failed to format the alert ! -_-' Please share this with the Elementary team via https://elementary-data.com/community or a https://github.com/elementary-data/elementary/issues/new|GitHub issue."

To Reproduce
Steps to reproduce the behavior:

  1. Run dbt tests
  2. Execute command "edr monitor --teams-webhook "
  3. Getting error as below, even though i have ?api-version=2016-06-01& in webhook url
    image

Also getting below error in PowerAutomate
image

Expected behavior
I was expecting to see alerts in Teams channel

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • Elementary CLI (edr) version: [e.g. 0.5.3], can be found by running pip show elementary-data
    Version: 0.15.1
  • Elementary dbt package version: [e.g. 0.4.1], can be found in packages.yml file
    version: 0.14.1
  • dbt version you're using [e.g. 1.8.1]
    1.7.10
  • Data warehouse [e.g. snowflake]
    Redshift
  • Infrastructure details (e.g. operating system, prod / dev / staging, deployment infra, CI system, etc)

Additional context
Add any other context about the problem here.

Would you be willing to contribute a fix for this issue?
Let us know if you're willing to contribute so that we'll be able to provide guidance.

@JyotsnaMalode JyotsnaMalode added Bug Something isn't working Triage 👀 labels Jul 22, 2024
@dapollak
Copy link
Contributor

Hey @JyotsnaMalode,
How do you pass the webhook url to the CLI ?

@JyotsnaMalode
Copy link
Author

hi @dapollak , I am using below command
edr monitor --teams-webhook "url"

I am using Teams workflow and not Teams webhook as it will soon be not supported

When I check the raw output for "When a Teams webhook request is received" in Wrokflow, I get below data
"title": "Oops, we failed to format the alert ! --' Please share this with the Elementary team via https://elementary-data.com/community or a https://github.com/elementary-data/elementary/issues/new|GitHub issue.",_

@mriveraFacephi
Copy link

Hi, same error here. pymsteams appears not to have a solution yet.

A workaround could be sent alerts in plain text and allowing ok status code between 200 <= x < 300.
Not the best solution but at least it allows to continue receive alerts.

@JyotsnaMalode
Copy link
Author

Thanks @mriveraFacephi , will try your suggestion.

@Jonathan-MW
Copy link

I also tried to setup MS Teams alerts with the new Power Automate workflow method, because the connector already does not work anymore for us.
Getting this error: "ERROR — Unable to send alert via Teams: 'Response' object has no attribute 'body'"

image

@JyotsnaMalode
Copy link
Author

Update - workaround suggested by @mriveraFacephi worked for me to receive the text message.

@FredRodrigues
Copy link

Hey @Jonathan-MW , I am having the same issue. Did you try the solution provided by @mriveraFacephi ?

I tried but does not seem to work :/

@FredRodrigues
Copy link

In short, we will need for an update from pymsteams in order to solve this issue or is there any other option we can use to solve this problem?

@FrankTub
Copy link

FrankTub commented Nov 4, 2024

@FredRodrigues I think we could also suggest to the elementary team to start using https://github.com/AeroFlorian/MsTeamsAdaptiveCard/tree/master instead of pymsteams to switch to the new MS Teams way of doing things.

@FrankTub
Copy link

@ellakz I just looked into this issue and verified that moving the existing webhooks we have to the new webhooks as defined by MS Teams, but then I also get an error:

ERROR — Unable to send alert fallback via Teams: 'Response' object has no attribute 'body'

When looking at the documentation of MS Teams we will have to migrate to a new solution before 31 januari 2025:

URLs for all webhook-based connectors must be updated by January 31, 2025 for the connectors to continue posting messages into Teams

What do you think about my above suggestion to start using another package to send messages to MS Teams?

@MikaKerman
Copy link
Contributor

MikaKerman commented Nov 19, 2024

Hi @JyotsnaMalode,
Thanks for raising the issue!

As the deprecation deadline for Office 365 Connectors approaches, we're working on adding support for sending alerts to Teams via workflows.
Workflows only support the new Adaptive Cards format (see the note here), whereas elementary currently uses the older message card format supported by Office 365 Connectors.

Currently, when we can't format the message, we send a fallback message with JSON data to ensure you're notified about the alert. However, this fallback message is also not in the Adaptive Cards format, so it fails in workflows, which is why you see the error in Power Automate. You might also see another error related to the original alert.

Until we roll out support for the Adaptive Cards format, I suggest using the Incoming Webhook application as a temporary solution, if possible. You can pass the URL generated by the Incoming Webhook to edr monitor, and alerts will route correctly.

Regarding the ERROR log, @FrankTub, this was a minor bug in logging the actual error when sending the alert. I've deployed a quick fix for this today, so it should now print a more accurate message.

@FrankTub
Copy link

@MikaKerman , can I test this by pinning the version of elementary to use as the master branch?

@FrankTub
Copy link

@MikaKerman , I tried to test it out for you but this results in worse behavior in my opinion, see explanation below.

I pinned Elementary to the last commit on master in my requirements.txt by adding it to:

elementary-data @ git+https://github.com/elementary-data/elementary@master

Next I ran something like:

edr monitor --profiles-dir $DBT_PROFILES_DIR --teams-webhook "$MS_TEAMS_DATA_QUALITY_WEBHOOK_URL" --report-url "$DATA_QUALITY_REPORT_URL"

This does not result in an error, but no messages are sent to the Teams webhook:

    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2024-11-25 11:32:52 — INFO — Running with edr=0.16.1
2024-11-25 11:32:54 — INFO — edr (0.16.1) and Elementary's dbt package (0.16.2) are compatible.
2024-11-25 11:32:55 — INFO — Elementary's database and schema: '"staging.elementary"'
2024-11-25 11:32:55 — INFO — Running internal dbt run to populate alerts
2024-11-25 11:32:55 — INFO — Running dbt command run -m elementary_cli.alerts.alerts_v2 --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt --vars {"days_back": 1}
2024-11-25 11:32:58 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_pending_alerts", "macro_args": {"days_back": 1, "type": null}} --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt
2024-11-25 11:33:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_last_alert_sent_times", "macro_args": {"days_back": 1, "type": null}} --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt
2024-11-25 11:33:01 — INFO — Update skipped alerts
Sending alerts |████████████████████████████████████████| 20/20 [100%] in 0.0s (872.94/s)
2024-11-25 11:33:01 — INFO — Update sent alerts

@FrankTub
Copy link

@MikaKerman , I tried out following some time ago and that works for this new webhook in Teams:

import requests
from adaptive_cards.card import AdaptiveCard
from adaptive_cards.elements import TextBlock, Image
import adaptive_cards.card_types as types

# Replace with your Teams webhook URL
webhook_url = "some-url"

text_block: TextBlock = TextBlock(
    text="It's your third card",
    color=types.Colors.ACCENT,
    size=types.FontSize.EXTRA_LARGE,
    horizontal_alignment=types.HorizontalAlignment.CENTER,
)

image: Image = Image(url="https://adaptivecards.io/content/bf-logo.png")

version: str = "1.5"
card = AdaptiveCard.new() \
        .version(version) \
        .add_items([text_block, image]) \
        .create()

payload = {
    "type": "message",
    "attachments": [
        {
            "contentType": "application/vnd.microsoft.card.adaptive",
            "content": card.to_dict()
        }
    ]
}

response = requests.post(webhook_url, json=payload)

if response.status_code == 200:
    print("Message sent successfully!")
else:
    print(f"Failed to send message. Status code: {response.status_code}, response: {response.text}")

Resulting output in Teams:

image

Perhaps this can point you in the right direction?

@FredRodrigues
Copy link

@FrankTub dont forget the suppression-interval is by default 24hours

edr monitor --profiles-dir $DBT_PROFILES_DIR --teams-webhook "$MS_TEAMS_DATA_QUALITY_WEBHOOK_URL" --report-url "$DATA_QUALITY_REPORT_URL"

I would add --supression-interval 0 and should send messages to teams.

We are currently facing a similar issue, we get the message "ERROR — Unable to send alert via Teams: 'Response' object has no attribute 'body'" but somehow it still sends the message, we are using workflows on teams to parse the message and post the alert on the channel because for our use case we only need the title and description.

@FrankTub
Copy link

@FredRodrigues , pretty sure that suppression-interval is irrelevant in my test case for the master branch since I just generated some sample errors that I could verify with edr report. Also you can see that there are 20/20 test results sent according to elementary, however none are received in the Teams channel.

@MikaKerman
Copy link
Contributor

Hi @FrankTub
Thank you for the suggestion!
I'm currently adding support for the adaptive cards format.
We'll update you on how to migrate to the new method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Triage 👀
Projects
None yet
Development

No branches or pull requests

7 participants