Skip to content

enh: collector_mail_attach Decrypt GPG attachments #2623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

e3rd
Copy link
Member

@e3rd e3rd commented Jul 15, 2025

Our partner sends us a GPG-encrypted attachment. This PR solves this.

Before merging, let's discuss. Should it be part of a collector bot, or should it be rather a GPG parser?

Current pipeline (current PR)

graph LR;
a["mail collector attach"]--> b["generic csv parser"];
Loading

Another possibility

graph LR;
a["mail collector attach"]--> c["gpg parser"] --> b["generic csv parser"];
Loading

Another possibility

graph LR;
a["mail collector attach"]--> b["generic csv parser allowing GPG decryption"];
Loading

Another possibility

graph LR;
a["mail collector attach"]--> b["gpg parser, inheriting from generic csv parser"];
Loading

@e3rd e3rd force-pushed the mail-attach-encrypt branch 2 times, most recently from 57b27f8 to b3c0b05 Compare July 15, 2025 16:09
@sebix sebix added this to the 3.5.0 milestone Jul 15, 2025
@sebix
Copy link
Member

sebix commented Jul 15, 2025

A separate OpenPGP parser (first variant) would strictly adhere to the rule "one bot one job", but the current code is not designed for nor tested with two parsers in a row.
As you did it now, adding the decryption to the collector, is in line with other post-collection processing, such as extracting from archives, so that's fine for me.

Forth variant makes less sense than the third. In third, the CSV parser could use an "OpenPGP Mixin".

@sebix sebix removed this from the 3.5.0 milestone Jul 15, 2025
@e3rd
Copy link
Member Author

e3rd commented Jul 16, 2025

Great! Thanks for having investigated, sebix! :) I am then ready to be merged – or is there anything I could do better?

Copy link
Contributor

@kamil-certat kamil-certat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the recommendation from @sebix I like the change, it looks good - however I'd slightly suggest adding negative test to ensure proper handling GPG errors

@kamil-certat
Copy link
Contributor

Ah, unfortunately, you need to add license files for not code test files (see example https://github.com/certtools/intelmq/blob/develop/docs/static/images/Logo_Intel_MQ.png.license)

@e3rd e3rd force-pushed the mail-attach-encrypt branch 3 times, most recently from d1f5a25 to 22a7423 Compare July 17, 2025 15:11
@e3rd e3rd force-pushed the mail-attach-encrypt branch from 22a7423 to b86e313 Compare July 17, 2025 15:18
@e3rd
Copy link
Member Author

e3rd commented Jul 17, 2025

tests added, license solved :) I'm done now

Copy link
Member

@sebix sebix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this extensive work, including nice test cases.

I'm missing the documentation though (in docs/user/bots.md).


Files: intelmq/tests/bots/collectors/mail/gpg_ring/**
Copyright: 2025 Edvard Rejthar, CSIRT.cz <[email protected]>
License: AGPL-3.0-or-later
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing newline at end of file

@@ -28,11 +31,19 @@ class MailAttachCollectorBot(MailCollectorBot):
mail_user: str = "<user>"
rate_limit: int = 60
subject_regex: str = "<subject>"
decrypt: bool = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As OpenPGP is not the only e-mail encryption method (S/MIME is also widespread), I suggest naming the parameter decrypt_openpgp or similar.

Suggested change
decrypt: bool = False
decrypt_openpgp: Optional[bool] = False


def init(self):
super().init()
if self.attach_regex is None:
raise InvalidArgument('attach_regex', expected='string')
raise InvalidArgument("attach_regex", expected="string")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how this and other formatting changes are related to the decryption. It makes the diff harder to read and the blames harder to interpret and use.

Comment on lines +113 to +116
try:
from gnupg import GPG
except ImportError:
raise MissingDependencyError("python-gnupg", ">=0.5")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest doing the check in init (for quick user-feedback). If decyrption is activated, and the library is not installed, raise the exception.

@sebix sebix added feature Indicates new feature requests or new features component: bots labels Jul 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: bots feature Indicates new feature requests or new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants