Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(secret_scan): add all-secrets option #996

Conversation

gg-mmill
Copy link
Contributor

Allows to display all secrets, even those excluded. Also adds a message when excluded secrets are found, to advertise the feature

Context

Allows to display all secrets, even those excluded.

Also adds a message when excluded secrets are found, to advertise the feature

What has been done

  • add --all-secret option
  • update output handlers
    • json output is always extended
    • text output
      • when option is not passed, add line below file info showing number of hidden secrets
      • when option is passed, show all secrets, and add Exclude reason when appropriate
    • sarif output: ???

Validation

Run ggshield scans on a file containing secrets with ignored / resolved issues, with and without --all-secrets option

PR check list

  • As much as possible, the changes include tests (unit and/or functional)
  • If the changes affect the end user (new feature, behavior change, bug fix) then the PR has a changelog entry (see doc/dev/getting-started.md). If the changes do not affect the end user, then the skip-changelog label has been added to the PR.

@gg-mmill gg-mmill self-assigned this Oct 29, 2024
@gg-mmill gg-mmill force-pushed the mmillet/spi-515-add-all_secrets-option-to-ggshield-secret-scans branch 6 times, most recently from 4e88915 to da87671 Compare November 15, 2024 17:34
Copy link

codecov bot commented Nov 15, 2024

Codecov Report

Attention: Patch coverage is 73.17073% with 11 lines in your changes missing coverage. Please review.

Project coverage is 91.94%. Comparing base (7b28c3e) to head (f2657b8).
Report is 57 commits behind head on main.

Files with missing lines Patch % Lines
...ticals/secret/output/secret_text_output_handler.py 75.00% 3 Missing ⚠️
...ticals/secret/output/secret_json_output_handler.py 60.00% 2 Missing ⚠️
ggshield/core/text_utils.py 75.00% 1 Missing ⚠️
...ecret/output/secret_gitlab_webui_output_handler.py 50.00% 1 Missing ⚠️
...d/verticals/secret/output/secret_output_handler.py 66.66% 1 Missing ⚠️
...icals/secret/output/secret_sarif_output_handler.py 50.00% 1 Missing ⚠️
...gshield/verticals/secret/secret_scan_collection.py 83.33% 1 Missing ⚠️
ggshield/verticals/secret/secret_scanner.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #996      +/-   ##
==========================================
- Coverage   92.03%   91.94%   -0.10%     
==========================================
  Files         181      181              
  Lines        7701     7734      +33     
==========================================
+ Hits         7088     7111      +23     
- Misses        613      623      +10     
Flag Coverage Δ
unittests 91.94% <73.17%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gg-mmill gg-mmill requested a review from a team as a code owner November 18, 2024 15:52
@gg-mmill gg-mmill changed the title Draft: feat(secret_scan): add all-secrets option feat(secret_scan): add all-secrets option Nov 18, 2024
@gg-mmill gg-mmill force-pushed the mmillet/spi-515-add-all_secrets-option-to-ggshield-secret-scans branch 3 times, most recently from a6e5c06 to 3b80646 Compare November 18, 2024 15:58
Allows to display all secrets, even those excluded.
In text output, adds a message when excluded secrets are found, to advertise the feature
Related to previous commit, cassettes needed to be updated because the
query args to secret endpoint have changed
@gg-mmill gg-mmill force-pushed the mmillet/spi-515-add-all_secrets-option-to-ggshield-secret-scans branch from 3b80646 to f2657b8 Compare November 18, 2024 16:01
@@ -139,6 +139,17 @@ def _banlist_detectors_callback(
)


_all_secrets = click.option(
"--all-secrets",
is_flag=True,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to add a default=None otherwise the option won't work when it appears early on the command line (secret scan path --all-secrets . is OK, but secret --all-secrets scan path . is not).

is_flag=True,
help=(
"Display all secrets, including those that have been ignored for example. "
"Note that all returned secrets are considered as alerts."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is an "alert" in this context?

@@ -136,15 +136,28 @@ def file_info(
filename: str,
incident_count: int,
incident_status: IncidentStatus = IncidentStatus.DETECTED,
number_of_hidden_secrets: int = 0,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

naming: I would name this hidden_secret_count, it's shorter and consistent with incident_count.

@@ -14,6 +14,8 @@ class FlattenedPolicyBreak(BaseSchema):
incident_url = fields.String(required=True, dump_default="")
incident_details = fields.Nested(SecretIncidentSchema)
known_secret = fields.Bool(required=True, dump_default=False)
is_excluded = fields.Bool(required=False)
exclude_reason = fields.String(required=False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably a bit late, but I just realized exclude_reason does not sound like correct English. I think it should be exclusion_reason.

"--all-secrets",
is_flag=True,
help=(
"Display all secrets, including those that have been ignored for example. "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need more than this example, otherwise users are going to wonder what secrets would be listed with this option that would not be listed otherwise.

@gg-mmill gg-mmill marked this pull request as draft November 27, 2024 10:54
@gg-mmill gg-mmill closed this Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants