-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: Add documentation for contributing additional checks to the trivy policies repo #6234
Merged
Merged
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
b7136b9
update docs contributing guidelines for custom checks
AnaisUrlichs 2e80391
docs: update contributing custom checks
AnaisUrlichs 0c20731
docs: fix small errors in contributing checks docs
AnaisUrlichs 26ec116
merging main
AnaisUrlichs 4d951c2
merging main
AnaisUrlichs 06cf991
Update docs/community/contribute/checks/overview.md
AnaisUrlichs 7903c14
merging main
AnaisUrlichs 0ff7fc9
Update docs/community/contribute/checks/service-support.md
AnaisUrlichs fd549fd
Update docs/docs/scanner/misconfiguration/custom/index.md
AnaisUrlichs cc58e84
merging main
AnaisUrlichs 8af51c8
merging main
AnaisUrlichs 1390c66
add checks namespace information
AnaisUrlichs b5c7300
correct merge main
AnaisUrlichs d3b1952
correct merge main
AnaisUrlichs 7cad91a
correct merge main
AnaisUrlichs 2f15c96
Update docs/docs/scanner/misconfiguration/custom/contribute-checks.md
simar7 735a7e8
Apply suggestions from code review
simar7 d3c538b
Update docs/docs/scanner/misconfiguration/custom/contribute-checks.md
simar7 82e3881
Update docs/docs/scanner/misconfiguration/custom/index.md
simar7 aee33af
Update docs/community/contribute/checks/overview.md
simar7 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
# Contribute Rego Checks | ||
|
||
The following guide provides an overview of contributing checks to the default checks in Trivy. | ||
|
||
All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. Before you begin writing a check, ensure: | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
1. The check does not already exist as part of the default checks in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository. | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
2. The pull requests in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/pulls) repository to see whether someone else is already contributing the check that you wanted to add. | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
3. The [issues in Trivy](https://github.com/aquasecurity/trivy/issues) to see whether any specific checks are missing in Trivy that you can contribute. | ||
|
||
If anything is unclear, please [start a discussion](https://github.com/aquasecurity/trivy/discussions/new) and we will do our best to help. | ||
|
||
## Check structure | ||
|
||
Checks are written in Rego and follow a particular structure in Trivy. Below is an example check for AWS: | ||
|
||
```rego | ||
# METADATA | ||
# title: "RDS IAM Database Authentication Disabled" | ||
# description: "Ensure IAM Database Authentication is enabled for RDS database instances to manage database access" | ||
# scope: package | ||
# schemas: | ||
# - input: schema["aws"] | ||
# related_resources: | ||
# - https://docs.aws.amazon.com/neptune/latest/userguide/iam-auth.html | ||
# custom: | ||
# id: AVD-AWS-0176 | ||
# avd_id: AVD-AWS-0176 | ||
# provider: aws | ||
# service: rds | ||
# severity: MEDIUM | ||
# short_code: enable-iam-auth | ||
# recommended_action: "Modify the PostgreSQL and MySQL type RDS instances to enable IAM database authentication." | ||
# input: | ||
# selector: | ||
# - type: cloud | ||
# subtypes: | ||
# - service: rds | ||
# provider: aws | ||
|
||
package builtin.aws.rds.aws0176 | ||
|
||
deny[res] { | ||
instance := input.aws.rds.instances[_] | ||
instance.engine.value == ["postgres", "mysql"][_] | ||
not instance.iamauthenabled.value | ||
res := result.new("Instance does not have IAM Authentication enabled", instance.iamauthenabled) | ||
} | ||
``` | ||
|
||
## Verify the provider and service exists | ||
|
||
Every check for a cloud service references a cloud provider. The list of providers are found in the [Trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. | ||
|
||
Before writing a new check for a cloud provider, you need to verify if the cloud provider or resource type that your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. As a reference you can take a look at the AWS provider [here](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go). | ||
|
||
???+ note | ||
New Kubernetes and Dockerfile checks do not require any additional provider definitions. You can find an example of a Dockerfile check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/docker/add_instead_of_copy.rego) and a Kubernetes check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/kubernetes/general/CPU_not_limited.rego). | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
### Add Support for a New Service in an existing Provider | ||
|
||
[Please reference the documentation on adding Support for a New Service](./service-support.md). | ||
|
||
This guide also showcases how to add new properties for an existing Service. | ||
|
||
## Create a new .rego file | ||
|
||
The following directory in the trivy-policies repository contains all of our custom checks. Depending on what type of check you want to create, you will need to nest a new `.rego` file in either of the [subdirectories](https://github.com/aquasecurity/trivy-policies/tree/main/checks): | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
* cloud: All checks related to cloud providers and their services | ||
* docker: Docker specific checks | ||
* kubernetes: Kubernetes specific checks | ||
|
||
## Check Package name | ||
|
||
Have a look at the existing package names in the [built in checks](https://github.com/aquasecurity/trivy-policies/tree/main/checks). | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. | ||
|
||
## Generating an ID | ||
|
||
Every check has a custom ID that is referenced throughout the metadata of the check to uniquely identify the check. If you plan to contribue your check back into the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository, it will require a valid ID. | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Running `make id` in the root of the trivy-policies repository will provide you with the next available _ID_ for your rule. | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Check Schemas | ||
|
||
Rego Checks for Trivy can utilise Schemas to map the input to specific objects. The schemas available are listed [here.](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). | ||
|
||
More information on using the builtin schemas is provided in the [main documentation.](../../../docs/scanner/misconfiguration/custom/schema.md) | ||
|
||
## Check Metadata | ||
|
||
The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another check as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). | ||
|
||
For detailed information on each component of the Check Metadata, please refer to the [main documentation.](../../../docs/scanner/misconfiguration/custom/index.md) | ||
|
||
Note that while the Metadata is optional in your own custom checks for Trivy, if you are contributing your check to the Trivy builtin checks, the Metadata section will be required. | ||
|
||
|
||
## Writing Rego Rules | ||
|
||
Rules are defined using _OPA Rego_. You can find a number of examples in the `checks` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
```rego | ||
deny[res] { | ||
instance := input.aws.rds.instances[_] | ||
instance.engine.value == ["postgres", "mysql"][_] | ||
not instance.iamauthenabled.value | ||
res := result.new("Instance does not have IAM Authentication enabled", instance.iamauthenabled) | ||
} | ||
``` | ||
|
||
The rule should return a result, which can be created using `result.new`. This function does not need to be imported, it is defined internally and provided at runtime. The first argument is the message to display and the second argument is the resource that the issue was detected on. | ||
|
||
It is possible to pass any rego variable that references a field of the input document. | ||
|
||
## Generate docs | ||
|
||
Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` in the [trivy-policies](https://github.com/aquasecurity/trivy-policies) directory to generate the documentation for your new policy and submit a PR for us to take a look at. | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Adding Tests | ||
|
||
All Rego checks need to have tests. There are many examples of these in the `checks` directory for each check ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Example PR | ||
|
||
You can see a full example PR for a new rule being added here: [https://github.com/aquasecurity/defsec/pull/1000](https://github.com/aquasecurity/defsec/pull/1000). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# Add Service Support | ||
|
||
A service refers to a service by a cloud provider. This section details how to add a new service to an existing provider. All contributions need to be made to the [trivy repository](https://github.com/aquasecurity/trivy/). | ||
|
||
## Prerequisites | ||
|
||
Before you begin, verify that the [provider](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) does not already have the service that you plan to add. | ||
|
||
## Adding a new service to an existing provider | ||
|
||
Adding a new service involves two steps. The service will need a data structure to store information about the required resources that will be scanned. Additionally, the service will require one or more adapters to convert the scan targetes as input(s) into the aforementioned data structure. | ||
|
||
### Create a new file in the provider directory | ||
|
||
In this example, we are adding the CodeBuild service to the AWS provider. | ||
|
||
First, create a new directory and file for your new service under the provider directory: e.g. [aws/codebuild/codebuild.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/codebuild/codebuild.go) | ||
|
||
The CodeBuild service will require a structure `struct` to hold the information on the input that is scanned. The input is the CodeBuild resource that a user configured and wants to scan for misconfiguration. | ||
|
||
``` | ||
type CodeBuild struct { | ||
Projects []Project | ||
} | ||
``` | ||
|
||
The CodeBuild service manages `Project` resources. The `Project` struct has been added to hold information about each Project resources; `Project` Resources in turn manage `ArtifactSettings`: | ||
|
||
``` | ||
type Project struct { | ||
Metadata iacTypes.Metadata | ||
ArtifactSettings ArtifactSettings | ||
SecondaryArtifactSettings []ArtifactSettings | ||
} | ||
|
||
type ArtifactSettings struct { | ||
Metadata iacTypes.Metadata | ||
EncryptionEnabled iacTypes.BoolValue | ||
} | ||
``` | ||
|
||
The `iacTypes.Metadata` struct is embedded in all of the Trivy types and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined and the name of the resource. | ||
|
||
A resource in this example `Project` can have a name and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `iacTypes.Metadata` and `iacTypes.BoolValue`. These types wrap the raw values and provide additional metadata about the value. For instance, whether it was set by the user and the file and line number where the resource was defined. | ||
|
||
Have a look at the other providers and services in the [`iac/providers`](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) directory in Trivy. | ||
|
||
Next you'll need to add a reference to your new service struct in the [provider struct](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go) at `pkg/iac/providers/aws/aws.go`: | ||
|
||
``` | ||
type AWS struct { | ||
... | ||
CodeBuild codebuild.CodeBuild | ||
... | ||
} | ||
``` | ||
|
||
### Update Adapters | ||
|
||
Now you'll need to update all of the [adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters) which populate the struct of the provider that you have been using. Following the example above, if you want to add support for CodeBuild in Terraform, you'll need to update the Terraform AWS adatper as shown here: [`trivy/pkg/iac/adapters/terraform/aws/codebuild/adapt.go`](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/aws/codebuild/adapt.go). | ||
|
||
Another example for updating the adapters is provided in the [following PR.](https://github.com/aquasecurity/defsec/pull/1000/files) Additionally, please refer to the respective Terraform documentation on the provider to which you are adding the service. For instance, the Terraform documentation for AWS CodeBuild is provided [here.](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project) | ||
|
||
|
||
## Create a new Schema for your provider | ||
|
||
Once the new service has been added to the provider, you need to create the schema for the service as part of the provider schema. | ||
|
||
This process has been automated with mage commands. In the Trivy root directory run `mage schema:generate` to generate the schema for your new service and `mage schema:verify`. |
5 changes: 5 additions & 0 deletions
5
docs/docs/scanner/misconfiguration/custom/contribute-checks.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
## Contribute Rego Checks | ||
|
||
The contributing section provides detailed information on how to contribute custom checks to the [trivy-policies repository](../../../../community/contribute/checks/overview.md/) | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
This way, they become accessible as default [checks.](https://github.com/aquasecurity/trivy-policies) | ||
simar7 marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there's naturally some overlap between this and the other doc about custom checks. given the difference between them is only an extra step (do you want to contribute it or keep it to yourself), it highlights the inconsistency between the otherwise similar docs. for example, this doc basically documents how to write an AWS check, but I wouldn't think to read it if I didn't want to contribute to Trivy. If it's not too much, I would refactor the content into: 1) creating custom checks - should fully document all the guidelines related to the rego file, testing, metadata, AWS examples etc. this can be multiple docs if needed. 2) contributing a custom check to trivy - refers to the other doc and explains how to take a check you wrote (not explained here) and just contribute it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I rewrote it -- some components that are not relevant for anyone writing custom checks for their usage of Trivy are however a requirement when contributing the check back into Trivy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a difference between
This docs is the first