CLI | Support Container Engine Scan (AST-36237, AST-36236, AST-36238) #672

AlvoBen · 2024-03-06T10:31:54Z

By submitting a PR to this repository, you agree to the terms within the Checkmarx Code of Conduct. Please see the contributing guidelines for how to create and submit a high-quality PR for this repo.

Description

Start Container Scan - Trigger Container Resolver For Containers and Scans.
Support Containers as New Scan Type With Realted Flags.

References

https://checkmarx.atlassian.net/jira/software/c/projects/AST/boards/471?assignee=712020%3A8b115cb9-1cab-4fd5-9878-130ce7842e3f&selectedIssue=AST-36237

https://checkmarx.atlassian.net/jira/software/c/projects/AST/boards/471?assignee=712020%3A8b115cb9-1cab-4fd5-9878-130ce7842e3f&selectedIssue=AST-36236

Testing

Added unit and integration tests.

Checklist

I have added documentation for new/changed functionality in this PR (if applicable).
I have updated the CLI help for new/changed functionality in this PR (if applicable).
All active GitHub checks for tests, formatting, and security are passing
The correct base branch is being used

github-actions · 2024-03-06T10:34:30Z

Checkmarx One – Scan Summary & Details – 66f21fde-5ec5-4f66-9529-290bed36607a

Policy Management Violations

Policy Name	Rule(s)	Break Build
[SAST-ML0] Not allowed NEW Sast vulnerabilities		true

New Issues

Severity	Issue	Source File / Package	Checkmarx Insight
	IAM Access Analyzer Not Enabled	/positive1.tf: 1	IAM Access Analyzer should be enabled and configured to continuously monitor resource permissions

OrShamirCM

Great work!

OrShamirCM · 2024-03-06T13:17:22Z

internal/commands/scan.go

@@ -185,6 +189,7 @@ func NewScanCommand(
 kicsRealtimeCmd := scanRealtimeSubCommand()

 scaRealtimeCmd := scarealtime.NewScaRealtimeCommand(scaRealTimeWrapper)
+ containerResolver = containerResolverWrapper


move it to the start of the method

OrShamirCM · 2024-03-06T13:17:52Z

internal/commands/scan.go

@@ -502,7 +509,8 @@ func scanCreateSubCommand(
 "",
 fmt.Sprintf("Parameters to use in SCA resolver (requires --%s).", commonParams.ScaResolverFlag),
 )
- createScanCmd.PersistentFlags().String(commonParams.ScanTypes, "", "Scan types, ex: (sast,iac-security,sca,api-security)")
+ createScanCmd.PersistentFlags().String(commonParams.ContainerImagesFlag, "", "List of container images to scan, ex: manuelbcd/vulnapp:latest,debian:10")


the flag suppose to be under FF (in help too)

OrShamirCM · 2024-03-06T13:21:52Z

internal/commands/scan.go

+ scanTypes, _ := cmd.Flags().GetString(commonParams.ScanTypes)
+ var scanTypeList []string
+ if len(scanTypes) > 0 {
+ scanTypeList = strings.Split(scanTypes, ",")


if possible to add space *

OrShamirCM · 2024-03-06T13:25:27Z

internal/commands/scan.go

+ return true
+ }
+ //check if user license supports only container scanning
+ actualScanTypeList := strings.Split(actualScanTypes, ",")


change the name form actual to permissions / licenses (if it's not too complex)

internal/commands/scan.go

OrShamirCM · 2024-03-06T16:13:53Z

internal/commands/scan.go

+ containerImagesList = strings.Split(strings.TrimSpace(containerImages), ",")
+ for _, containerImageName := range containerImagesList {
+ if containerImagesErr := validateContainerImageFormat(containerImageName); containerImagesErr != nil {
+ return containerImagesErr


add log error in validate when error

OrShamirCM · 2024-03-06T16:14:42Z

internal/commands/scan.go

+ }
+ containerResolverERR := containerResolver.Resolve(directoryPath, directoryPath, containerImagesList, debug)
+ if containerResolverERR != nil {
+ return containerResolverERR


add log error for container resolver

OrShamirCM · 2024-03-06T16:16:42Z

internal/commands/scan.go

@@ -2474,6 +2554,14 @@ func validateCreateScanFlags(cmd *cobra.Command) error {
 return nil
 }

+func validateContainerImageFormat(containerImage string) error {
+ imageParts := strings.Split(containerImage, ":")


you can add trim

OrShamirCM · 2024-03-06T16:20:15Z

internal/commands/scan_test.go

+ execCmdNilAssertion(t, append(baseArgs, "-s", blankSpace+"."+blankSpace)...)
+}
+
+func TestCreateScan_ContainerImagesFlagWithoutValue_FailCreatingScan(t *testing.T) {


is single container type with license

OrShamirCM · 2024-03-06T16:21:06Z

internal/commands/scan_test.go

+ err := execCmdNotNilAssertion(t, "scan", "create", "--project-name", "MOCK", "-s", dummyRepo, "-b", "dummy_branch", "--container-images")
+ assert.Assert(t, err.Error() == "flag needs an argument: --container-images")
+}
+


add zip case for unit without flags / filters + flags/ filters

tamarleviCm

I reviewed up to utils_test.go

tamarleviCm · 2024-03-24T10:20:18Z

internal/commands/scan.go

@@ -502,7 +509,18 @@ func scanCreateSubCommand(
 "",
 fmt.Sprintf("Parameters to use in SCA resolver (requires --%s).", commonParams.ScaResolverFlag),
 )
- createScanCmd.PersistentFlags().String(commonParams.ScanTypes, "", "Scan types, ex: (sast,iac-security,sca,api-security)")
+ if len(wrappers.FeatureFlags) == 0 {


why do you need this condition?
if FF is not received you have to know it before (when you call the FF service) and fail the command if FF is necessary
but maybe you have to ignore containers when FF was not received

tamarleviCm · 2024-03-24T10:22:28Z

internal/commands/scan.go

@@ -1000,6 +1022,7 @@ func addScaScan(cmd *cobra.Command, resubmitConfig []wrappers.Config) map[string
 scaConfig.Filter, _ = cmd.Flags().GetString(commonParams.ScaFilterFlag)
 scaConfig.LastSastScanTime, _ = cmd.Flags().GetString(commonParams.LastSastScanTime)
 scaConfig.PrivatePackageVersion, _ = cmd.Flags().GetString(commonParams.ScaPrivatePackageVersionFlag)
+ scaConfig.EnableContainersScan = !(wrappers.FeatureFlags[wrappers.ContainerEngineCLIEnabled] && userAllowedEngines[commonParams.ContainersType])


Suggested change

scaConfig.EnableContainersScan = !(wrappers.FeatureFlags[wrappers.ContainerEngineCLIEnabled] && userAllowedEngines[commonParams.ContainersType])

scaConfig.EnableContainersScan = (wrappers.FeatureFlags[wrappers.ContainerEngineCLIEnabled] && userAllowedEngines[commonParams.ContainersType])

It should be - not the condition

tamarleviCm · 2024-03-24T10:27:58Z

internal/commands/scan.go

@@ -1106,6 +1143,11 @@ func compressFolder(sourceDir, filter, userIncludeFilter, scaResolver string) (s
 return outputFile.Name(), err
 }

+func isSingleContainerScanTriggered() bool {
+ scanTypeList := strings.Split(actualScanTypes, ",")


actualScanTypes could include spaces?

tamarleviCm · 2024-03-24T10:32:41Z

internal/wrappers/feature-flags.go

@@ -21,6 +21,10 @@ var FeatureFlagsBaseMap = []CommandFlags{
 Name: MinioEnabled,
 Default: true,
 },
+ {


OrShamirCM

Should it be closed? all merged to other branch?
If so please close and delete the branch

…ainer scan tests

tamarleviCm · 2024-03-26T08:43:53Z

internal/wrappers/jwt-helper.go


 var defaultEngines = map[string]bool{
 "sast": true,
 "sca": true,
 "api-security": true,
 "iac-security": true,
+ "containers": true,


what is the bool here?

if this engines is allowed, I deleted the containers engine if the FF is off

tamarleviCm · 2024-03-26T09:02:09Z

internal/wrappers/scans.go

@@ -138,6 +138,9 @@ type ScaConfig struct {
 ExploitablePath string `json:"ExploitablePath,omitempty"`
 LastSastScanTime string `json:"LastSastScanTime,omitempty"`
 PrivatePackageVersion string `json:"privatePackageVersion,omitempty"`
+ EnableContainersScan bool `json:"enableContainersScan,omitempty"`
+}
+type ContainerConfig struct {


why do you need it if it empty

In the future if we will required to pass some information there.
This is the same configuration handling we do in all of our engines, I thought it will be a good idea to do with container engine the same.

BenAlvo1 added 5 commits February 29, 2024 18:23

AST-36236 | Support Containers as New Scan Type With Realted Flags

785ef78

AST-36236 | fix lint errors

c669f9f

AST-36237 | added container resolver functionality

2cbb883

AST-36237 | added container resolver functionality

da60826

AST-36237 | added integration tests and refactor compressFolder func

fda814a

pedrompflopes requested review from a team, margaritalm and pedrompflopes and removed request for a team March 6, 2024 10:32

github-actions bot added the feature label Mar 6, 2024

AlvoBen requested review from tiagobcx and OrShamirCM and removed request for margaritalm and pedrompflopes March 6, 2024 10:32

AlvoBen marked this pull request as draft March 6, 2024 10:32

BenAlvo1 and others added 5 commits March 6, 2024 12:44

update go.mod

130717e

update go.sum

02c398a

Merge branch 'main' into feature/benalvo/add-container-engine-logic

d96c614

update functionallity

31cb42d

update functionallity

0f01d99

OrShamirCM reviewed Mar 6, 2024

View reviewed changes

BenAlvo1 added 8 commits March 10, 2024 13:07

resolve conversation comments

747dc82

resolve conversation comments

b1d6467

resolve conversation comments

04a254f

fix compilation errors

c40d3d6

fix compilation errors

05ea5e8

fix compilation errors

8801d9b

fix compilation errors

89cf40d

fix compilation errors

beb0711

BenAlvo1 added 2 commits March 21, 2024 11:49

delete default initialization to ContainerEngineCliEnabled flag

160a0f5

added featurflagWrapper to scanCommand constructor

098b327

tamarleviCm reviewed Mar 24, 2024

View reviewed changes

OrShamirCM reviewed Mar 24, 2024

View reviewed changes

BenAlvo1 added 8 commits March 24, 2024 17:54

remove default initialization of ContainerEngineCLIEnabled

4b85af3

upgrade container resolver to version 1.0.3

a7f41e6

update unitests

6f74c7d

update tests

a6b3bc2

upgrade container-resolver version and changed test zip file for cont…

daa0347

…ainer scan tests

fix containerEngine FF handling

2c135ee

fix containerEngine FF handling

6c5cffb

fix containerEngine FF handling

21c81c2

tamarleviCm reviewed Mar 26, 2024

View reviewed changes

BenAlvo1 and others added 3 commits March 26, 2024 11:13

fix containerEngine FF handling

e459682

Merge branch 'main' into feature/benalvo/add-container-engine-logic

0d61618

fix containerEngine FF handling

7db192c

AlvoBen changed the title ~~Feature/benalvo/add container engine logic (AST-36237, AST-36236)~~ CLI | Support Container Engine Scan (AST-36237, AST-36236) Mar 26, 2024

BenAlvo1 added 4 commits March 26, 2024 16:36

check unit tests

437e4b4

check unit tests

e3d7986

check unit tests

008cad2

check unit tests

bfcd0e3

AlvoBen marked this pull request as ready for review March 26, 2024 17:27

pedrompflopes requested review from a team and hmmachadocx and removed request for a team March 26, 2024 17:28

finish tests

0a31598

AlvoBen changed the title ~~CLI | Support Container Engine Scan (AST-36237, AST-36236)~~ CLI | Support Container Engine Scan (AST-36237, AST-36236, AST-36238) Mar 27, 2024

BenAlvo1 added 2 commits March 27, 2024 11:15

initialized feature flags in root.go

1395fa3

add featureflag check to tests

6bddf3f

tamarleviCm closed this Jun 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLI | Support Container Engine Scan (AST-36237, AST-36236, AST-36238) #672

CLI | Support Container Engine Scan (AST-36237, AST-36236, AST-36238) #672

AlvoBen commented Mar 6, 2024 •

edited

github-actions bot commented Mar 6, 2024 •

edited

OrShamirCM left a comment

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

OrShamirCM Mar 6, 2024

tamarleviCm left a comment

tamarleviCm Mar 24, 2024

tamarleviCm Mar 24, 2024

AlvoBen Mar 24, 2024

tamarleviCm Mar 24, 2024

AlvoBen Mar 24, 2024

tamarleviCm Mar 24, 2024

OrShamirCM left a comment

tamarleviCm Mar 26, 2024

AlvoBen Mar 26, 2024

tamarleviCm Mar 26, 2024

AlvoBen Mar 26, 2024

	scaConfig.EnableContainersScan = !(wrappers.FeatureFlags[wrappers.ContainerEngineCLIEnabled] && userAllowedEngines[commonParams.ContainersType])
	scaConfig.EnableContainersScan = (wrappers.FeatureFlags[wrappers.ContainerEngineCLIEnabled] && userAllowedEngines[commonParams.ContainersType])

CLI | Support Container Engine Scan (AST-36237, AST-36236, AST-36238) #672

CLI | Support Container Engine Scan (AST-36237, AST-36236, AST-36238) #672

Conversation

AlvoBen commented Mar 6, 2024 • edited

Description

References

Testing

Checklist

github-actions bot commented Mar 6, 2024 • edited

Policy Management Violations

New Issues

OrShamirCM left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tamarleviCm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OrShamirCM left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlvoBen commented Mar 6, 2024 •

edited

github-actions bot commented Mar 6, 2024 •

edited