Scan cis-dil-benchmark can't start. CIS level warn

[nbublikov]

Describe the bug
I got WARN: Input 'cis_level' does not have a value, when in config cis level exist (see attached screen)

Expected behavior
i got scan my sshd server, but all test are skipped or failed

Actual behavior

Example code
sudo inspec exec /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb -t ssh://root@xxx --password=

OS / Environment

uname -a

Linux comm99-dev 4.14.98 #1 SMP PREEMPT Wed Mar 17 21:18:09 MSK 2021 armv7l GNU/Linux

Inspec Version
4.37.17

Baseline Version
https://github.com/dev-sec/cis-dil-benchmark

Additional context
please see attached screenshot

[nbublikov]

please see all output from command

sudo inspec exec /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb -t ssh://root@xxxx --password= [2021-05-26T13:03:53+03:00] WARN: Input 'cis_level' does not have a value. Use --input-file or --input to provide a value for 'cis_level' or specify a value with input('cis_level', value: 'somevalue', ...). [2021-05-26T13:03:53+03:00] WARN: Input 'cis_level' does not have a value. Use --input-file or --input to provide a value for 'cis_level' or specify a value with input('cis_level', value: 'somevalue', ...)`.

Profile: tests from /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb (tests from .opt.inspec.test.cis-dil-benchmark.controls.5_2_ssh_server_configuration.rb)
Version: (not specified)
Target: ssh://root@xxxx:22

× cis-dil-benchmark-5.2.1: Ensure permissions on /etc/ssh/sshd_config are configured (Scored) (8 failed)
✔ File /etc/ssh/sshd_config is expected to exist
× File /etc/ssh/sshd_config is expected not to be readable by group
expected File /etc/ssh/sshd_config not to be readable by group
× File /etc/ssh/sshd_config is expected not to be writable by group
expected File /etc/ssh/sshd_config not to be writable by group
× File /etc/ssh/sshd_config is expected not to be executable by group
expected File /etc/ssh/sshd_config not to be executable by group
× File /etc/ssh/sshd_config is expected not to be readable by other
expected File /etc/ssh/sshd_config not to be readable by other
× File /etc/ssh/sshd_config is expected not to be writable by other
expected File /etc/ssh/sshd_config not to be writable by other
× File /etc/ssh/sshd_config is expected not to be executable by other
expected File /etc/ssh/sshd_config not to be executable by other
× File /etc/ssh/sshd_config uid is expected to cmp == 0

 expected: 0
      got: 
 
 (compared using `cmp` matcher)

 ×  File /etc/ssh/sshd_config gid is expected to cmp == 0
 
 expected: 0
      got: 
 
 (compared using `cmp` matcher)

↺ cis-dil-benchmark-5.2.4: Ensure SSH Protocol is set to 2 (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.5: Ensure SSH LogLevel is appropriate (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.6: Ensure SSH X11 forwarding is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.7: Ensure SSH MaxAuthTries is set to 4 or less (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.8: Ensure SSH IgnoreRhosts is enabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.9: Ensure SSH HostbasedAuthentication is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.10: Ensure SSH root login is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.11: Ensure SSH PermitEmptyPasswords is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.12: Ensure SSH PermitUserEnvironment is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.13: Ensure only strong Ciphers are used (Scored) (1 failed) (1 skipped)
↺ Can't find file: /etc/ssh/sshd_config
× Control Source Code Error /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb:286
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.14: Ensure only strong MAC algorithms are used (Scored) (1 failed) (1 skipped)
↺ Can't find file: /etc/ssh/sshd_config
× Control Source Code Error /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb:324
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.15: Ensure only strong Key Exchange algorithms are used (Scored) (1 failed) (1 skipped)
↺ Can't find file: /etc/ssh/sshd_config
× Control Source Code Error /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb:357
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.16: Ensure SSH Idle Timeout Interval is configured (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.17: Ensure SSH LoginGraceTime is set to one minute or less (Scored)
↺ Can't find file: /etc/ssh/sshd_config
× cis-dil-benchmark-5.2.18: Ensure SSH access is limited (Scored) (4 failed)
× SSHD Configuration AllowUsers
Can't find file: /etc/ssh/sshd_config
× SSHD Configuration AllowGroups
Can't find file: /etc/ssh/sshd_config
× SSHD Configuration DenyUsers
Can't find file: /etc/ssh/sshd_config
× SSHD Configuration DenyGroups
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.19: Ensure SSH warning banner is configured (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.20: Ensure SSH PAM is enabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.21: Ensure SSH AllowTcpForwarding is disabled (Scored)
↺ Skipped control due to only_if condition.
↺ cis-dil-benchmark-5.2.22: Ensure SSH MaxStartups is configured (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.23: Ensure SSH MaxSessions is set to 4 or less (Scored)
↺ Can't find file: /etc/ssh/sshd_config

Profile Summary: 0 successful controls, 5 control failures, 16 controls skipped
Test Summary: 1 successful, 15 failures, 19 skipped
`

[nbublikov]

Many test are skipped, for example
↺ cis-dil-benchmark-5.2.10: Ensure SSH root login is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config

actually root login is enabled, but why test can't catch this?

[nbublikov]

upd: i see cis level WARN when i start any test form "Controls"

[deric4]

I think there a few things going on:

Example code
sudo inspec exec /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb -t ssh://root@xxx --password=

When using a "Local single test" rather than the specifying the_profile directory_ , inputs and custom resources aren't allowed

Typically if I'm testing a specific section (i.e. 5.2), I would run the following which respects the default input values for the profile

$  inspec exec /opt/inspec/test/cis-dil-benchmark -t ssh://root@<ip address> --password < your pass> --controls=/cis-dil-benchmark-5.2/ --sudo

I can't think of a reason why inspec can't find /etc/ssh/sshd_config when connecting as root though. Not being able to find the file seems to be whats causing all the failures you posted 🤔

Would you be able to provide the output from re-running the command like the example above (specifying profile directory, and controls w/ regex)?

[nbublikov]

Hi deric4, thank for feedback and help
Please, see output

`Profile: CIS Distribution Independent Linux Benchmark Profile (cis-dil-benchmark)
Version: 0.4.11
Target: ssh://root@xxxx

× cis-dil-benchmark-5.2.1: Ensure permissions on /etc/ssh/sshd_config are configured (Scored) (8 failed)
✔ File /etc/ssh/sshd_config is expected to exist
× File /etc/ssh/sshd_config is expected not to be readable by group
expected File /etc/ssh/sshd_config not to be readable by group
× File /etc/ssh/sshd_config is expected not to be writable by group
expected File /etc/ssh/sshd_config not to be writable by group
× File /etc/ssh/sshd_config is expected not to be executable by group
expected File /etc/ssh/sshd_config not to be executable by group
× File /etc/ssh/sshd_config is expected not to be readable by other
expected File /etc/ssh/sshd_config not to be readable by other
× File /etc/ssh/sshd_config is expected not to be writable by other
expected File /etc/ssh/sshd_config not to be writable by other
× File /etc/ssh/sshd_config is expected not to be executable by other
expected File /etc/ssh/sshd_config not to be executable by other
× File /etc/ssh/sshd_config uid is expected to cmp == 0

 expected: 0
      got: 
 
 (compared using `cmp` matcher)

 ×  File /etc/ssh/sshd_config gid is expected to cmp == 0
 
 expected: 0
      got: 
 
 (compared using `cmp` matcher)

↺ cis-dil-benchmark-5.2.4: Ensure SSH Protocol is set to 2 (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.5: Ensure SSH LogLevel is appropriate (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.6: Ensure SSH X11 forwarding is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.7: Ensure SSH MaxAuthTries is set to 4 or less (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.8: Ensure SSH IgnoreRhosts is enabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.9: Ensure SSH HostbasedAuthentication is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.10: Ensure SSH root login is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.11: Ensure SSH PermitEmptyPasswords is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.12: Ensure SSH PermitUserEnvironment is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.13: Ensure only strong Ciphers are used (Scored) (1 failed) (1 skipped)
↺ Can't find file: /etc/ssh/sshd_config
× Control Source Code Error /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb:286
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.14: Ensure only strong MAC algorithms are used (Scored) (1 failed) (1 skipped)
↺ Can't find file: /etc/ssh/sshd_config
× Control Source Code Error /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb:324
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.15: Ensure only strong Key Exchange algorithms are used (Scored) (1 failed) (1 skipped)
↺ Can't find file: /etc/ssh/sshd_config
× Control Source Code Error /opt/inspec/test/cis-dil-benchmark/controls/5_2_ssh_server_configuration.rb:357
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.16: Ensure SSH Idle Timeout Interval is configured (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.17: Ensure SSH LoginGraceTime is set to one minute or less (Scored)
↺ Can't find file: /etc/ssh/sshd_config
× cis-dil-benchmark-5.2.18: Ensure SSH access is limited (Scored) (4 failed)
× SSHD Configuration AllowUsers
Can't find file: /etc/ssh/sshd_config
× SSHD Configuration AllowGroups
Can't find file: /etc/ssh/sshd_config
× SSHD Configuration DenyUsers
Can't find file: /etc/ssh/sshd_config
× SSHD Configuration DenyGroups
Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.19: Ensure SSH warning banner is configured (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.20: Ensure SSH PAM is enabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.21: Ensure SSH AllowTcpForwarding is disabled (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.22: Ensure SSH MaxStartups is configured (Scored)
↺ Can't find file: /etc/ssh/sshd_config
↺ cis-dil-benchmark-5.2.23: Ensure SSH MaxSessions is set to 4 or less (Scored)
↺ Can't find file: /etc/ssh/sshd_config

Profile Summary: 0 successful controls, 5 control failures, 16 controls skipped
Test Summary: 1 successful, 15 failures, 19 skipped`

[nbublikov]

also i try write simple sshd test, it was skipped

maybe cause in what my OS not supported by inspec?
see last messages
dev-sec/ssh-baseline#196