Audit cookbook does not automatically skip profiles for a different platform type

[grdnrio]

Cookbook version

4.0.0

Chef-client version

12.19.36

Platform Details

Amazon Linux and Windows 2012 R2 (AWS default AMIs)

Scenario:

I have an audit role that is added to a base role in the run-list of all nodes. My attributes are set as follows:

{
  "audit": {
    "fetcher": "chef-server",
    "reporter": "chef-automate",
    "profiles": [
      {
        "name": "linux-baseline",
        "compliance": "delivery/linux-baseline"
      },
      {
        "name": "linux-patch-baseline",
        "compliance": "delivery/linux-patch-baseline"
      },
      {
        "name": "windows-baseline",
        "compliance": "delivery/windows-baseline"
      },
      {
        "name": "windows-patch-baseline",
        "compliance": "delivery/windows-patch-baseline"
      }
    ]
  }
}

Steps to Reproduce:

See attributes and versions above.

Expected Result:

I'd like the audit cookbook to determine the platform and skip profiles that are not relevant.

Actual Result:

At the moment an error is thrown:

Running handlers:
[2017-06-20T13:46:51+00:00] INFO: Running report handlers
[2017-06-20T13:46:52+00:00] WARN: Format is json
[2017-06-20T13:46:52+00:00] INFO: Initialize InSpec 1.25.1
[2017-06-20T13:46:52+00:00] ERROR: Report handler Chef::Handler::AuditReport raised #<RuntimeError: This OS/platform (amazon) is not supported by this profile.>
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.25.1/lib/inspec/runner.rb:188:in `supports_profile?'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/inspec-1.25.1/lib/inspec/runner.rb:175:in `add_target'
[2017-06-20T13:46:52+00:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:134:in `block in call'
[2017-06-20T13:46:52+00:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:134:in `each'
[2017-06-20T13:46:52+00:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:134:in `call'
[2017-06-20T13:46:52+00:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:57:in `report'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:259:in `run_report_unsafe'
[2017-06-20T13:46:52+00:00] ERROR: /var/chef/cache/cookbooks/audit/files/default/handler/audit_report.rb:75:in `run_report_safely'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:125:in `block in run_report_handlers'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:123:in `each'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:123:in `run_report_handlers'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/handler.rb:135:in `block in <class:Handler>'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:441:in `block in run_completed_successfully'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:440:in `each'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:440:in `run_completed_successfully'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/client.rb:299:in `run'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:295:in `block in fork_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:283:in `fork_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:248:in `block in run_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/local_mode.rb:44:in `with_server_connectivity'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:236:in `run_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `loop'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:450:in `interval_run_chef_client'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application/client.rb:434:in `run_application'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/lib/chef/application.rb:59:in `run'
[2017-06-20T13:46:52+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36/bin/chef-client:26:in `<top (required)>'
[2017-06-20T13:46:52+00:00] ERROR: /usr/bin/chef-client:57:in `load'
[2017-06-20T13:46:52+00:00] ERROR: /usr/bin/chef-client:57:in `<main>'
  - Chef::Handler::AuditReport
Running handlers complete
[2017-06-20T13:46:52+00:00] INFO: Report handlers complete
Chef Client finished, 0/11 resources updated in 02 seconds

[grdnrio]

If I remove the Windows references from the attributes file and run chef-client the audit completes successfully.

[upUJTGifCXdJnqACJgxz]

@grdnrio What path did you take here? I'm polluting my attributes file with conditions like

when 'centos'
    default['audit']['profiles'] = [
        { "name": "CIS CentOS Level 1", "compliance": "cis/cis-centos6-level1" }
    ]
end

For "platforms" that don't have a matching profile, I'm going to see if you can create a comment-only InSpec rule. The cookbook fails if default['audit']['profiles'] is set to an empty array.

[username-is-already-taken2]

I've noticed this is still an issue with version of 7.0.1 of the cookbook. I've made use of inspec's ability to know what controls need to run so I have developed multi os controls around individual products profiles.

Do you think this will resolved in the future? as I will would need to start splitting out my profiles as I would like to move to the audit cookbook approach for running my compliance scans.

[grdnrio]

Need @chris-rock to comment on this :)

[chris-rock]

I understand this is indeed an issue. The reason why this happens is that each profile executes individually and fails because it is not applicable. You could work around that if you place all the profiles into a wrapper profile that depends on all the profiles.

I think it makes sense for the audit cookbook to understand the read and skip exception. As part of inspec/inspec#3158 we discussed to define a mechanism to report a skipped profile which would be beneficial for that issue as well.