chore(terraform): hook into evaluateStep behavior with custom hooks

[Emyrk]

Description

Let me know if I am missing some other means to make this work

evaluateStep handles many terraform semantics such as default values for 'variables'. A hook into these steps allows defining additional semantics, likely to mirror those of the actual provider implementation.

This must be done in evaluateStep, as things such as expandBlocks occur afterwards. Without this hook, the ExpandBlock must be called manually outside of the executor.

data "your_custom_data" "this" {
  default = ["a", "b", "c"]
}

data "random_thin" "that" {
  dynamic "repeated" {
    for_each = data.your_custom_data.this.value
    content {
      value = repeated.value
    }
  }
}

Conversation

Calling ExpandBlock outside is ok, however I wonder if calling EvaluateStep again is required in order to be correct. EvaluateSteps is called here after ExpandBlocks

https://github.com/aquasecurity/trivy/blob/main/pkg/iac/scanners/terraform/parser/evaluator.go#L145

Hence why this hook was added. To place similar semantic code to variable defaults in the same spot.

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[nikpivkin]

Hi @Emyrk ! Interesting idea. We have a way to set predefined values for blocks here, but this isn't called in the parser directly, making it impossible to pass custom options.

@simar7 Your thoughts on using iac/terraform as a library?

[Emyrk]

@nikpivkin I did see those options. I had considered seeing if those presets could be added in as hooks.

I am also unsure if I can make my provider block implementation purely in the context of terraform. I am working with the coder/coder provider. Specifically coder_parameter which has a default attribute and acts very similarly to how variables work.

In terraform, the user defined values are passed in as environment variables. For the presets block, I do not like the idea of using os.Env, and I'm not sure how to pass in arbitrary context.

You can see the hook I made here: https://github.com/Emyrk/terraform-eval/blob/4d10683c15ecf202a4fa1377dd9ddc7d331f5dac/engine/coderism/extract.go#L46-L81

---

@simar7 Your thoughts on using iac/terraform as a library?

I appreciate the fast feedback on my PRs and thoughts. Your package is pretty amazing that I can evaluate the language without having to run a terraform plan.

As you are likely aware, the terraform language is behind /internal in the terraform main repository. Making it very difficult to use without copying/forking.

[Emyrk]

@nikpivkin, @simar7 any updated thoughts?

I am happy to explore other methods to accomplish this. I am hoping this is the last change (:crossed_fingers:) to fully enable my use case using this as a library over some others.

[simar7]

hi @Emyrk sorry for the late response. While I like the idea you have, I'm not sure how beneficial it is to the Trivy project.

As for my thoughts on making iac/terraform a library, while I'm not against it, it does add an additional maintenance overhead which I'm not sure if we have the capacity at this time for. Making a package into a library establishes an inherent contract with the users of the library, which we will have to abide by.

Certainly open to ideas.

[Emyrk]

@simar7

As for my thoughts on making iac/terraform a library, while I'm not against it, it does add an additional maintenance overhead which I'm not sure if we have the capacity at this time for.

If it adds any value, if the project I am working on adopts trivy's tf parsing as the underlying library, we are happy to add our resources to maintaining the common overlap. The being the evaluation of the terraform.

Making a package into a library establishes an inherent contract with the users of the library, which we will have to abide by.

I do understand, and do not intend to drift from the primary purpose of the package. I admit I am still in the evaluating stage of things, so if my use case is not an intended case, I understand if I have to look elsewhere.

If it's possible and not too much work, a simple doc.go or README that outlines some of the objectives and scope of the package would be very helpful.

[Emyrk]

@simar7 apologies to keep bringing this up.

If I back up a little bit, and define the things I am trying to solve, maybe it would be easier to determine if these features have a place in this library.

The two things I am trying to do:

1. Preload the eval context with values. I intend to load in terraform plan output, and possibly even tfstate. This is so I can reference resolved data blocks if I have this information.
2. Apply semantics to arbitrary data blocks. This effectively "simulates" what the terraform provider would do for a given block, when the semantics can be done locally (without some external api or something). This is very similar to the presets you referred to in how some of your aws blocks are handled today.