Only run terraform plan for changes in the working directory

[cfsnate]

Feature description 💡

When using a monorepo and multiple workspaces, default behavior is to auto plan on every commit.

Would be nice to have an option to plan only on commits with changes that occur within the workspace working directory path.

Anything else?

No response

[cfsnate]

To add on to this. I may be missing something but it appears the automatic action is to run the Plan + Apply template on commits.

Is it possible to modify this behavior to run a different template on VCS changes? Or is the best action to modify the stock Plan + Apply template to do what's wanted?

[alfespa17]

Hello @cfsnate

The default behavior is that it will execute a "plan+apply template in the organization" if you do a push in the same branch that you used when creating the workspace.

You could change the template and for example add an approval part, you can go to the organization settings and change a little bit the template, it could be something like this:

flow:
  - name: "Plan"
    type: "terraformPlan"
    step: 100
  - type: "approval"
    name: "Approve Plan"
    step: 150
    team: "<APPROVER GROUP BASED ON DEX CONFIG>"
  - name: "Apply"
    type: "terraformApply"
    step: 200

The webhooks support was added recently, for now you can only use the "plan+apply" template but you could do some customization like the above logic to add an approval or any other customization.

In the future we could add a feature to execute differente templates depending on webhooks events, like for example just running a plan for changes in some specific branches.

[alfespa17]

@jcanizalez you added the webhook support recently not sure if I am missing something in the above coment.

[cfsnate]

Awesome, yes this answers my second question regarding Plan + Apply. Will test with adding the approval, should work. The downside would be that the Team in step 150 would be the same for every workspace within the org. If you wanted different approvers you'd need to be able to set a template per workspace that's triggered via vcs push instead of using the Plan + Apply.

The first question is around monitoring changes per workspace only in the subfolder thats relevant to that workspace. This is largely related to monorepos only.

When you have many workspaces that all reside within the same Git Repo, you only want to trigger the run based on commits which affect files in the subfolder which is associated with that workspace. Does that make sense?

[alfespa17]

Yes that is correct, if you change the template in the step 150 it will affect every workspace in the organization. Maybe the best option in this case would be to add something like an option to define the default template when creating the workspace so you can have different approvers.

Regarding your last question maybe @jcanizalez can help you because I am not sure if the webhook logic that was added check for specific folder changes.

[jcanizalez]

hi @cfsnate
At this moment Terrakube don't have the option to filter the folder that triggers the webhook. Basically the webhook is executed after each push. But I think we can add this feature, don't looks complicated.

Regarding the default template, in the design we consider the scenario to have different different webhooks for multiple events (so you can run a template when a pr is created, or you can run a different action if someone added a comment). In future versions we will complete this feature. At the moment, the only work around if you need to change the template for each workspace, you can do it after you create the workspace using the api:

PUT <api url>/api/v1/webhook/<id>
templateMapping: Id of the template
referenceId: Id of the Workspace

Or directly in the database, in the webhook table.

• template_mapping: Id of the template
• reference_id : Id of the Workspace

[cfsnate]

Excellent, thanks for the info!

-Nate

[alfespa17]

This was partially fix by #663 it will work for workspaces using a Github VCS connection and will only run a job for the affected folder and commit id.

But I think the webhook code will need some refactor because it is creating one webhook for each workspace and there is a limit for webhooks inside a repository according to this.

So I guess it will be limited to 20 workspaces in a repository for now

[alfespa17]

Fixed for gitlab workspaces using a private vcs connection in #678. Now changes in push events will only affect workspaces with files changes.

Not sure if gitlab has the same restriction for 20 webhooks.

[alfespa17]

Fixe file changes in a working directory for bitbucket workspace using a private vcs connection in PR #681

[adamlc]

In Terraform Cloud and OTF, which I'm migrating from. You could specify the paths that would cause the run to trigger.

We too use a mono repo setup for our different environments. But we also have a modules folder in the root directory, which should also trigger the build, but the working directory isn't in the root.

For example here are the options in OTF and it was also the same in terraform cloud when we used that. This would be amazing if it could be added. It's great it already works for the workspace path!

[alfespa17]

In Terraform Cloud and OTF, which I'm migrating from. You could specify the paths that would cause the run to trigger.

We too use a mono repo setup for our different environments. But we also have a modules folder in the root directory, which should also trigger the build, but the working directory isn't in the root.

For example here are the options in OTF and it was also the same in terraform cloud when we used that. This would be amazing if it could be added. It's great it already works for the workspace path!

Hello @adamlc, I have created a new issue based on your feedback here