feat: Support project-level Terraform distribution selection #5167

abborg · 2024-12-15T22:13:39Z

what

Add terraform_distribution project config value
Project-level terraform_distribution config value overrides server default
Refactor terraform client to avoid cyclic dependencies

why

Enable Atlantis to manage multiple stacks or repos with different TF distributions
Supports migration from Terraform to OpenTofu project-by-project to de-risk migration

tests

I have tested my changes by running unit tests
I have tested my changes by creating a test PR that exercises the new functionality

references

Supports #3741

Add support for terraform_distribution config value in project config. This config value behaves similarly to terraform_version whereby defaults taken from the server config will be overridden by project-level values. Also refactor to prevent terraform client slightly to prevent cyclical dependencies. Signed-off-by: Andrew Borg <[email protected]>

jamengual · 2024-12-16T07:18:53Z

@abborg thanks for the contribution, please gives us some time to review.

X-Guardian · 2024-12-17T12:27:41Z

@abborg, can you update the relevant documentation in the runatlantis.io directory with your changes.

Signed-off-by: Andrew Borg <[email protected]>

abborg · 2024-12-17T14:25:55Z

I've updated the relevant documentation. Please let me know if there's any further changes desired.

Signed-off-by: Andrew Borg <[email protected]>

X-Guardian

Looking good @abborg. Just a few suggestions.

X-Guardian · 2024-12-18T13:13:27Z

cmd/server.go

+	// TFDistributionFlag is deprecated for DefaultTFDistributionFlag
+	TFDistributionFlag        = "tf-distribution"
+	TFDownloadFlag            = "tf-download"
+	TFDownloadURLFlag         = "tf-download-url"
+	UseTFPluginCache          = "use-tf-plugin-cache"
+	VarFileAllowlistFlag      = "var-file-allowlist"
+	VCSStatusName             = "vcs-status-name"
+	IgnoreVCSStatusNames      = "ignore-vcs-status-names"
+	TFEHostnameFlag           = "tfe-hostname"
+	TFELocalExecutionModeFlag = "tfe-local-execution-mode"
+	TFETokenFlag              = "tfe-token"
+	WriteGitCredsFlag         = "write-git-creds" // nolint: gosec
+	WebBasicAuthFlag          = "web-basic-auth"
+	WebUsernameFlag           = "web-username"
+	WebPasswordFlag           = "web-password"
+	WebsocketCheckOrigin      = "websocket-check-origin"


Suggested change

// TFDistributionFlag is deprecated for DefaultTFDistributionFlag

TFDistributionFlag = "tf-distribution"

TFDownloadFlag = "tf-download"

TFDownloadURLFlag = "tf-download-url"

UseTFPluginCache = "use-tf-plugin-cache"

VarFileAllowlistFlag = "var-file-allowlist"

VCSStatusName = "vcs-status-name"

IgnoreVCSStatusNames = "ignore-vcs-status-names"

TFEHostnameFlag = "tfe-hostname"

TFELocalExecutionModeFlag = "tfe-local-execution-mode"

TFETokenFlag = "tfe-token"

WriteGitCredsFlag = "write-git-creds" // nolint: gosec

WebBasicAuthFlag = "web-basic-auth"

WebUsernameFlag = "web-username"

WebPasswordFlag = "web-password"

WebsocketCheckOrigin = "websocket-check-origin"

TFDistributionFlag = "tf-distribution" // deprecated for DefaultTFDistributionFlag

TFDownloadFlag = "tf-download"

TFDownloadURLFlag = "tf-download-url"

UseTFPluginCache = "use-tf-plugin-cache"

VarFileAllowlistFlag = "var-file-allowlist"

VCSStatusName = "vcs-status-name"

IgnoreVCSStatusNames = "ignore-vcs-status-names"

TFEHostnameFlag = "tfe-hostname"

TFELocalExecutionModeFlag = "tfe-local-execution-mode"

TFETokenFlag = "tfe-token"

WriteGitCredsFlag = "write-git-creds" // nolint: gosec

WebBasicAuthFlag = "web-basic-auth"

WebUsernameFlag = "web-username"

WebPasswordFlag = "web-password"

WebsocketCheckOrigin = "websocket-check-origin"

Reduce the diff.

X-Guardian · 2024-12-18T13:20:01Z

cmd/server_test.go

@@ -73,6 +73,7 @@ var testFlags = map[string]interface{}{
 	CheckoutStrategyFlag:             CheckoutStrategyMerge,
 	CheckoutDepthFlag:                0,
 	DataDirFlag:                      "/path",
+	DefaultTFDistributionFlag:        "terraform",


Can you add a ValidateDefaultTFDistribution test?

X-Guardian · 2024-12-18T13:29:54Z

server/core/config/raw/project.go

+func validDistribution(value interface{}) error {
+	distribution := value.(*string)
+	if distribution != nil && *distribution != "terraform" && *distribution != "opentofu" {
+		return fmt.Errorf("%q is not a valid terraform_distribution, only %q and %q are supported", *distribution, "terraform", "opentofu")


Suggested change

return fmt.Errorf("%q is not a valid terraform_distribution, only %q and %q are supported", *distribution, "terraform", "opentofu")

return fmt.Errorf("'%s' is not a valid terraform_distribution, only '%s' and '%s' are supported", *distribution, "terraform", "opentofu")

%q produces a double quoted string which then needs to be escaped in the JSON log output. Single quoted strings are better as they don't have to be escaped.

X-Guardian · 2024-12-18T13:33:15Z

server/core/runtime/apply_step_runner.go

@@ -39,19 +41,27 @@ func (a *ApplyStepRunner) Run(ctx command.ProjectContext, extraArgs []string, pa

 	ctx.Log.Info("starting apply")
 	var out string
+	tfDistribution := a.DefaultTFDistribution
+	if ctx.TerraformDistribution != nil {


Is there a test that covers this condition? Same question for all runners.

X-Guardian · 2024-12-18T14:09:30Z

server/core/terraform/tfclient/terraform_client.go

@@ -14,7 +14,7 @@
 // Modified hereafter by contributors to runatlantis/atlantis.
 //
 // Package terraform handles the actual running of terraform commands.


Suggested change

// Package terraform handles the actual running of terraform commands.

// Package tfclient handles the actual running of terraform commands.

X-Guardian · 2024-12-18T14:09:53Z

server/core/terraform/tfclient/terraform_client.go

+	"github.com/runatlantis/atlantis/server/core/terraform"
+


Suggested change

"github.com/runatlantis/atlantis/server/core/terraform"

"github.com/runatlantis/atlantis/server/core/terraform"

Remove blank line

X-Guardian · 2024-12-18T14:13:14Z

server/user_config.go

+	// TFDistribution is deprecated in favor of DefaultTFDistribution
+	TFDistribution        string          `mapstructure:"tf-distribution"`


Suggested change

// TFDistribution is deprecated in favor of DefaultTFDistribution

TFDistribution string `mapstructure:"tf-distribution"`

TFDistribution string `mapstructure:"tf-distribution"` // deprecated in favor of DefaultTFDistribution

Reduce diff

X-Guardian · 2024-12-18T14:20:37Z

There are some test failures too: https://github.com/runatlantis/atlantis/actions/runs/12394308257/job/34599029347?pr=5167

abborg requested review from a team as code owners December 15, 2024 22:13

abborg requested review from chenrui333, lukemassa and nitrocode and removed request for a team December 15, 2024 22:13

github-actions bot added the go Pull requests that update Go code label Dec 15, 2024

dosubot bot added the feature New functionality/enhancement label Dec 15, 2024

abborg changed the title ~~Support project-level Terraform distribution selection~~ feat:Support project-level Terraform distribution selection Dec 15, 2024

abborg changed the title ~~feat:Support project-level Terraform distribution selection~~ feat: Support project-level Terraform distribution selection Dec 15, 2024

abborg force-pushed the main branch from 6718320 to 20e0cc3 Compare December 15, 2024 22:37

abborg and others added 2 commits December 15, 2024 17:41

Merge branch 'main' into main

4f9dd08

Merge branch 'main' into main

ac3a7f6

X-Guardian added the waiting-on-response Waiting for a response from the user label Dec 17, 2024

Update documentation for Terraform distribution

fc81e67

Signed-off-by: Andrew Borg <[email protected]>

github-actions bot added the docs Documentation label Dec 17, 2024

abborg and others added 3 commits December 17, 2024 13:14

Fix mis-named deprecation warning

ee74bb7

Signed-off-by: Andrew Borg <[email protected]>

Merge branch 'main' into main

13ab59e

Fix markdown formatting

5bbf855

X-Guardian requested changes Dec 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Support project-level Terraform distribution selection #5167

feat: Support project-level Terraform distribution selection #5167

abborg commented Dec 15, 2024

jamengual commented Dec 16, 2024

X-Guardian commented Dec 17, 2024

abborg commented Dec 17, 2024

X-Guardian left a comment

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian commented Dec 18, 2024

	return fmt.Errorf("%q is not a valid terraform_distribution, only %q and %q are supported", *distribution, "terraform", "opentofu")
	return fmt.Errorf("'%s' is not a valid terraform_distribution, only '%s' and '%s' are supported", *distribution, "terraform", "opentofu")

	// Package terraform handles the actual running of terraform commands.
	// Package tfclient handles the actual running of terraform commands.

	"github.com/runatlantis/atlantis/server/core/terraform"
	"github.com/runatlantis/atlantis/server/core/terraform"

		// TFDistribution is deprecated in favor of DefaultTFDistribution
		TFDistribution string `mapstructure:"tf-distribution"`

feat: Support project-level Terraform distribution selection #5167

Are you sure you want to change the base?

feat: Support project-level Terraform distribution selection #5167

Conversation

abborg commented Dec 15, 2024

what

why

tests

references

jamengual commented Dec 16, 2024

X-Guardian commented Dec 17, 2024

abborg commented Dec 17, 2024

X-Guardian left a comment

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian commented Dec 18, 2024