Skip to content

Terraform does not use AWS Task Roles to fetch modules from s3 #20439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
repl-david-winiarski opened this issue Feb 22, 2019 · 9 comments · Fixed by #26762
Closed

Terraform does not use AWS Task Roles to fetch modules from s3 #20439

repl-david-winiarski opened this issue Feb 22, 2019 · 9 comments · Fixed by #26762

Comments

@repl-david-winiarski
Copy link

Terraform Version

Terraform v0.11.11

Terraform Configuration Files

module "s3-module" {
  source                   = "BUCKET.s3-us-west-2.amazonaws.com/1/terraform/2/module.zip"
 somekey                 = "somevalue"
}

Debug Output


2019/02/21 20:58:46 [INFO] Terraform version: 0.11.11 ac4fff416318bf0915a0ab80e062a99ef3724334
2019/02/21 20:58:46 [INFO] Go runtime version: go1.11.1
2019/02/21 20:58:46 [INFO] CLI args: []string{"/bin/terraform", "init", "-no-color"}
2019/02/21 20:58:46 [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2019/02/21 20:58:46 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/02/21 20:58:46 [INFO] CLI command args: []string{"init", "-no-color"}
2019/02/21 20:58:46 [DEBUG] command: loading backend config file: /app
Initializing modules...
2019/02/21 20:58:46 [TRACE] module source: "./local-module"
2019/02/21 20:58:46 [TRACE] detected module source "file:///app/local-module"
2019/02/21 20:58:46 [DEBUG] fetching module from file:///app/local-module
2019/02/21 20:58:46 [DEBUG] fetching "file:///app/local-module" with key "1.local_module;./local-module"
- module.local-module
 Getting source "./local-module"
2019/02/21 20:58:46 [DEBUG] found "file:///app/local-module" in ".terraform/modules/7d7d4b57b2fd167574633e7a7d85c9b5": true
2019/02/21 20:58:46 [TRACE] "file:///app/local-module" stored in ".terraform/modules/7d7d4b57b2fd167574633e7a7d85c9b5"
- module.s3-module
2019/02/21 20:58:46 [TRACE] module source: "bucket.s3-us-west-2.amazonaws.com/1/terraform/2/s3-module.zip"
2019/02/21 20:58:46 [TRACE] detected module source "s3::https://s3-us-west-2.amazonaws.com/bucket/1/terraform/2/s3-module.zip"
2019/02/21 20:58:46 [DEBUG] fetching module from s3::https://s3-us-west-2.amazonaws.com/bucket/1/terraform/2/s3-module.zip
2019/02/21 20:58:46 [DEBUG] fetching "s3::https://s3-us-west-2.amazonaws.com/bucket/1/terraform/2/s3-module.zip" with key "1.s3_module;bucket.s3-us-west-2.amazonaws.com/1/terraform/2/s3-module.zip"
 Getting source "bucket.s3-us-west-2.amazonaws.com/1/terraform2/s3-module.zip"
Error downloading modules: Error loading modules: NoCredentialProviders: no valid providers in chain. Deprecated.
2019/02/21 20:59:07 [DEBUG] plugin: waiting for all plugin processes to complete...
 For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Crash Output

Expected Behavior

Terraform should of been able to fetch the module from s3.

Actual Behavior

Terraform init failed to fetch the module and exited with error: Error downloading modules: Error loading modules: NoCredentialProviders: no valid providers in chain. Deprecated.

Steps to Reproduce

Attempt to pull a s3 module with proper IAM permissions to access the s3 location but via a ECS / Codebuild task role. It is specifically accessing it via the Task Role that causes this issue. Using the EC2 role works and passing in he Access ID, Key and Session token work as well.

Additional Context

Terraform is being executed in a codebuild stage using a task role. It can access our remote state in s3 using the same task role with no issues. The only issue is with fetching a module from s3.

References

@repl-david-winiarski repl-david-winiarski changed the title Terraform does not use AWS Task Roles to fetch modules Terraform does not use AWS Task Roles to fetch modules from s3 Feb 22, 2019
@hashibot hashibot added config and removed modules labels Aug 15, 2019
@tbondarchuk
Copy link

Looks like the terraform's code responsible for downloading modules from s3 could work only with aws_access_key_id/aws_secret_access_key/aws_secret_access_key set explicitly or grabbed from EC2 metadata endpoint, but not with assumed role or credentials from EcsContainer metadata endpoint.

Tested by configuring custom profiles in .aws/config and running terraform as AWS_PROFILE=test terraform init.

@solarmosaic-kflorence
Copy link

The root issue is here: hashicorp/go-getter#152 in the go-getter library

@solarmosaic-kflorence
Copy link

Hm, it seems like hashicorp/go-getter#152 has been resolved by hashicorp/go-getter#218 which made it into this release https://github.com/hashicorp/go-getter/releases/tag/v1.4.2 -- however, I just tested this against terraform 0.13.5 and the problem remains. Terraform seems to be using some variant of 1.4.2: https://github.com/hashicorp/terraform/blob/master/go.mod#L57 -- but I can't tell if it includes this change or not.

@solarmosaic-kflorence
Copy link

solarmosaic-kflorence commented Oct 30, 2020

My guess is that the version used by terraform is some pre-release version of 1.4.2 which does not include the change. Any chance we can update to 1.4.2 stable, or even 1.5.0? @alisdair @jbardin seems like you have made several updates to dependencies lately, perhaps you can provide some guidance.

@alisdair
Copy link
Contributor

@solarmosaic-kflorence Here's a PR upgrading go-getter: #26762. Are you able to build that branch locally and test if it solves the issue?

If not, can you provide detailed reproduction steps so that I can verify this issue is fixed? I'm not familiar with the use of AWS Task Roles.

@solarmosaic-kflorence
Copy link

@alisdair oh cool, thanks. I have not built terraform locally before but will try to do so later today if time permits.

This can be easily reproduced by running terraform within AWS CodeBuild which includes a module with an S3 bucket as the source: https://www.terraform.io/docs/modules/sources.html#s3-bucket -- on my case it is the result of executing terratest.

The result currently is a failure that looks like this:

TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66: Downloading s3::https://s3.amazonaws.com/path/to/module.zip for module...
--
119 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66:
120 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66: Error: Failed to download module
121 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66:
122 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66: Could not download module "module" (../../../main.tf:142) source code from
123 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66: "s3::https://s3.amazonaws.com/path/to/module.zip":
124 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66: NoCredentialProviders: no valid providers in chain. Deprecated.
125 | TestUnit_PlanWithDefaults 2020-10-30T17:53:22Z logger.go:66:    For verbose messaging see aws.Config.CredentialsChainVerboseErrors

@alisdair
Copy link
Contributor

alisdair commented Nov 4, 2020

@solarmosaic-kflorence I looked into setting up CodeBuild or even just ECS to test this, but was not able to get anything up and running in a reasonable amount of time. The upstream changes really ought to fix it, so I'm marking this closed, but it would be good to get your confirmation if you're able to test the next 0.14 beta when it's released. Thanks!

@solarmosaic-kflorence
Copy link

Awesome, really appreciate it @alisdair -- happy to test it when the beta is out.

@ghost
Copy link

ghost commented Dec 5, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants