New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Copy hidden directories using CopyTerraformFolderToTemp #1021
Comments
This has come up a few times. We should probably add a new function called something like |
See also #963. |
Hi! First of all thank you very much for this great package! I've been hitting this one, and it'd be great to add this feature. Can I work on it? |
Currently -
CopyTerraformFolderToTemp
allows us to copy around terraform modules to temp locations - which is particularly nice for running a bunch of parallel tests, however - it doesn't copy hidden directories such as.terraform
which prevents me from trying to cache both my plugins and modules prior to running my tests.Something I've run into lately when running some tests in parallel, when running
InitAndApply
is hitting infrequent - but enough to be annoying transient network errors for fetching the various modules used that are basicallyFailed to download module
type errors.I ended up hitting a wall with causing the errors to go away since it's out of my immediate control, my first approach to solving this issue was to run a single
terraform init
prior to running my tests to populate.terraform/plugins
and.terraform/modules
prior to running the tests and just use those - reducing my number of Inits, and also improving the efficiency of the pipeline slightly.I ended up finding out however that
CopyTerraformFolderToTemp
doesn't end up copying the.terraform
directory (and seems to exclude all hidden directories here). While ultimately I'm able to useRetryableTerraformErrors
to just retry the step to get around the problem for my for specific use-case, I still wanted to make the suggestion to potentially allowCopyTerraformFolderToTemp
to copy hidden directories (or just.terraform
) to the temp directories.While Terraform's native functionality does offer the ability to have a separate directory for the plugin-cache by setting a
plugin_cache_dir
env var - there (currently) exists no matching functionality to be able to manage a separatemodules-cache
type directory - so all downloaded modules are expected to be in.terraform/modules
(see hashicorp/terraform#16268).I think a solution built into
CopyTerraformFolderToTemp
, or a separate function to allow this would be more desirable than a hacky workaround where we copy modules around manually for those who want to make use of a module cache while still being able to useCopyTerraformFolderToTemp
I don't mind (attempting to) implement something, but wanted to put the idea out there to collect any sort of feedback prior to making any sort of attempts.
Thanks!
The text was updated successfully, but these errors were encountered: