New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v3.114.0 FileArchive Issue: file not found when using a relative path of more than 2 levels above. Lambda zip: no such file or directory. #16092
Comments
Just been replicating this issue for someone over on Slack, and can confirm this is a v3.114.0 regression. I suspect this PR is related: #15607 It seems the engine runs in the cwd of where pulumi is executed from, disregarding the value specified in Minimal replication for # Pulumi.yaml
name: pulumi-chdir
runtime:
name: python
options:
virtualenv: venv
main: stack/
description: A minimal Python Pulumi program # stack/__main__.py
import pulumi
import pulumi_aws
import os
stub_path = "../stub"
print(os.getcwd()) ## reports in directory of this file for both v3.113.3 and v3.114.0
asset = pulumi.FileArchive(stub_path)
# create a bucket, and upload the asset
bucket = pulumi_aws.s3.Bucket("my-bucket")
bucket_object = pulumi_aws.s3.BucketObject(
"my-bucket-object",
bucket=bucket.id,
source=asset, # engine reading this seems to be from the cwd of the command, instead of the project path specified in the Pulumi.yaml
# make it private
acl="private",
) # stub/stub.py
# just a stub file |
Yup can confirm this is asset/archive hash calculations. I think it's a pretty easy fix of threading the current directory down to the right places. |
…ues (#16100) <!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes #16092. Two main changes in this PR. Firstly we add a test for assets/archives to the conformance tests, and had to fix up some small issues in the test runner to enable this (copying test data files, correcting project directory layout, handling multi-word package names). The other change is to fix the engine to respect the programs working directory when serialising/deserialising assets and archives. It does this by passing the programs working directory as a MarshalOption (n.b this option is only really needed in the engine, program SDK doesn't really need it because it doesn't change directories). ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
Still facing the same problem with 3.115.0 pulumi about:
pip show, as the pulumi sdk version doesn't seem to get detected:
Presumably this will be fixed once providers are updated to build against 3.115.0? |
Can you post the error message you're getting? |
|
I just tested with FileAsset, and this is fixed in 3.115.0 (broken in 3.114). So it's just FileArchive that still has the working directory missing. |
Thanks I'll check that. |
Yup can confirm, we fixed file archive to handle files but the path lookup is wrong for folders. I'll add that to the tests and get it fixed, should be able to get a release out early next week with this fixed. |
<!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes #16092. Last part of the fix started in #16100. That last PR fixed assets and archive files, but not archive folders. ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [x] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
confirmed working with v3.115.1, thanks |
What happened?
Our CI process automatically pulls the latest version of pulumi. Seems like in the last update, we experience an issue where the lambda code reference is not finding our zip file. We have several lambda services across a few micro services, but only two of them reference a zip file that is 2 levels above the current directory. These two lambda services are failing. After troubleshooting, and ruling out any code changes, I found that reverting to the previous version works fine (3.113.3).
Example
Pulumi Error:
We have other lambda apps that use:
or
without issue.
When reverting back to pulumi version 3.113.3, the above works as expected.
To reproduce:
pulumi preview
-> should haveno such file or directory
error.Output of
pulumi about
CLI
Version 3.114.0
Go Version go1.22.2
Go Compiler gc
Plugins
KIND NAME VERSION
language nodejs unknown
Host
OS ubuntu
Version 20.04
Arch x86_64
Additional context
We will fix the version to v3.113.3 for now. What's the best approach to making sure our builds don't break to do these very frequent new versions? We need to ensure better consistency and we may have to keep versions static and update periodically. What is the standard practise others are using?
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
The text was updated successfully, but these errors were encountered: