Allow version to be set dynamically #510

Zaid-Ajaj · 2023-10-10T14:07:51Z

Fixes #508

This PR changes the definition of Version from a static *ast.StringExpr to ast.Expr so that it can a value which can be evaluated at runtime when running the program rather than just static strings. Anywhere else where want to know the version and we don't have an evaluator yet, we check if the version is a static string and get its value, otherwise we skip it.

Zaid-Ajaj · 2023-10-10T14:10:02Z

pkg/pulumiyaml/run.go

- ctx.addErrDiag(node.Key.Syntax().Syntax().Range(),
- "Version conflicts with the default provider version",
- fmt.Sprintf("Try removing this option on resource \"%s\"", k))
+ stringValue := func(expr ast.Expr) string {


I feel like this function maybe needs access to an evaluator such that it can evaluate the version if it is set dynamically but I am not sure how we go about it or whether it is 100% necessary 🤔 cc @iwahbe @AaronFriel thoughts?

This function as used on L254 will definitely need an evaluator.

Updated the setDefaultProviders function to take an evaluator and actually get the value from the visited resources ✅

Frassle · 2023-10-10T15:28:00Z

pkg/pulumiyaml/run_options_test.go

 resources:
 provider-a:
 type: pulumi:providers:test
+ version: ${version}
 provider-b:


Shouldn't something later in this test assert that provider-a is actually registered with version "1.0.0"?

Yeah you are right, will add it to the test

Added an assertion that the registered provider has the version evaluated correctly ✅

Frassle · 2023-10-10T15:31:48Z

pkg/pulumiyaml/semver.go

- if v == nil || v.Value == "" {
- return nil, nil
- }
+func ParseVersion(expr ast.Expr) (*semver.Version, error) {


Is this even used anymore? If we're allowing version to be any expression it probably isn't safe to have a function laying around that looks like it correctly gives a *semver.Version from an Expr.

This is used in a few places where there isn't an evaluator available to evaluate the expression, so it tries to get the value of the version when it is statically known (i.e. *ast.StringExpr).

it probably isn't safe to have a function laying around that looks like it correctly gives a *semver.Version from an Expr

The function isn't total, it will try to return a *semver.Version when it can from a string expression, otherwise it will return nil which the rest of the caller code handles already (when resolving resource/invoke versions)

Maybe adding a bool to the returned tuple could make the signature a bit more clear. Returning false when the Expr is anything but *ast.StringExpr

Are we allowing version to be any expression, or just a non-literal? Specifically, are we allowing version to be a outputty value?

I would strongly recommend that version must be knowable at preview time.

Syntactically it can be any expression but the evaluation should return string at runtime. We check that when we run evaluateExpr:

if versionExpr, ok := e.evaluateExpr(t.CallOpts.Version); ok { if version, ok := versionExpr.(string); ok { opts = append(opts, pulumi.Version(version)) } else { e.error(t.CallOpts.Version, "version must be a string value") } }

Let's make sure that there is an issue to update the documentation on this change then.

pkg/pulumiyaml/analyser.go

iwahbe · 2023-10-10T18:32:01Z

pkg/pulumiyaml/analyser.go

+ var version *semver.Version
+ if v.Options.Version != nil {
+ switch versionValue := v.Options.Version.(type) {
+ case *ast.StringExpr:
+ parsedVersion, err := ParseVersion(versionValue)
+ if err != nil {
+ ctx.error(v.Type, fmt.Sprintf("unable to parse resource %v provider version: %v", k, err))
+ return true
+ }
+ version = parsedVersion


I'm nervous about this for two reasons:

We don't actually check that v.Options.Version is typed as a string, so it might evaluate to some other type later. Have you tried running pulumi up against this branch and passing in a value of another type here?

Correct version information is necessary to type check correctly. If we fail to resolve the correct version we could reject programs that should work or vice versa.

Updated such that the type checker computes the type of the Version and makes sure it is assignable to schema.StringType. Then at runtime when we evaluate Version, if it has outputs, we return an error.

iwahbe · 2023-10-10T18:34:18Z

pkg/pulumiyaml/run.go

- ctx.addErrDiag(node.Key.Syntax().Syntax().Range(),
- "Version conflicts with the default provider version",
- fmt.Sprintf("Try removing this option on resource \"%s\"", k))
+ stringValue := func(expr ast.Expr) string {


This function as used on L254 will definitely need an evaluator.

iwahbe · 2023-10-10T19:48:34Z

pkg/pulumiyaml/semver.go

- if v == nil || v.Value == "" {
- return nil, nil
- }
+func ParseVersion(expr ast.Expr) (*semver.Version, error) {


Are we allowing version to be any expression, or just a non-literal? Specifically, are we allowing version to be a outputty value?

I would strongly recommend that version must be knowable at preview time.

iwahbe · 2023-10-10T19:49:56Z

pkg/pulumiyaml/run.go

+ } else {
+ e.error(t.CallOpts.Version, "version must be a string value")
+ }


This should be an internal error. It version resolves to something not a string, then the type checker should have caught it.

Yeah added a type-check for Version such that it must be assignable to string, so removed this one ✅

iwahbe · 2023-10-10T19:50:30Z

pkg/pulumiyaml/run.go

+ overallOk = false
+ }
+ } else {
+ e.error(v.Options.Version, "version must be not be an output")


I don't remember if the type checker can catch this kind of thing, but if it can that would be great.

Frassle

I think we need a test of what happens if you try and set the expression to something that resolves to unknown at preview time. But otherwise this looks ok.

Zaid-Ajaj requested a review from a team October 10, 2023 14:07

Zaid-Ajaj commented Oct 10, 2023

View reviewed changes

Frassle reviewed Oct 10, 2023

View reviewed changes

iwahbe reviewed Oct 10, 2023

View reviewed changes

Allow version to be set dynamically.

bdbb737

Zaid-Ajaj force-pushed the zaid/allow-version-resource-option-to-be-dynamic branch from 88eb37a to bdbb737 Compare October 11, 2023 14:46

Zaid-Ajaj requested review from iwahbe and Frassle October 11, 2023 14:46

Frassle approved these changes Nov 2, 2023

View reviewed changes

Frassle mentioned this pull request Nov 14, 2023

Quoted variable values incorrectly converted to integers #453

Closed

jamest-pin mentioned this pull request Nov 15, 2023

Allow protect to be set dynamically #525

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow version to be set dynamically #510

Allow version to be set dynamically #510

Zaid-Ajaj commented Oct 10, 2023

Zaid-Ajaj Oct 10, 2023

iwahbe Oct 10, 2023

Zaid-Ajaj Oct 11, 2023

Frassle Oct 10, 2023

Zaid-Ajaj Oct 10, 2023

Zaid-Ajaj Oct 11, 2023

Frassle Oct 10, 2023

Zaid-Ajaj Oct 10, 2023

iwahbe Oct 10, 2023

Zaid-Ajaj Oct 10, 2023

iwahbe Oct 10, 2023 •

edited

iwahbe Oct 10, 2023

Zaid-Ajaj Oct 11, 2023

iwahbe Oct 10, 2023

iwahbe Oct 10, 2023

iwahbe Oct 10, 2023

Zaid-Ajaj Oct 11, 2023

iwahbe Oct 10, 2023

Frassle left a comment

Allow version to be set dynamically #510

Are you sure you want to change the base?

Allow version to be set dynamically #510

Conversation

Zaid-Ajaj commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iwahbe Oct 10, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frassle left a comment

Choose a reason for hiding this comment

iwahbe Oct 10, 2023 •

edited