Resource properties named id are ignored

[leinyl]

What happened?

Regardless of the id format, and even though it matches the regular expression on the error, the resource creation always returns the following error.

*error: azure-native:apimanagement:PolicyFragment resource has a problem: 'id' does not match expression '(^[\w]+$)|(^[\w][\w\-]+[\w]$)'*

The error showed initially using v1.104.0 of the Pulumi.AzureNative library. Upgrading the library to v2.3.0 did not resolve the error.

Expected Behavior

The PolicyFragment resource is created.

Steps to reproduce

Create a PolicyFragment resource using C# as explained in the documentation here:
https://www.pulumi.com/registry/packages/azure-native/api-docs/apimanagement/policyfragment/#azure-native-apimanagement-policyfragment

Not even the provided sample passes the id validation:

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using AzureNative = Pulumi.AzureNative;

return await Deployment.RunAsync(() => 
{
    var policyFragment = new AzureNative.ApiManagement.PolicyFragment("policyFragment", new()
    {
        Description = "A policy fragment example",
        Format = "xml",
        Id = "policyFragment1",
        ResourceGroupName = "rg1",
        ServiceName = "apimService1",
        Value = "<fragment><json-to-xml apply=\"always\" consider-accept-header=\"false\" /></fragment>",
    });

});

Output of pulumi about

CLI
Version 3.78.1
Go Version go1.20.7
Go Compiler gc

Plugins
NAME VERSION
dotnet unknown

Host
OS Microsoft Windows 10 Enterprise
Version 10.0.19044 Build 19044
Arch x86_64

This project is written in dotnet: executable='C:\Program Files\dotnet\dotnet.exe' version='7.0.304'

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[danielrbradley]

I'm currently unable to reproduce this issue. I've recreated your program and it completes pulumi preview without error and fails at the point of actually creating the Fragment due to the service not actually existing (which is expected).

Here's a few next steps you could try to narrow down the issue:

1. Do you see the failure during a preview?
2. You could also try enabling verbose logging to see if there's any modification happening to the value of the id.
3. You also don't actually need to define the id yourself as this will be taken from the resource's identifier (policyFragment in your program) by default.

[leinyl]

Hi Bradley,

Thanks for your reply, yes I was still running the preview. I tried the verbose earlier and it was still showing only the error without much context.

Your third comment though, made me realize that it may not be the Azure resource id, but the id in the stack, which was the case.

We're standardizing our resources' names with <standardized-resource-type>:<resource-name>:<instance-name>. For instance in this case it would be apim-policy-fragment:grm-sp-backend-routing:apim-dev, which is what's failing for the policy fragment type.

The same naming convention works fine for all the other resources, like the policy where I'm referencing the fragment: apim-policy:grm-api-policy:apim-dev.

As a workaround, I think I could change the naming format for the policy fragment to not contain colon symbols while the resource doesn't support them, and then use an alias later once the naming format is accepted.

[danielrbradley]

Thanks for the speedy reply @leinyl

I'm not quite sure if I follow what exactly you mean by "the id in the stack".

Do you still consider this a use-case which should work correctly in Azure and therefore needs addressing in the provider? If so could you provide a repro which demonstrates the id in the stack being incorrect. Thanks!

[leinyl]

I'm referring to the name property in the resource. I originally thought it was the Id in the arguments as the other resources accept the naming format we have been using, and the error message was pointing an id property.

            _ = new PolicyFragment(
                $"apim-policy-fragment:grm-sp-backend-routing:{ApimServiceName}",
                new PolicyFragmentArgs
                {
                    Id = "grm-sp-backend-routing",
                    Description = description,
                    Format = PolicyFragmentContentFormat.Rawxml,
                    ResourceGroupName = ApimResourceGroup,
                    ServiceName = ApimServiceName,
                    Value = ReadFileWithoutLineEndings(finalPolicyFragment)
                },
                MergeWithDefaultResourceOptions());

I changed it to only dashes for the time being $"grm-sp-backend-routing-{ApimServiceName}" , but it would be good to have it consistent with the rest of the resources in the stack at some point.

[leinyl]

Well the issue seems to be that the resource is ignoring the Id property in PolicyFragmentArgs, it's just using the name parameter in the PolicyFragment regardless of the Id value in the arguments.
For specific reasons, we're deploying to multiple APIMs at this time, and we're maintaining the APIM instance name in the resource name in the stack. While I can bypass the validation issue using a name that matches the regular expression, it will make harder to reference the Id within other polices as the name will change per APIM instance.
I cannot use the same name either as it will detect the duplicate and the run will fail.

[danielrbradley]

Ok, I can now reproduce the error with the following program:

using Pulumi;
using Pulumi.AzureNative.ApiManagement;

return await Deployment.RunAsync(() =>
{
  _ = new PolicyFragment(
      $"apim-policy-fragment:grm-sp-backend-routing:ApimServiceName",
      new PolicyFragmentArgs
      {
        Id = "grm-sp-backend-routing",
        Description = "A policy fragment example",
        Format = PolicyFragmentContentFormat.Rawxml,
        ResourceGroupName = "rg1",
        ServiceName = "apimService1",
        Value = "<fragment><json-to-xml apply=\"always\" consider-accept-header=\"false\" /></fragment>",
      });
});

From the verbose log I can see that the id argument is missing:

registered default provider for package azure-native-2.4.0: urn:pulumi:dev::sdf::pulumi:providers:azure-native::default_2_4_0
Unmarshaling property for RPC[ResourceMonitor.RegisterResource(azure-native:apimanagement:PolicyFragment,apim-policy-fragment:grm-sp-backend-routing:ApimServiceName)]: description={A policy fragment example}
Unmarshaling property for RPC[ResourceMonitor.RegisterResource(azure-native:apimanagement:PolicyFragment,apim-policy-fragment:grm-sp-backend-routing:ApimServiceName)]: format={rawxml}
Unmarshaling property for RPC[ResourceMonitor.RegisterResource(azure-native:apimanagement:PolicyFragment,apim-policy-fragment:grm-sp-backend-routing:ApimServiceName)]: resourceGroupName={rg1}
Unmarshaling property for RPC[ResourceMonitor.RegisterResource(azure-native:apimanagement:PolicyFragment,apim-policy-fragment:grm-sp-backend-routing:ApimServiceName)]: serviceName={apimService1}
Unmarshaling property for RPC[ResourceMonitor.RegisterResource(azure-native:apimanagement:PolicyFragment,apim-policy-fragment:grm-sp-backend-routing:ApimServiceName)]: value={<fragment><json-to-xml apply="always" consider-accept-header="false" /></fragment>}
ResourceMonitor.RegisterResource received: t=azure-native:apimanagement:PolicyFragment, name=apim-policy-fragment:grm-sp-backend-routing:ApimServiceName, custom=true, #props=5, parent=urn:pulumi:dev::sdf::pulumi:pulumi:Stack::sdf-dev, protect=false, provider=urn:pulumi:dev::sdf::pulumi:providers:azure-native::default_2_4_0::17191e02-9014-4b8d-8574-fdb2e840ba44, deps=[], deleteBeforeReplace=<nil>, ignoreChanges=[], aliases=[{  azure-native:apimanagement/v20211201preview:PolicyFragment    false} {  azure-native:apimanagement/v20220401preview:PolicyFragment    false} {  azure-native:apimanagement/v20220801:PolicyFragment    false} {  azure-native:apimanagement/v20220901preview:PolicyFragment    false} {  azure-native:apimanagement/v20230301preview:PolicyFragment    false}], customTimeouts=, providers=map[], replaceOnChanges=[], retainOnDelete=false, deletedWith=

This is due to this line in the Pulumi Dotnet SDK:

            return SerializeFilteredPropertiesAsync(
                label, args,
                key => key != Constants.IdPropertyName && key != Constants.UrnPropertyName,
                keepResources, keepOutputValues: keepOutputValues);

I don't think the exclusion of entries named id and urn is correct here as the dictionary is only formed of the reflected ResourceArgs object. Here's the call site within PrepareResourceAsync:

            var dictionary = await args.ToDictionaryAsync().ConfigureAwait(false);
            var (serializedProps, propertyToDirectDependencies) =
                await SerializeResourcePropertiesAsync(
                        label,
                        dictionary,
                        await this.MonitorSupportsResourceReferences().ConfigureAwait(false),
                        keepOutputValues: remote && await MonitorSupportsOutputValues().ConfigureAwait(false)).ConfigureAwait(false);

I'm going to transfer this over to the dotnet project for them to check my analysis and put in their planning.

[Zaid-Ajaj]

Opened an issue on pulumi/pulumi pulumi/pulumi#13781 to discuss the implications of this since it is not only a dotnet SDK problem