"Resource type azure-native:dns:RecordSet not found" on TRE redeployment

[cptanalatriste]

✅ Checklist

• I have searched open and closed issues for duplicates.
• This is a problem observed when deploying a Data Safe Haven.
• I can reproduce this with the latest version.
• I have read through the documentation.
• This isn't an open-ended question (open a discussion if it is).

💻 System information

• Operating System: macOS
• Data Safe Haven version: v5.6.0

📦 Packages

List of packages

Paste list of packages here

🚫 Describe the problem

When trying to re-deploy a v5.6.0 TRE, it fails with the error: Pulumi error: error: Resource type azure-native:dns:RecordSet not found (see below for more details), after a failed deployment using DSH v5.5.1 (see below for more details).

The error is related to running refresh when updating pulumi-azure-native for version v2 to v3 (or viceversa, see: pulumi/pulumi-azure-native#4113 ). Running refresh does not run the Pulumi program again (see: https://www.pulumi.com/docs/iac/cli/commands/pulumi_refresh/ ) hence it fails due to unknown resource types. We can force refresh to run the program again using the --run-program flag (see: https://www.pulumi.com/blog/improved-refresh-destroy-experience/ ), but at the moment there's no way to accomplish it using the dsh cli (dsh pulumi run fails).

As a workaround, you can hardcode the --run-program option in the dsh source code and that does fix the problem (the ProjectManager.refresh() method in project_manager.py). Ideally, the dsh cli should warn the user that the dsh version of the CLI is different to the version of the TRE to deploy. And if that's the case, refresh should be run with the --run-program flag.

🌳 Log messages

Relevant log messages

@ refreshing....                                                                                                                     
    pulumi:pulumi:Stack data-safe-haven-shm-prod5-sre-newinsightscvdnet **failed** 1 error; 3 messages              
Diagnostics:                                                                                                                                
  pulumi:pulumi:Stack (data-safe-haven-shm-prod5-sre-newinsightscvdnet):                                                                       
    WARNING: All log messages before absl::InitializeLog() is called are written to STDERR                                                              
    I0000 00:00:1766144841.773646  688987 fork_posix.cc:77] Other threads are currently calling into gRPC, skipping fork() handlers                  
    I0000 00:00:1766144841.785518  688987 fork_posix.cc:77] Other threads are currently calling into gRPC, skipping fork() handlers                  
                                                                                                                                                           
    error: update failed                                                                                                                  
                                                                                                                                                           
  azure-native:dns:RecordSet (sre_networking_ns_record):                                                                                       
    error: Resource type azure-native:dns:RecordSet not found                                                                             
                                                                                                                                                           
Outputs:                                                                                                                                    
    allowlist_share_filenames: {                                                                                                                  
        cran: "cran.allowlist"                                                                                                           
        pypi: "pypi.allowlist"                                                                                                           
    }                                                                                                                                          
    allowlist_share_name     : "software-repositories-nexus-allowlists"                                                                  
    data                     : {                                                                                                               
        key_vault_name                         : "shmprod5srenewinssecrets"                                                              
        password_user_database_admin_secret    : "password-user-database-admin"                                                          
        storage_account_data_configuration_name: "shmprodsrenewiconfigdata"                                                              
    }                                                                                                                                          
    ldap                     : {                                                                                                               
        admin_group_name          : "Data Safe Haven SRE newinsightscvdnet Administrators"                                               
        privileged_user_group_name: "Data Safe Haven SRE newinsightscvdnet Privileged Users"                                             
        user_group_name           : "Data Safe Haven SRE newinsightscvdnet Users"                                                        
    }                                                                                                                                          
    remote_desktop           : {                                                                                                               
        connection_db_name       : "guacamole"                                                                                           
        connection_db_server_name: "shm-prod5-sre-newinsightscvdnet-db-server-guacamole"                                                 
        container_group_name     : "shm-prod5-sre-newinsightscvdnet-container-group-remote-desktop"                                      
        disable_copy             : true                                                                                                  
        disable_paste            : true                                                                                                  
        resource_group_name      : "shm-prod5-sre-newinsightscvdnet-rg"                                                                  
    }                                                                                                                                          
    sre_fqdn                 : "newinsightscvdnet.prod5.turingsafehaven.ac.uk"                                                           
    sre_resource_group       : "shm-prod5-sre-newinsightscvdnet-rg"                                                                      
    workspaces               : {                                                                                                               
        vm_outputs: [                                                                                                                          
            [0]: {                                                                                                                             
                ip_address: "10.0.2.4"                                                                                                   
                name      : "shm-prod5-sre-newinsightscvdnet-vm-workspace-01"                                                            
                sku       : "Standard_D8ds_v5"                                                                                           
            }                                                                                                                                  
        ]                                                                                                                                      
    }                                                                                                                                          
                                                                                                                                                        
Resources:                                                                                                                                  
    314 unchanged                                                                                                                                          
    2 errored                                                                                                                                   
                                                                                                                                                           
Duration: 2m9s                                                                                                                              
                                                                                                                                                           
Pulumi error:  ~  azure-native:dns:RecordSet sre_networking_ns_record refreshing (0s) error: Resource type       
azure-native:dns:RecordSet not found                                                                                                                    
Pulumi error:  ~  azure-native:dns:RecordSet sre_networking_ns_record **refreshing failed** error: Resource type       
azure-native:dns:RecordSet not found                                                                                                                    
Pulumi error:     pulumi:pulumi:Stack data-safe-haven-shm-prod5-sre-newinsightscvdnet running error: update failed               
Pulumi error:     error: update failed                                                                                                    
Pulumi error:     error: Resource type azure-native:dns:RecordSet not found                                                               
Pulumi error:  stderr:                                                                                                                                     
Pulumi refresh failed.                                                                                                                                     
Pulumi deployment failed.                                                                                                                                  
Could not deploy Secure Research Environment 'newinsight

♻️ To reproduce

1. Deploy a TRE using DSH v5.6.0.
2. Try to redeploy using DSH v5.5.1.
3. Deployment should fail.
4. Attempt to redeploy again using the correct DSH version (v5.6.0).
5. Deployment should fail, with an Resource type azure-native:dns:RecordSet not found error.