curl: command not found when using docker image >= 2.54.0 in an ARM deploymentScript w/ supportingScriptUris

[Yunzlez]

Describe the bug

When using "kind": "AzureCLI" inside of a Microsoft.Resources/deploymentScripts inside an ARM deployment, the script fails when one or more supportingScriptUris are set.

This happens because when supportingScriptUris is set, the entrypoint script (/mnt/azscripts/azscriptinput/DeploymentScript.sh, provided by the deployment) attempts to retrieve these additional resources using cURL.

cURL was removed in 2.54.0, in this commit, to remove build dependencies from the docker image.

As there seems to be no way to change the default entrypoint script in a Microsoft.Resources/deploymentScripts, this is now broken for any AZ CLI version >= 2.54.0

Related command

docker run -it mcr.microsoft.com/azure-cli:2.61.0 /bin/bash -c 'curl'

Errors

Uri: https://amastorageprodus.blob.core.windows.net/applicationdefinitions/<snip>/artifacts/install.sh failed to download. Detail: /mnt/azscripts/azscriptinput/DeploymentScript.sh: line 93: curl: command not found

Issue script & Debug output

Sample Microsoft.Resources/deploymentScripts definition:

    {
      "type": "Microsoft.Resources/deploymentScripts",
      "apiVersion": "2023-08-01",
      "name": "test-script",
      "location": "[parameters('location')]",
      "kind": "AzureCLI",
      "properties": {
        "azCliVersion": "2.59.0",
        "supportingScriptUris": [
          "[uri(deployment().properties.templateLink.uri, 'artifacts/install.sh')]"
        ],
        "scriptContent": "/bin/bash ./install.sh"
      }
    }

Expected behavior

Using supportingScriptUris inside of a Microsoft.Resources/deploymentScripts ARM resource pulls the external artifacts.

Environment Summary

docker image mcr.microsoft.com/azure-cli with tags >= 2.54.0

Additional context

The only workaround I've found is to fall back to 2.53.0.

[yonzhan]

Thank you for opening this issue, we will look into it.

[bebound]

Why does /mnt/azscripts/azscriptinput/DeploymentScript.sh assume that the curl is installed?

[Yunzlez]

I don't know. To be clear: this is not a script we wrote. This script was created by Microsoft and is automatically injected and ran during the ARM deployment. As far as I can tell, there is no way for us to change this behaviour.

[XavierGeerinck]

Any updates on this?

[bebound]

@zhoxing-ms for help

[zhoxing-ms]

This happens because when supportingScriptUris is set, the entrypoint script (/mnt/azscripts/azscriptinput/DeploymentScript.sh, provided by the deployment) attempts to retrieve these additional resources using cURL.

@jorgecotillo Could you please help explain does supportingScriptUris assume that the curl is installed? If so, what is the reason?

[Yunzlez]

Appreciate you looking into this - is there any update on this issue currently?

[anthony-c-martin]

This happens because when supportingScriptUris is set, the entrypoint script (/mnt/azscripts/azscriptinput/DeploymentScript.sh, provided by the deployment) attempts to retrieve these additional resources using cURL.

@jorgecotillo Could you please help explain does supportingScriptUris assume that the curl is installed? If so, what is the reason?

@zhoxing-ms it is used to fetch user scripts to be executed inside the container. Please see https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-script-template if you'd like more information about the product. If you would like more specifics (e.g. a link to the script itself), please reach out to me internally - my alias is antmarti.

We've been able to work around this by switching from curl to wget in our initialization script. DeploymentScripts is a heavily utilized resource on Azure; please keep us in mind if you need to make any further destructive changes to the image.

[bebound]

@anthony-c-martin curl and wget are not required in CLI docker image. We don't explicitly install them in the dockerfile. Please make sure these packages are installed before using them in your script.

PS: These packages are provided by the base OS of our image, but it may change in the future. For example, python:3.11-alpine includes only wget, while azurelinux3 has curl.

[XavierGeerinck]

@anthony-c-martin curl and wget are not required in CLI docker image. We don't explicitly install them in the dockerfile. Please make sure these packages are installed before using them in your script.

PS: These packages are provided by the base OS of our image, but it may change in the future. For example, python:3.11-alpine includes only wget, while azurelinux3 has curl.

It might make sense to provide those by default though. They are both small packages and are heavily utilized, outweighing the disadvantages if we had to manually provide them

[anthony-c-martin]

It might make sense to provide those by default though. They are both small packages and are heavily utilized, outweighing the disadvantages if we had to manually provide them

+1

Please make sure these packages are installed before using them in your script.

Thanks for clarifying; sounds like we may need to move to publishing our own container image - installing at runtime is going to be too slow and unreliable for use in deploymentScripts.

[bebound]

We adhere to Docker best practices:

Don't install unnecessary packages

If the official image doesn't meet your needs, you should build your own image on top of it.

PS: I'm not saying we won't add more Linux tools in the image, but creating an all-encompassing development environment is not the goal of CLI image.

[XavierGeerinck]

We adhere to Docker best practices:

Don't install unnecessary packages

If the official image doesn't meet your needs, you should build your own image on top of it.

PS: I'm not saying we won't add more Linux tools in the image, but creating an all-encompassing development environment is not the goal of CLI image.

This is a DX question, not a best practices question.

"Does Microsoft want to provide a great DX, or do they want devs to struggle"

In the question above, currently it's the "struggle" part.

We are with 3 devs here already saying the same "It's a HELL" with no guidance. Creating own containers creates and brings a hassle that organizations do not want to deal with. Especially for something such as curl or wget

[anthony-c-martin]

We adhere to Docker best practices:

Don't install unnecessary packages

If the official image doesn't meet your needs, you should build your own image on top of it.

@bebound bear in mind that this container image is used directly for https://github.com/marketplace/actions/azure-cli-action. The Docker guidance on "publish your own container image" does not really apply for this use case, because it's not trivial to swap out images - your users would have to completely replace their usage of this action.

PS: I'm not saying we won't add more Linux tools in the image, but creating an all-encompassing development environment is not the goal of CLI image.

I think you're taking this guidance to the extreme. wget & curl aren't just part of an "all-encompassing development environment" - they're very frequently used for scripting in CI environments.