CopyToRemote not resolving paths

[GuardMoony]

What happened?

When using the ~ as a reference remote path the function doesn't resolve the destination. This is inconsistent with the Command function, which does resolve the path.
This causes the function to try and place files under a folder called ~ instead.
The same also happens if you try to use environment variables like $HOME.

In case of the example:
The Command creates the directory "/home/user/destination"
The CopyToRemote tries to copy to "/home/user/~/destination/" and causing an error.
error: failed to create remote directory ~/destination/: file does not exist

Example

const archive = new pulumi.asset.FileArchive(path.join(__dirname, "./resources/"));
const dest = "~/destination/";
const verify = new cmd.remote.Command(
      `verify_dest`, 
      {
        connection: connection,
        create: pulumi.interpolate`mkdir -p ${dest}`,
        delete: pulumi.interpolate`rm -rf ${dest}`
      }
);
const copy = new cmd.remote.CopyToRemote(
      `copy_file`, 
      {
        connection: connection,
        source: archive,
        remotePath: dest
      },
      { dependsOn: [verify_dest]}
);

Output of pulumi about

CLI
Version      3.163.0
Go Version   go1.24.2
Go Compiler  gc

Plugins
KIND      NAME        VERSION
resource  command     1.0.2
resource  kubernetes  4.22.2
language  nodejs      3.163.0

Host
OS       ubuntu
Version  24.04
Arch     x86_64

This project is written in nodejs: executable='/usr/bin/node' version='v18.19.1'

Current Stack: 

TYPE                      URN
pulumi:providers:command  urn:pulumi:local::Neos2-IaC::pulumi:providers:command::default_1_0_2
command:remote:Command    urn:pulumi:local::Neos2-IaC::command:remote:Command::verify_dest_10.103.8.35


Found no pending operations associated with local

Backend
Name         XXXXXX
URL            file://~
User           XXXXXX
Organizations
Token type     personal

Dependencies:
NAME                VERSION
typescript          5.8.3
@pulumi/command     1.0.2
@pulumi/kubernetes  4.22.2
@pulumi/pulumi      3.162.0
@types/node         18.19.86
@types/nunjucks     3.2.6
nunjucks            3.2.4

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[mjeffryes]

Hi @GuardMoony; I can understand why you might expect that expansion to work, but it's actually pretty tricky to make it work right.

In the case of the Command resource, you're passing in a command that is being interpreted by the shell on the remote host, but CopyToRemote is directly asking the remote ssh server to write the file, so there's no remote shell involved to expand the '~' to the remote user directory. It might help to think of it like you're running scp (which has the same limitation).

[GuardMoony]

@mjeffryes thank you for the response. But looking at the man page https://man7.org/linux/man-pages/man1/scp.1.html

-O Use the legacy SCP protocol for file transfers instead of
the SFTP protocol. Forcing the use of the SCP protocol
may be necessary for servers that do not implement SFTP,
for backwards-compatibility for particular filename
wildcard patterns and for expanding paths with a ‘~’
prefix for older SFTP servers.

This should mean SCP is able to use home paths. And from experience i used it in old backup scripts.

[mjeffryes]

That's interesting; I wonder what's different about the "legacy SCP protocol for file transfers" that allows path expansion. Perhaps it actually runs a shell on the target?

I suspect the analogy doesn't really translate to the go libraries we're using, but it's an interesting datapoint.