ansible provisioner fails with "failed to transfer file"

[hc-github-team-packer]

This issue was originally opened by @stefangweichinger in hashicorp/packer#11783 and has been migrated to this repository. The original issue description is below.

---

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

When filing a bug, please include the following headings if possible. Any
example text in this template can be deleted.

Overview of the Issue

I have a packer config using the virtualbox-iso builder for building Debian VMs.
This config uses the ansible provisioner and worked fine before.

Recently I see failed builds because the provisioning step fails:

2022/05/17 07:51:59 ui:     debian.virtualbox-iso.bullseye: failed to transfer file to /home/sgw/.ansible/tmp/ansible-local-45402qjbtzya8/tmpeyxtinre /home/vagrant/.ansible/tmp/ansible-tmp-1652766718.7643352-45407-8201740687254/AnsiballZ_setup.py:

Reproduction Steps

Run "packer build" with a packer build using the ansible provisioner.

Packer version

1.8.0

Simplified Packer Template

the part with the provisioner:

  provisioner "ansible" {
    playbook_file = "ansible/vagrant-debian-11-guest-additions.yml"
    user          = "${var.ssh_username}"
    ansible_env_vars = [
      "ANSIBLE_STDOUT_CALLBACK=debug",
      "ANSIBLE_HOST_KEY_CHECKING=False",
      "ANSIBLE_SSH_ARGS='-o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s'",
      "ANSIBLE_NOCOLOR=True"
      ]
    ansible_ssh_extra_args = [
      "-o HostKeyAlgorithms=+ssh-rsa -o PubkeyAcceptedKeyTypes=+ssh-rsa"
    ]
  }

These extra_args come from fixing an issue with ssh not able to log into the created VM.
If I comment these vars it fails as well, but with another issue around "host key type" (yes, I found the related issue, that's where my extra_vars come from).

Operating system and Environment details

Fedora 36 Workstation, VirtualBox 6.1.34

 ansible --version
ansible [core 2.12.3]
  config file = None
  configured module search path = ['/home/sgw/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/sgw/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/sgw/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/sgw/.local/bin/ansible
  python version = 3.10.4 (main, Mar 25 2022, 00:00:00) [GCC 12.0.1 20220308 (Red Hat 12.0.1-0)]
  jinja version = 3.0.3
  libyaml = True

[mtumalewicz]

After some testing with solution presented in: hashicorp/packer#11783 (comment)
I think that it might be painfull to manage additional logic at top of packerfiles to determine versions of openssh and provide scp legacy flag or not.

To handle it properly use ssh_transfer_method = smart or ssh_transfer_method = piped in your ansible config - ansible docs.

[onitake]

This seems to be same issue as #100 .

hashicorp/packer#11783 (comment) mentioned another workaround: Add extra_arguments = [ "--scp-extra-args", "'-O'" ] to the provisioner.

ssh_transfer_method = smart should look like this, if you put it into your ansible.cfg:

[ssh_connection]
transfer_method = smart

This will work, but generate lots of warnings.

[hswong3i]

My ansible.cfg working with Ubuntu 22.10 + openssh-server 9.0p1:

[ssh_connection]
control_path = %(directory)s/%%h-%%p-%%r
pipelining = true
scp_extra_args = -O
scp_if_ssh = true
ssh_args = -o ControlMaster=auto -o ControlPersist=600s
transfer_method = smart

[defaults]
ansible_managed = Ansible managed: Do NOT edit this file manually!
display_failed_stderr = true
forks = 20
host_key_checking = false
interpreter_python = auto
nocows = 1
remote_tmp = $HOME/.ansible/tmp
retry_files_enabled = false

[samgrewal]

Also seeing this issue with packer 1.8.4 against RHEL 8.6 + OpenSSH 8.0p1

The specific error message:
Failed to connect to the host via scp: bash: /usr/lib/sftp-server: No such file or directory\nscp: Connection closed\r\n

Minimum change to resolve was adding extra_arguments = [ "--scp-extra-args", "'-O'" ]