Unable to use collections alongside the playbook

[zeridon]

Overview of the Issue

We follow the practice of locating collections and roles alongside the playbooks for more compact delivery/CI purposes. Currently ansible-remote provisioner does not honor neither ansible.cfg nor ANSIBLE_COLLECTIONS_PATHS environment variable

Reproduction Steps

[defaults]
display_skipped_hosts = false
host_key_checking = False
nocolor = True
# locate everything bellow the dir for comletenes
roles_path = roles
collections_paths = collections

Install necessary collections/roles
Run the packer build

Neither relative nor absolute paths in ansible.cfg work.

Plugin and Packer version

Packer v1.8.0

Simplified Packer Buildfile

packer {
  required_version = ">= 1.8.0"
  required_plugins {
    vsphere = {
      version = ">= v1.0.3"
      source  = "github.com/hashicorp/vsphere"
    }
    ansible = {
      version = ">= 1.0.2"
      source  = "github.com/hashicorp/ansible"
    }
  }
}

source "vsphere-iso" "linux-ubuntu" {
...
  // Communicator Settings and Credentials
  communicator       = "ssh"
  ssh_username       = var.build_username
  ssh_password       = var.build_password
  ssh_port           = var.communicator_port
  ssh_timeout        = var.communicator_timeout
}

build {
  sources = ["source.vsphere-iso.linux-ubuntu"]

  provisioner "ansible" {
    playbook_file          = "${path.cwd}/ansible/main.yml"
    roles_path             = "${path.cwd}/ansible/roles"
    collections_path       = "${path.cwd}/ansible/collections"
    galaxy_file            = "${path.cwd}/ansible/requirements.yml"
    galaxy_force_install   = true
    galaxy_force_with_deps = true
    ansible_env_vars = [
      "ANSIBLE_CONFIG=${path.cwd}/ansible/ansible.cfg",
      "ANSIBLE_COLLECTIONS_PATHS=${path.cwd}/ansible/collections",
    ]
    extra_arguments = [
      "-v",
      "--extra-vars", "display_skipped_hosts=false",
      "--extra-vars", "BUILD_USERNAME=${var.build_username}",
      "--extra-vars", "BUILD_SECRET='${var.build_key}'",
    ]
  }
}

Operating system and Environment details

Ubuntu 20.04

Log Fragments and crash.log files

==> vsphere-iso.linux-ubuntu: Connected to SSH!
2022/05/26 09:57:15 packer-plugin-vsphere_v1.0.4_x5.0_linux_amd64 plugin: 2022/05/26 09:57:15 Running the provision hook
2022/05/26 09:57:15 [INFO] (telemetry) Starting provisioner <sensitive>
2022/05/26 09:57:15 packer-plugin-<sensitive>_v1.0.2_x5.0_linux_amd64 plugin: 2022/05/26 09:57:15 <sensitive>-playbook version: 2.12.6
==> vsphere-iso.linux-ubuntu: Provisioning with Ansible...
    vsphere-iso.linux-ubuntu: Setting up proxy adapter for Ansible....
2022/05/26 09:57:15 packer-plugin-<sensitive>_v1.0.2_x5.0_linux_amd64 plugin: 2022/05/26 09:57:15 Creating inventory file for Ansible run...
2022/05/26 09:57:15 packer-plugin-<sensitive>_v1.0.2_x5.0_linux_amd64 plugin: 2022/05/26 09:57:15 SSH proxy: serving on 127.0.0.1:46145
    vsphere-iso.linux-ubuntu: Executing Ansible Galaxy
    vsphere-iso.linux-ubuntu: Starting galaxy role install process

    vsphere-iso.linux-ubuntu: Executing Ansible Galaxy
    vsphere-iso.linux-ubuntu: Starting galaxy collection install process
    vsphere-iso.linux-ubuntu: [WARNING]: The specified collections path
    vsphere-iso.linux-ubuntu: '/home/zeridon/work/infrastructure/template-maker/<sensitive>/collections' is not
    vsphere-iso.linux-ubuntu: part of the configured Ansible collections paths
    vsphere-iso.linux-ubuntu: Process install dependency map

==> vsphere-iso.linux-ubuntu: Executing Ansible: <sensitive>-playbook -e packer_build_name="linux-ubuntu" -e packer_<sensitive>_type=vsphere-iso -e packer_http_addr=192.168.166.170:0 --ssh-extra-args '-o IdentitiesOnly=yes' -v --extra-vars display_skipped_hosts=false --extra-vars BUILD_USERNAME=<sensitive> --extra-vars BUILD_SECRET=***** --extra-vars ANSIBLE_USERNAME=<sensitive> --extra-vars ANSIBLE_SECRET=***** -e <sensitive>_ssh_private_key_file=/tmp/<sensitive>-key3069387022 -i /tmp/packer-provisioner-<sensitive>3139017408 /home/zeridon/work/infrastructure/template-maker/<sensitive>/main.yml

    vsphere-iso.linux-ubuntu: ERROR! couldn't resolve module/action 'community.general.parted'. This often indicates a misspelling, missing collection, or incorrect module path.

ls -l ansible/collections/ansible_collections/community/general/plugins/modules/system/parted.py 
-rw-r--r-- 1 zeridon zeridon 25730 май 26 09:57 ansible/collections/ansible_collections/community/general/plugins/modules/system/parted.py

[esolitos]

I have encountered a similar issue, to my understanding this was introduced in c64d013