vfio_assigned_device: add device-BAR peer-to-peer DMA

[jstarks]

An assigned PCI device that wants to DMA into another assigned device's BAR (peer-to-peer) cannot do so through the existing host-VA mapping path: MMIO pages generally cannot be pinned by virtual address, so the mapping silently fails and the DMA faults in the IOMMU. The kernel's vfio-dmabuf feature solves this by exporting a device BAR region as a dmabuf whose physical MMIO can be programmed into an IOAS via IOMMU_IOAS_MAP_FILE, routing the mapping through the PCI P2PDMA provider instead of pinning host VA.

Wire this into the iommufd/cdev assignment path. Each mmappable BAR area is exported as a dmabuf at device setup and registered in a per-IOAS registry keyed by the area's intrinsic identity (the VFIO cdev inode plus its BAR-region file offset), which is stable across BAR moves and MMIO enable/disable. When a device programs peer DMA to that BAR, the DMA target looks the area up in the registry and maps it by file rather than by host VA. Devices sharing one iommufd land in the same IOAS and thus the same registry, so peer BARs become reachable; the legacy type1 path has no registry and falls back to best-effort VA mapping.

Ordering of the guest's Command-register MMIO enable is made explicit so the physical enable lands before the IOAS import (a revoked dmabuf makes the map-by-file import return ENODEV), and sub-page mmappable regions are aligned down so they stay trap-and-emulate instead of tripping the page-aligned MemoryRange invariant.

To exercise the path end to end, the aarch64 QEMU-TCG incubator gains a generic multi-device VFIO story: profiles can declare arbitrary devices (each advertising a capability derived from its name), and a new profile pairs QEMU's edu device (a register-programmed DMA initiator) with an ivshmem-plain device (a RAM-backed peer BAR target). A new driverless P2P test drives edu's DMA engine over pipette to round-trip a value through the ivshmem BAR, which only succeeds if the dmabuf import engaged. The incubator's L1 host kernel is switched to a build that ships the required P2PDMA / vfio-dmabuf / iommufd config.

[github-actions]

⚠️ Unsafe Code Detected

This PR modifies files containing unsafe Rust code. Extra scrutiny is required during review.

For more on why we check whole files, instead of just diffs, check out the Rustonomicon

[Copilot]

Pull request overview

This PR extends the VFIO assigned-device iommufd/cdev path to support peer-to-peer DMA into another assigned device’s BAR by exporting mmappable BAR areas as VFIO dmabufs and mapping them into the shared IOAS via IOMMU_IOAS_MAP_FILE. It also updates the aarch64 QEMU-TCG incubator to provision multiple VFIO-bound devices and adds an end-to-end P2P DMA integration test.

Changes:

• Add dmabuf export + per-IOAS dmabuf registry plumbing to enable BAR-target P2P DMA in the iommufd/cdev assignment path.
• Make VFIO region mmappability handling more robust (page-align down sub-page “mmap” regions; explicit MMIO enable/disable ordering).
• Enhance the aarch64 TCG incubator to support arbitrary VFIO-bound devices and add a driverless edu→ivshmem P2P DMA test; switch incubator host kernel to one with required configs.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
vmm_tests/vmm_tests/tests/tests/aarch64_exclusive.rs	Adds a new aarch64 TCG incubator test that validates BAR P2P DMA via edu↔ivshmem using busybox devmem.
vmm_tests/vmm_test_macros/src/lib.rs	Updates macro docs to reflect the new incubator capability name (test_disk).
vm/devices/user_driver/vfio_sys/src/lib.rs	Adds VFIO dmabuf feature ioctl support, dmabuf export helper APIs, and aligns sub-page mmappable regions down to page size.
vm/devices/pci/vfio_assigned_device/src/resolver.rs	Adjusts resolver call site for updated VfioAssignedPciDevice::new signature.
vm/devices/pci/vfio_assigned_device/src/manager.rs	Adds a per-IOAS dmabuf registry and teaches the iommufd DMA target to map device BARs by-file when a dmabuf is registered.
vm/devices/pci/vfio_assigned_device/src/lib.rs	Exports BAR dmabufs at setup (iommufd/cdev path), registers them, and enforces MMIO enable/disable ordering around IOMMU mapping.
vm/devices/pci/vfio_assigned_device/Cargo.toml	Adds parking_lot dependency for registry synchronization.
petri/petri_artifacts_common/src/lib.rs	Updates known capability list for incubator-provisioned VFIO devices used by tests.
petri/incubator/src/qemu.rs	Adds root-port slot assignment, supports edu and ivshmem-plain devices, and generalizes VFIO binding across extra devices.
petri/incubator/src/profile.rs	Introduces new incubator device config types (edu, ivshmem-plain) and derives capabilities from device names.
petri/incubator/profiles/aarch64-tcg-pcie.toml	Extends the aarch64 TCG PCIe profile with an edu+ivshmem P2P DMA device pair.
flowey/flowey_lib_hvlite/src/resolve_openvmm_test_linux_kernel.rs	Adds an aarch64-only test kernel variant intended for incubator VFIO/P2P coverage and enforces arch availability.
flowey/flowey_lib_hvlite/src/_jobs/local_build_and_run_nextest_vmm_tests.rs	Switches incubator kernel selection to the new incubator-specific kernel version.
flowey/flowey_lib_hvlite/src/_jobs/consume_and_test_nextest_vmm_tests_archive.rs	Switches incubator kernel selection to the new incubator-specific kernel version.
Cargo.lock	Adds parking_lot to the resolved dependency graph.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

[github-actions]

2 Petri tests failed (2 unstable)