BUG: invalid cache use through cross-subfolder buildstring collision

[h-vetinari]

It could be argued that this issue should be in https://github.com/conda/conda, but on the other hand, the issue only appears in conda-build (and more specifically, when cross-compiling), so I'm opening it here for now. Don't mind if it's moved, or I can also open a parallel issue in conda, if desired.

The issue is that two artefacts with the same exact artefact name, but for different architectures end up clobbering each other in the package cache, which presumably does not take the architecture into account because the normal conda workflows are all on the same architecture.

However, when using conda-build to cross-compile between architectures, it's possible to hit this, and it leads to extremely hard to debug situations. Recently, we had such a case, where¹

     linux-64/binutils-impl_linux-aarch64-2.45.0-default_1234567_3
linux-aarch64/binutils-impl_linux-aarch64-2.45.0-default_1234567_3

were both in use during the build of pytorch, and the fact that they both end up in pkgs/binutils-impl_linux-aarch64-2.45.0-default_1234567_3 caused very bizarre errors. Luckily, @isuruf was able to diagnose the problem, because I had no idea what's going on (and I do consider myself relatively experienced in the idiosyncrasies of conda-build).

A similar error occurred in the past in conda-forge/numpy-feedstock#241 and was fixed by conda-forge/blas-feedstock#77; at the time I had even less of an idea of why that happens and was just happy once it got fixed. But IMO, this is a massive footgun that, when it triggers, is almost impossible to debug.

The "obvious" fix (to me as an outsider) would be to make the package cache aware of the conda subfolder of the respective packages. I guess it's currently also theoretically possible to coincide between noarch and yesarch packages in the same way, though very unlikely due to the way the hashes are computed? I.e. if there's a package/version/string combination where

linux-64/pkg-version-string
  noarch/pkg-version-string

collide exactly and both get used in the same build process for some reason, the build cache might similarly get confused.

Footnotes

1. the 1234567 hash was due https://github.com/conda-forge/binutils-feedstock/issues/104 ↩