Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.conda packages uploaded to nvidia channel not mirroring to CDN #945

Closed
jakirkham opened this issue May 22, 2024 · 29 comments
Closed

.conda packages uploaded to nvidia channel not mirroring to CDN #945

jakirkham opened this issue May 22, 2024 · 29 comments
Labels
severity::2 critical; broken functionality with an unacceptably complex workaround source::partner created by or for an Anaconda, Inc. partner company

Comments

@jakirkham
Copy link
Member

jakirkham commented May 22, 2024

Recently we uploaded .conda packages for CUDA 12.5 to the nvidia channel. However we discovered they were not mirroring to CDN.

We noticed this by trying to install CUDA 12.5 from the nvidia channel, but got the following error:

$ conda create -c nvidia -n cuda12.5 cuda=12.5
Channels:
 - nvidia
 - defaults
Platform: linux-aarch64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - cuda=12.5*

Current channels:

  - https://conda.anaconda.org/nvidia
  - defaults

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

However I can see the package is available via conda search

$ conda search 'nvidia/label/main::cuda[subdir=linux-aarch64]=12.5' 
Loading channels: done
# Name                       Version           Build  Channel             
cuda                          12.5.0               0  nvidia/label/main 

Also via Anaconda.org

Screenshot 2024-05-22 at 1 54 58 PM

Since these were uploaded, we were recommended to upload .tar.bz2 packages to the nvidia channel, which the CDN appears to handle better

Still it would be good to support .conda packages on the nvidia channel CDN too. This issue remains open to address that

@jakirkham
Copy link
Member Author

An interesting wrinkle is dropping the /label/main bit or use the -c channel syntax with conda search, it returns no packages found. For example

$ conda search --skip-flexible-search -c nvidia 'cuda[subdir=linux-aarch64]=12.5' 
Loading channels: done

PackagesNotFoundError: The following packages are not available from current channels:

  - cuda=12.5[subdir=linux-aarch64]

Current channels:

  - https://conda.anaconda.org/nvidia/linux-aarch64
  - https://conda.anaconda.org/nvidia/noarch
  - https://repo.anaconda.com/pkgs/main/linux-aarch64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-aarch64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Note: Flexible search is skipped otherwise we pick up cuda-python, which is not what we are looking for here

@jakirkham
Copy link
Member Author

cc @adibbley (for vis)

@jaimergp
Copy link
Contributor

Maybe adding label/main bypasses the CDN, where it is missing?

@jakirkham
Copy link
Member Author

That is what I'm thinking as well

For example, the same conda create command above using nvidia/label/main works

conda create -c nvidia/label/main -n cuda12.5 cuda=12.5 --dry-run
Channels:
 - nvidia/label/main
 - defaults
Platform: linux-aarch64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/conda/envs/cuda12.5

  added / updated specs:
    - cuda=12.5


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    _libgcc_mutex-0.1          |             main           2 KB  defaults
    _openmp_mutex-5.1          |           51_gnu         1.4 MB  defaults
    binutils_impl_linux-aarch64-2.38|       h0c9fd12_1         5.5 MB  defaults
    binutils_linux-aarch64-2.38.0|       h0c7b1b4_0          24 KB  defaults
    bzip2-1.0.8                |       h998d150_6         210 KB  defaults
    ca-certificates-2024.3.11  |       hd43f75c_0         127 KB  defaults
    cuda-12.5.0                |                0          17 KB  nvidia/label/main
    cuda-cccl-12.5.39          |                0          16 KB  nvidia/label/main
    cuda-cccl_linux-aarch64-12.5.39|                0         1.3 MB  nvidia/label/main
    cuda-command-line-tools-12.5.0|                0          17 KB  nvidia/label/main
    cuda-compiler-12.5.0       |                0          17 KB  nvidia/label/main
    cuda-crt-dev_linux-aarch64-12.5.40|                0          79 KB  nvidia/label/main
    cuda-crt-tools-12.5.40     |                0          19 KB  nvidia/label/main
    cuda-cudart-12.5.39        |                0          17 KB  nvidia/label/main
    cuda-cudart-dev-12.5.39    |                0          17 KB  nvidia/label/main
    cuda-cudart-dev_linux-aarch64-12.5.39|                0         352 KB  nvidia/label/main
    cuda-cudart-static-12.5.39 |                0          17 KB  nvidia/label/main
    cuda-cudart-static_linux-aarch64-12.5.39|                0         715 KB  nvidia/label/main
    cuda-cudart_linux-aarch64-12.5.39|                0         191 KB  nvidia/label/main
    cuda-cuobjdump-12.5.39     |                0         242 KB  nvidia/label/main
    cuda-cupti-12.5.39         |                0         1.6 MB  nvidia/label/main
    cuda-cupti-dev-12.5.39     |                0         6.4 MB  nvidia/label/main
    cuda-cuxxfilt-12.5.39      |                0         202 KB  nvidia/label/main
    cuda-driver-dev-12.5.39    |                0          17 KB  nvidia/label/main
    cuda-driver-dev_linux-aarch64-12.5.39|                0          32 KB  nvidia/label/main
    cuda-gdb-12.5.39           |                0         5.1 MB  nvidia/label/main
    cuda-libraries-12.5.0      |                0          17 KB  nvidia/label/main
    cuda-libraries-dev-12.5.0  |                0          17 KB  nvidia/label/main
    cuda-nvcc-12.5.40          |                0          17 KB  nvidia/label/main
    cuda-nvcc-dev_linux-aarch64-12.5.40|                0        10.8 MB  nvidia/label/main
    cuda-nvcc-impl-12.5.40     |                0          18 KB  nvidia/label/main
    cuda-nvcc-tools-12.5.40    |                0        19.2 MB  nvidia/label/main
    cuda-nvcc_linux-aarch64-12.5.40|                0          18 KB  nvidia/label/main
    cuda-nvdisasm-12.5.39      |                0        47.6 MB  nvidia/label/main
    cuda-nvml-dev-12.5.39      |                0         158 KB  nvidia/label/main
    cuda-nvprune-12.5.39       |                0          56 KB  nvidia/label/main
    cuda-nvrtc-12.5.40         |                0        17.8 MB  nvidia/label/main
    cuda-nvrtc-dev-12.5.40     |                0          27 KB  nvidia/label/main
    cuda-nvtx-12.5.39          |                0          28 KB  nvidia/label/main
    cuda-nvvm-dev_linux-aarch64-12.5.40|                0          17 KB  nvidia/label/main
    cuda-nvvm-impl-12.5.40     |                0         8.4 MB  nvidia/label/main
    cuda-nvvm-tools-12.5.40    |                0        10.8 MB  nvidia/label/main
    cuda-profiler-api-12.5.39  |                0          19 KB  nvidia/label/main
    cuda-runtime-12.5.0        |                0          17 KB  nvidia/label/main
    cuda-sanitizer-api-12.5.39 |                0         9.8 MB  nvidia/label/main
    cuda-toolkit-12.5.0        |                0          17 KB  nvidia/label/main
    cuda-tools-12.5.0          |                0          17 KB  nvidia/label/main
    cuda-version-12.5          |                3          17 KB  nvidia/label/main
    cuda-visual-tools-12.5.0   |                0          17 KB  nvidia/label/main
    dbus-1.13.18               |       h821dc26_0         544 KB  defaults
    expat-2.6.2                |       h419075a_0         159 KB  defaults
    fontconfig-2.14.1          |       h652894d_2         323 KB  defaults
    freetype-2.12.1            |       h6df46f4_0         632 KB  defaults
    gcc_impl_linux-aarch64-11.2.0|       h1234567_1        21.0 MB  defaults
    gcc_linux-aarch64-11.2.0   |       h3fc98b3_0          25 KB  defaults
    gds-tools-1.10.0.4         |                0        37.8 MB  nvidia/label/main
    glib-2.78.4                |       h419075a_0         517 KB  defaults
    glib-tools-2.78.4          |       h419075a_0         126 KB  defaults
    gmp-6.2.1                  |       h22f4aa5_3         515 KB  defaults
    gxx_impl_linux-aarch64-11.2.0|       h1234567_1        10.2 MB  defaults
    gxx_linux-aarch64-11.2.0   |       h0c7b1b4_0          24 KB  defaults
    icu-73.1                   |       h419075a_0        26.2 MB  defaults
    kernel-headers_linux-aarch64-4.14.214|      h1cb168f_14        1018 KB  defaults
    ld_impl_linux-aarch64-2.38 |       h8131f2d_1         778 KB  defaults
    libcublas-12.5.2.13        |                0       240.2 MB  nvidia/label/main
    libcublas-dev-12.5.2.13    |                0          71 KB  nvidia/label/main
    libcufft-11.2.3.18         |                0       148.6 MB  nvidia/label/main
    libcufft-dev-11.2.3.18     |                0          28 KB  nvidia/label/main
    libcufile-1.10.0.4         |                0         843 KB  nvidia/label/main
    libcufile-dev-1.10.0.4     |                0          29 KB  nvidia/label/main
    libcurand-10.3.6.39        |                0        39.7 MB  nvidia/label/main
    libcurand-dev-10.3.6.39    |                0         245 KB  nvidia/label/main
    libcusolver-11.6.2.40      |                0        79.0 MB  nvidia/label/main
    libcusolver-dev-11.6.2.40  |                0          56 KB  nvidia/label/main
    libcusparse-12.4.1.24      |                0       113.8 MB  nvidia/label/main
    libcusparse-dev-12.4.1.24  |                0          41 KB  nvidia/label/main
    libffi-3.4.4               |       h419075a_1         140 KB  defaults
    libgcc-devel_linux-aarch64-11.2.0|       h1234567_1         311 KB  defaults
    libgcc-ng-11.2.0           |       h1234567_1         1.3 MB  defaults
    libglib-2.78.4             |       hd439bcf_0         1.6 MB  defaults
    libgomp-11.2.0             |       h1234567_1         466 KB  defaults
    libiconv-1.16              |       h998d150_3         772 KB  defaults
    libnpp-12.3.0.116          |                0        94.5 MB  nvidia/label/main
    libnpp-dev-12.3.0.116      |                0         460 KB  nvidia/label/main
    libnvfatbin-12.5.39        |                0         737 KB  nvidia/label/main
    libnvfatbin-dev-12.5.39    |                0          21 KB  nvidia/label/main
    libnvjitlink-12.5.40       |                0        15.6 MB  nvidia/label/main
    libnvjitlink-dev-12.5.40   |                0          20 KB  nvidia/label/main
    libnvjpeg-12.3.2.38        |                0         2.3 MB  nvidia/label/main
    libnvjpeg-dev-12.3.2.38    |                0          27 KB  nvidia/label/main
    libpng-1.6.39              |       h998d150_0         320 KB  defaults
    libstdcxx-devel_linux-aarch64-11.2.0|       h1234567_1         2.7 MB  defaults
    libstdcxx-ng-11.2.0        |       h1234567_1         779 KB  defaults
    libuuid-1.41.5             |       h998d150_0          29 KB  defaults
    libxcb-1.15                |       h2f4d8fa_0         484 KB  defaults
    libxkbcommon-1.0.1         |       h998d150_1         601 KB  defaults
    libxml2-2.10.4             |       h6097fa9_2         808 KB  defaults
    ncurses-6.4                |       h419075a_0         1.1 MB  defaults
    nsight-compute-2024.2.0.16 |                2       227.6 MB  nvidia/label/main
    nspr-4.35                  |       h419075a_0         251 KB  defaults
    nss-3.89.1                 |       h419075a_0         2.1 MB  defaults
    openssl-3.0.13             |       h2f4d8fa_2         5.3 MB  defaults
    pcre2-10.42                |       hcfaa891_1         1.3 MB  defaults
    pip-24.0                   |  py312hd43f75c_0         3.3 MB  defaults
    python-3.12.3              |       h8edadfe_1        14.2 MB  defaults
    readline-8.2               |       h998d150_0         381 KB  defaults
    setuptools-69.5.1          |  py312hd43f75c_0         1.3 MB  defaults
    sqlite-3.45.3              |       h998d150_0         1.5 MB  defaults
    sysroot_linux-aarch64-2.26 |      h4744778_14        11.3 MB  defaults
    tk-8.6.14                  |       h987d8db_0         3.5 MB  defaults
    tzdata-2024a               |       h04d1e81_0         116 KB  defaults
    wheel-0.43.0               |  py312hd43f75c_0         142 KB  defaults
    xz-5.4.6                   |       h998d150_1         662 KB  defaults
    zlib-1.2.13                |       h998d150_1         113 KB  defaults
    ------------------------------------------------------------
                                           Total:        1.24 GB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-aarch64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-aarch64::_openmp_mutex-5.1-51_gnu 
  binutils_impl_lin~ pkgs/main/linux-aarch64::binutils_impl_linux-aarch64-2.38-h0c9fd12_1 
  binutils_linux-aa~ pkgs/main/linux-aarch64::binutils_linux-aarch64-2.38.0-h0c7b1b4_0 
  bzip2              pkgs/main/linux-aarch64::bzip2-1.0.8-h998d150_6 
  ca-certificates    pkgs/main/linux-aarch64::ca-certificates-2024.3.11-hd43f75c_0 
  cuda               nvidia/label/main/linux-aarch64::cuda-12.5.0-0 
  cuda-cccl          nvidia/label/main/linux-aarch64::cuda-cccl-12.5.39-0 
  cuda-cccl_linux-a~ nvidia/label/main/noarch::cuda-cccl_linux-aarch64-12.5.39-0 
  cuda-command-line~ nvidia/label/main/linux-aarch64::cuda-command-line-tools-12.5.0-0 
  cuda-compiler      nvidia/label/main/linux-aarch64::cuda-compiler-12.5.0-0 
  cuda-crt-dev_linu~ nvidia/label/main/noarch::cuda-crt-dev_linux-aarch64-12.5.40-0 
  cuda-crt-tools     nvidia/label/main/linux-aarch64::cuda-crt-tools-12.5.40-0 
  cuda-cudart        nvidia/label/main/linux-aarch64::cuda-cudart-12.5.39-0 
  cuda-cudart-dev    nvidia/label/main/linux-aarch64::cuda-cudart-dev-12.5.39-0 
  cuda-cudart-dev_l~ nvidia/label/main/noarch::cuda-cudart-dev_linux-aarch64-12.5.39-0 
  cuda-cudart-static nvidia/label/main/linux-aarch64::cuda-cudart-static-12.5.39-0 
  cuda-cudart-stati~ nvidia/label/main/noarch::cuda-cudart-static_linux-aarch64-12.5.39-0 
  cuda-cudart_linux~ nvidia/label/main/noarch::cuda-cudart_linux-aarch64-12.5.39-0 
  cuda-cuobjdump     nvidia/label/main/linux-aarch64::cuda-cuobjdump-12.5.39-0 
  cuda-cupti         nvidia/label/main/linux-aarch64::cuda-cupti-12.5.39-0 
  cuda-cupti-dev     nvidia/label/main/linux-aarch64::cuda-cupti-dev-12.5.39-0 
  cuda-cuxxfilt      nvidia/label/main/linux-aarch64::cuda-cuxxfilt-12.5.39-0 
  cuda-driver-dev    nvidia/label/main/linux-aarch64::cuda-driver-dev-12.5.39-0 
  cuda-driver-dev_l~ nvidia/label/main/noarch::cuda-driver-dev_linux-aarch64-12.5.39-0 
  cuda-gdb           nvidia/label/main/linux-aarch64::cuda-gdb-12.5.39-0 
  cuda-libraries     nvidia/label/main/linux-aarch64::cuda-libraries-12.5.0-0 
  cuda-libraries-dev nvidia/label/main/linux-aarch64::cuda-libraries-dev-12.5.0-0 
  cuda-nvcc          nvidia/label/main/linux-aarch64::cuda-nvcc-12.5.40-0 
  cuda-nvcc-dev_lin~ nvidia/label/main/noarch::cuda-nvcc-dev_linux-aarch64-12.5.40-0 
  cuda-nvcc-impl     nvidia/label/main/linux-aarch64::cuda-nvcc-impl-12.5.40-0 
  cuda-nvcc-tools    nvidia/label/main/linux-aarch64::cuda-nvcc-tools-12.5.40-0 
  cuda-nvcc_linux-a~ nvidia/label/main/linux-aarch64::cuda-nvcc_linux-aarch64-12.5.40-0 
  cuda-nvdisasm      nvidia/label/main/linux-aarch64::cuda-nvdisasm-12.5.39-0 
  cuda-nvml-dev      nvidia/label/main/linux-aarch64::cuda-nvml-dev-12.5.39-0 
  cuda-nvprune       nvidia/label/main/linux-aarch64::cuda-nvprune-12.5.39-0 
  cuda-nvrtc         nvidia/label/main/linux-aarch64::cuda-nvrtc-12.5.40-0 
  cuda-nvrtc-dev     nvidia/label/main/linux-aarch64::cuda-nvrtc-dev-12.5.40-0 
  cuda-nvtx          nvidia/label/main/linux-aarch64::cuda-nvtx-12.5.39-0 
  cuda-nvvm-dev_lin~ nvidia/label/main/noarch::cuda-nvvm-dev_linux-aarch64-12.5.40-0 
  cuda-nvvm-impl     nvidia/label/main/linux-aarch64::cuda-nvvm-impl-12.5.40-0 
  cuda-nvvm-tools    nvidia/label/main/linux-aarch64::cuda-nvvm-tools-12.5.40-0 
  cuda-profiler-api  nvidia/label/main/linux-aarch64::cuda-profiler-api-12.5.39-0 
  cuda-runtime       nvidia/label/main/linux-aarch64::cuda-runtime-12.5.0-0 
  cuda-sanitizer-api nvidia/label/main/linux-aarch64::cuda-sanitizer-api-12.5.39-0 
  cuda-toolkit       nvidia/label/main/linux-aarch64::cuda-toolkit-12.5.0-0 
  cuda-tools         nvidia/label/main/linux-aarch64::cuda-tools-12.5.0-0 
  cuda-version       nvidia/label/main/noarch::cuda-version-12.5-3 
  cuda-visual-tools  nvidia/label/main/linux-aarch64::cuda-visual-tools-12.5.0-0 
  dbus               pkgs/main/linux-aarch64::dbus-1.13.18-h821dc26_0 
  expat              pkgs/main/linux-aarch64::expat-2.6.2-h419075a_0 
  fontconfig         pkgs/main/linux-aarch64::fontconfig-2.14.1-h652894d_2 
  freetype           pkgs/main/linux-aarch64::freetype-2.12.1-h6df46f4_0 
  gcc_impl_linux-aa~ pkgs/main/linux-aarch64::gcc_impl_linux-aarch64-11.2.0-h1234567_1 
  gcc_linux-aarch64  pkgs/main/linux-aarch64::gcc_linux-aarch64-11.2.0-h3fc98b3_0 
  gds-tools          nvidia/label/main/linux-aarch64::gds-tools-1.10.0.4-0 
  glib               pkgs/main/linux-aarch64::glib-2.78.4-h419075a_0 
  glib-tools         pkgs/main/linux-aarch64::glib-tools-2.78.4-h419075a_0 
  gmp                pkgs/main/linux-aarch64::gmp-6.2.1-h22f4aa5_3 
  gxx_impl_linux-aa~ pkgs/main/linux-aarch64::gxx_impl_linux-aarch64-11.2.0-h1234567_1 
  gxx_linux-aarch64  pkgs/main/linux-aarch64::gxx_linux-aarch64-11.2.0-h0c7b1b4_0 
  icu                pkgs/main/linux-aarch64::icu-73.1-h419075a_0 
  kernel-headers_li~ pkgs/main/noarch::kernel-headers_linux-aarch64-4.14.214-h1cb168f_14 
  ld_impl_linux-aar~ pkgs/main/linux-aarch64::ld_impl_linux-aarch64-2.38-h8131f2d_1 
  libcublas          nvidia/label/main/linux-aarch64::libcublas-12.5.2.13-0 
  libcublas-dev      nvidia/label/main/linux-aarch64::libcublas-dev-12.5.2.13-0 
  libcufft           nvidia/label/main/linux-aarch64::libcufft-11.2.3.18-0 
  libcufft-dev       nvidia/label/main/linux-aarch64::libcufft-dev-11.2.3.18-0 
  libcufile          nvidia/label/main/linux-aarch64::libcufile-1.10.0.4-0 
  libcufile-dev      nvidia/label/main/linux-aarch64::libcufile-dev-1.10.0.4-0 
  libcurand          nvidia/label/main/linux-aarch64::libcurand-10.3.6.39-0 
  libcurand-dev      nvidia/label/main/linux-aarch64::libcurand-dev-10.3.6.39-0 
  libcusolver        nvidia/label/main/linux-aarch64::libcusolver-11.6.2.40-0 
  libcusolver-dev    nvidia/label/main/linux-aarch64::libcusolver-dev-11.6.2.40-0 
  libcusparse        nvidia/label/main/linux-aarch64::libcusparse-12.4.1.24-0 
  libcusparse-dev    nvidia/label/main/linux-aarch64::libcusparse-dev-12.4.1.24-0 
  libffi             pkgs/main/linux-aarch64::libffi-3.4.4-h419075a_1 
  libgcc-devel_linu~ pkgs/main/linux-aarch64::libgcc-devel_linux-aarch64-11.2.0-h1234567_1 
  libgcc-ng          pkgs/main/linux-aarch64::libgcc-ng-11.2.0-h1234567_1 
  libglib            pkgs/main/linux-aarch64::libglib-2.78.4-hd439bcf_0 
  libgomp            pkgs/main/linux-aarch64::libgomp-11.2.0-h1234567_1 
  libiconv           pkgs/main/linux-aarch64::libiconv-1.16-h998d150_3 
  libnpp             nvidia/label/main/linux-aarch64::libnpp-12.3.0.116-0 
  libnpp-dev         nvidia/label/main/linux-aarch64::libnpp-dev-12.3.0.116-0 
  libnvfatbin        nvidia/label/main/linux-aarch64::libnvfatbin-12.5.39-0 
  libnvfatbin-dev    nvidia/label/main/linux-aarch64::libnvfatbin-dev-12.5.39-0 
  libnvjitlink       nvidia/label/main/linux-aarch64::libnvjitlink-12.5.40-0 
  libnvjitlink-dev   nvidia/label/main/linux-aarch64::libnvjitlink-dev-12.5.40-0 
  libnvjpeg          nvidia/label/main/linux-aarch64::libnvjpeg-12.3.2.38-0 
  libnvjpeg-dev      nvidia/label/main/linux-aarch64::libnvjpeg-dev-12.3.2.38-0 
  libpng             pkgs/main/linux-aarch64::libpng-1.6.39-h998d150_0 
  libstdcxx-devel_l~ pkgs/main/linux-aarch64::libstdcxx-devel_linux-aarch64-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-aarch64::libstdcxx-ng-11.2.0-h1234567_1 
  libuuid            pkgs/main/linux-aarch64::libuuid-1.41.5-h998d150_0 
  libxcb             pkgs/main/linux-aarch64::libxcb-1.15-h2f4d8fa_0 
  libxkbcommon       pkgs/main/linux-aarch64::libxkbcommon-1.0.1-h998d150_1 
  libxml2            pkgs/main/linux-aarch64::libxml2-2.10.4-h6097fa9_2 
  ncurses            pkgs/main/linux-aarch64::ncurses-6.4-h419075a_0 
  nsight-compute     nvidia/label/main/linux-aarch64::nsight-compute-2024.2.0.16-2 
  nspr               pkgs/main/linux-aarch64::nspr-4.35-h419075a_0 
  nss                pkgs/main/linux-aarch64::nss-3.89.1-h419075a_0 
  openssl            pkgs/main/linux-aarch64::openssl-3.0.13-h2f4d8fa_2 
  pcre2              pkgs/main/linux-aarch64::pcre2-10.42-hcfaa891_1 
  pip                pkgs/main/linux-aarch64::pip-24.0-py312hd43f75c_0 
  python             pkgs/main/linux-aarch64::python-3.12.3-h8edadfe_1 
  readline           pkgs/main/linux-aarch64::readline-8.2-h998d150_0 
  setuptools         pkgs/main/linux-aarch64::setuptools-69.5.1-py312hd43f75c_0 
  sqlite             pkgs/main/linux-aarch64::sqlite-3.45.3-h998d150_0 
  sysroot_linux-aar~ pkgs/main/noarch::sysroot_linux-aarch64-2.26-h4744778_14 
  tk                 pkgs/main/linux-aarch64::tk-8.6.14-h987d8db_0 
  tzdata             pkgs/main/noarch::tzdata-2024a-h04d1e81_0 
  wheel              pkgs/main/linux-aarch64::wheel-0.43.0-py312hd43f75c_0 
  xz                 pkgs/main/linux-aarch64::xz-5.4.6-h998d150_1 
  zlib               pkgs/main/linux-aarch64::zlib-1.2.13-h998d150_1 



DryRunExit: Dry run. Exiting.

So sounds like this is just an issue mirroring to CDN

@jakirkham
Copy link
Member Author

cc @chenghlee (for vis)

@chenghlee
Copy link
Contributor

chenghlee commented May 23, 2024

Huh. Is still an issue on your end? It seems to be working for me, so if you're still seeing the issue, very likely some weird CDN thing.

$ conda create --dry-run --platform=linux-aarch64 --json -n __test__ \
  --override-channels -c nvidia -c main cuda=12.5 | \
  jq -rS '.actions.LINK|.[]|[.channel,.platform,.dist_name]|join("/")' | \
  sort -t/ -k3
main/linux-aarch64/binutils_impl_linux-aarch64-2.38-h0c9fd12_1
main/linux-aarch64/binutils_linux-aarch64-2.38.0-h0c7b1b4_0
main/linux-aarch64/bzip2-1.0.8-h998d150_6
main/linux-aarch64/ca-certificates-2024.3.11-hd43f75c_0
nvidia/linux-aarch64/cuda-12.5.0-0
nvidia/linux-aarch64/cuda-cccl-12.5.39-0
nvidia/noarch/cuda-cccl_linux-aarch64-12.5.39-0
nvidia/linux-aarch64/cuda-command-line-tools-12.5.0-0
nvidia/linux-aarch64/cuda-compiler-12.5.0-0
nvidia/noarch/cuda-crt-dev_linux-aarch64-12.5.40-0
nvidia/linux-aarch64/cuda-crt-tools-12.5.40-0
nvidia/linux-aarch64/cuda-cudart-12.5.39-0
nvidia/linux-aarch64/cuda-cudart-dev-12.5.39-0
nvidia/noarch/cuda-cudart-dev_linux-aarch64-12.5.39-0
nvidia/noarch/cuda-cudart_linux-aarch64-12.5.39-0
nvidia/linux-aarch64/cuda-cudart-static-12.5.39-0
nvidia/noarch/cuda-cudart-static_linux-aarch64-12.5.39-0
nvidia/linux-aarch64/cuda-cuobjdump-12.5.39-0
nvidia/linux-aarch64/cuda-cupti-12.5.39-0
nvidia/linux-aarch64/cuda-cupti-dev-12.5.39-0
nvidia/linux-aarch64/cuda-cuxxfilt-12.5.39-0
nvidia/linux-aarch64/cuda-driver-dev-12.5.39-0
nvidia/noarch/cuda-driver-dev_linux-aarch64-12.5.39-0
nvidia/linux-aarch64/cuda-gdb-12.5.39-0
nvidia/linux-aarch64/cuda-libraries-12.5.0-0
nvidia/linux-aarch64/cuda-libraries-dev-12.5.0-0
nvidia/linux-aarch64/cuda-nvcc-12.5.40-0
nvidia/noarch/cuda-nvcc-dev_linux-aarch64-12.5.40-0
nvidia/linux-aarch64/cuda-nvcc-impl-12.5.40-0
nvidia/linux-aarch64/cuda-nvcc_linux-aarch64-12.5.40-0
nvidia/linux-aarch64/cuda-nvcc-tools-12.5.40-0
nvidia/linux-aarch64/cuda-nvdisasm-12.5.39-0
nvidia/linux-aarch64/cuda-nvml-dev-12.5.39-0
nvidia/linux-aarch64/cuda-nvprune-12.5.39-0
nvidia/linux-aarch64/cuda-nvrtc-12.5.40-0
nvidia/linux-aarch64/cuda-nvrtc-dev-12.5.40-0
nvidia/linux-aarch64/cuda-nvtx-12.5.39-0
nvidia/noarch/cuda-nvvm-dev_linux-aarch64-12.5.40-0
nvidia/linux-aarch64/cuda-nvvm-impl-12.5.40-0
nvidia/linux-aarch64/cuda-nvvm-tools-12.5.40-0
nvidia/linux-aarch64/cuda-profiler-api-12.5.39-0
nvidia/linux-aarch64/cuda-runtime-12.5.0-0
nvidia/linux-aarch64/cuda-sanitizer-api-12.5.39-0
nvidia/linux-aarch64/cuda-toolkit-12.5.0-0
nvidia/linux-aarch64/cuda-tools-12.5.0-0
nvidia/noarch/cuda-version-12.5-3
nvidia/linux-aarch64/cuda-visual-tools-12.5.0-0
main/linux-aarch64/dbus-1.13.18-h821dc26_0
main/linux-aarch64/expat-2.6.2-h419075a_0
main/linux-aarch64/fontconfig-2.14.1-h652894d_2
main/linux-aarch64/freetype-2.12.1-h6df46f4_0
main/linux-aarch64/gcc_impl_linux-aarch64-11.2.0-h1234567_1
main/linux-aarch64/gcc_linux-aarch64-11.2.0-h3fc98b3_0
nvidia/linux-aarch64/gds-tools-1.10.0.4-0
main/linux-aarch64/glib-2.78.4-h419075a_0
main/linux-aarch64/glib-tools-2.78.4-h419075a_0
main/linux-aarch64/gmp-6.2.1-h22f4aa5_3
main/linux-aarch64/gxx_impl_linux-aarch64-11.2.0-h1234567_1
main/linux-aarch64/gxx_linux-aarch64-11.2.0-h0c7b1b4_0
main/linux-aarch64/icu-73.1-h419075a_0
main/noarch/kernel-headers_linux-aarch64-4.14.214-h1cb168f_14
main/linux-aarch64/ld_impl_linux-aarch64-2.38-h8131f2d_1
nvidia/linux-aarch64/libcublas-12.5.2.13-0
nvidia/linux-aarch64/libcublas-dev-12.5.2.13-0
nvidia/linux-aarch64/libcufft-11.2.3.18-0
nvidia/linux-aarch64/libcufft-dev-11.2.3.18-0
nvidia/linux-aarch64/libcufile-1.10.0.4-0
nvidia/linux-aarch64/libcufile-dev-1.10.0.4-0
nvidia/linux-aarch64/libcurand-10.3.6.39-0
nvidia/linux-aarch64/libcurand-dev-10.3.6.39-0
nvidia/linux-aarch64/libcusolver-11.6.2.40-0
nvidia/linux-aarch64/libcusolver-dev-11.6.2.40-0
nvidia/linux-aarch64/libcusparse-12.4.1.24-0
nvidia/linux-aarch64/libcusparse-dev-12.4.1.24-0
main/linux-aarch64/libffi-3.4.4-h419075a_1
main/linux-aarch64/libgcc-devel_linux-aarch64-11.2.0-h1234567_1
main/linux-aarch64/_libgcc_mutex-0.1-main
main/linux-aarch64/libgcc-ng-11.2.0-h1234567_1
main/linux-aarch64/libglib-2.78.4-hd439bcf_0
main/linux-aarch64/libgomp-11.2.0-h1234567_1
main/linux-aarch64/libiconv-1.16-h998d150_3
nvidia/linux-aarch64/libnpp-12.3.0.116-0
nvidia/linux-aarch64/libnpp-dev-12.3.0.116-0
nvidia/linux-aarch64/libnvfatbin-12.5.39-0
nvidia/linux-aarch64/libnvfatbin-dev-12.5.39-0
nvidia/linux-aarch64/libnvjitlink-12.5.40-0
nvidia/linux-aarch64/libnvjitlink-dev-12.5.40-0
nvidia/linux-aarch64/libnvjpeg-12.3.2.38-0
nvidia/linux-aarch64/libnvjpeg-dev-12.3.2.38-0
main/linux-aarch64/libpng-1.6.39-h998d150_0
main/linux-aarch64/libstdcxx-devel_linux-aarch64-11.2.0-h1234567_1
main/linux-aarch64/libstdcxx-ng-11.2.0-h1234567_1
main/linux-aarch64/libuuid-1.41.5-h998d150_0
main/linux-aarch64/libxcb-1.15-h2f4d8fa_0
main/linux-aarch64/libxkbcommon-1.0.1-h998d150_1
main/linux-aarch64/libxml2-2.10.4-h6097fa9_2
main/linux-aarch64/ncurses-6.4-h419075a_0
nvidia/linux-aarch64/nsight-compute-2024.2.0.16-2
main/linux-aarch64/nspr-4.35-h419075a_0
main/linux-aarch64/nss-3.89.1-h419075a_0
main/linux-aarch64/_openmp_mutex-5.1-51_gnu
main/linux-aarch64/openssl-3.0.13-h2f4d8fa_2
main/linux-aarch64/pcre2-10.42-hcfaa891_1
main/linux-aarch64/pip-24.0-py312hd43f75c_0
main/linux-aarch64/python-3.12.3-h8edadfe_1
main/linux-aarch64/readline-8.2-h998d150_0
main/linux-aarch64/setuptools-69.5.1-py312hd43f75c_0
main/linux-aarch64/sqlite-3.45.3-h998d150_0
main/noarch/sysroot_linux-aarch64-2.26-h4744778_14
main/linux-aarch64/tk-8.6.14-h987d8db_0
main/noarch/tzdata-2024a-h04d1e81_0
main/linux-aarch64/wheel-0.43.0-py312hd43f75c_0
main/linux-aarch64/xz-5.4.6-h998d150_1
main/linux-aarch64/zlib-1.2.13-h998d150_1

@jakirkham
Copy link
Member Author

Yes am still seeing it unfortunately as is a colleague. Reproduced just before writing this

We tried a bunch of things:

  • With VPN
  • Without VPN
  • Different VPN locations
  • Different machines at work

Always the same issue with creation

@jakirkham
Copy link
Member Author

Should add I wanted to confirm it wasn't something with corp networking. Have reproduced the same error on my personal laptop (which is x86_64 based)

@jakirkham
Copy link
Member Author

Should add the same issue is not present with CUDA 12.4. So this works

conda create -c nvidia -n cuda12.4 cuda=12.4 --dry-run

Mentioning in case this provides any clues as to the differences between CUDA 12.4 & 12.5 packages

@jakirkham
Copy link
Member Author

So I just tried downloading the repodata itself and am not seeing the packages there either

Commands:

curl -L https://conda.anaconda.org/nvidia/linux-aarch64/repodata.json > nvidia_linux-aarch64_repodata.json
cat nvidia_linux-aarch64_repodata.json | grep '12\.5\.0'

Output:

    "cuda-python-12.5.0-py310hdae0e22_0.tar.bz2": {
      "version": "12.5.0"
    "cuda-python-12.5.0-py311hdae0e22_0.tar.bz2": {
      "version": "12.5.0"
    "cuda-python-12.5.0-py312hdae0e22_0.tar.bz2": {
      "version": "12.5.0"
    "cuda-python-12.5.0-py39hdae0e22_0.tar.bz2": {
      "version": "12.5.0"

For some reason only cuda-python shows up, which was uploaded before any of the CTK packages were uploaded

@chenghlee
Copy link
Contributor

Can you try an anaconda login (if you have an anaconda.org) and then attempt the conda create command again?

@chenghlee
Copy link
Contributor

This has been escalated to Anaconda's infrastructure team. We have identified two possible causes and are actively investigating them.

@jakirkham
Copy link
Member Author

Thanks Cheng! 🙏

Hope everyone had a good weekend!

Am curious how things are going here

@chenghlee
Copy link
Contributor

It looks like the channel cloning CDN is having problems dealing with .conda format files on the nvidia channel. We're going to deploy an attempted fix in the morning EU time.

@jakirkham
Copy link
Member Author

Ah ok. That makes sense. This is our first release using .conda

Thanks Cheng! 🙏

Please let us know when we should recheck

@jakirkham
Copy link
Member Author

Did this fix get deployed?

@chenghlee
Copy link
Contributor

We tried it and it didn't work. 😞 We have an alternative approach, but that will take longer to roll out. (Still working on getting an estimate.)

In the meantime, you might want to consider falling back to the older .tar.bz2 format, at least for this release until we get the CDN sorted out.

@jakirkham
Copy link
Member Author

jakirkham commented May 29, 2024

Gotcha, thanks Cheng! 🙏

If we uploaded .tar.bz2 and kept .conda packages, would that work or would that cause issues? IOW should we delete the .conda packages and then upload .tar.bz2 packages?

@chenghlee
Copy link
Contributor

You can upload both formats (i.e., keep the .conda ones already there). The .org backend definitely supports that, and I don't think having both formats would cause any problems since bug in the CDN cloning process is [essentially] just ignoring the repodata "packages.conda" entries supplied by the backend.

BTW, the reason -c nvidia/label/main made the packages appear is that adding labels to the channel name bypasses the CDN clones and connects directly to the .org backend.

@adibbley
Copy link

We reuploaded the CUDA packages in .tar.bz2 format and things seem to be working as expected. Thanks!

@jakirkham jakirkham changed the title Unable to install available package from nvidia channel .conda packages uploaded to nvidia channel not mirroring to CDN May 30, 2024
@jakirkham
Copy link
Member Author

Thanks Cheng and Alex! 🙏

Have rewritten the OP and title to point out the remaining issue of mirroring nvidia channel .conda packages to CDN

@jezdez jezdez added source::partner created by or for an Anaconda, Inc. partner company severity::2 critical; broken functionality with an unacceptably complex workaround labels May 31, 2024
@jezdez jezdez moved this from 🆕 New to 🏗️ In Progress in 🧭 Planning May 31, 2024
@jakirkham
Copy link
Member Author

Am curious what the current status is on this issue. It would be nice to have .conda packages for future CUDA releases

Expect users would appreciate the smaller packages for downloads

Not to mention having access to package metadata without needing to do a full decompression is more noticeable with these larger packages

@jakirkham
Copy link
Member Author

This happened again with CUDA 12.5.1. We used .tar.bz2

@billysuh7
Copy link

The same commands which failed for 12.5.0 are working for me. In fact the same commands work for 12.5.1 as well, even though 12.5.1 does not yet have .conda packages uploaded. So I don't know if this issue has been fixed or not ? Please let me know if I missed something.

I'd assume it's not fixed yet as there has been no update of late. Could the conda infrastructure team please comment ? I appreciate it.

Click to expand
$ conda create -c nvidia -n cuda12.5 cuda=12.5
Channels:
 - nvidia
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/bsuh/miniforge3/envs/cuda12.5

  added / updated specs:
    - cuda=12.5


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    _libgcc_mutex-0.1          |             main           3 KB
    _openmp_mutex-5.1          |            1_gnu          21 KB
    _sysroot_linux-64_curr_repodata_hack-3|      haa98f57_10          12 KB
    binutils_impl_linux-64-2.38|       h2a08ee3_1         5.2 MB
    binutils_linux-64-2.38.0   |       hc2dff05_0          24 KB
    bzip2-1.0.8                |       h5eee18b_6         262 KB
    ca-certificates-2024.7.2   |       h06a4308_0         127 KB
    cuda-12.5.1                |                0          16 KB  nvidia
    cuda-cccl-12.6.37          |                0          17 KB  nvidia
    cuda-cccl_linux-64-12.6.37 |                0         1.3 MB  nvidia
    cuda-command-line-tools-12.5.1|                0          16 KB  nvidia
    cuda-compiler-12.5.1       |                0          16 KB  nvidia
    cuda-crt-dev_linux-64-12.6.68|                0          79 KB  nvidia
    cuda-crt-tools-12.6.68     |                0          19 KB  nvidia
    cuda-cudart-12.6.68        |                0          17 KB  nvidia
    cuda-cudart-dev-12.6.68    |                0          17 KB  nvidia
    cuda-cudart-dev_linux-64-12.6.68|                0         448 KB  nvidia
    cuda-cudart-static-12.6.68 |                0          17 KB  nvidia
    cuda-cudart-static_linux-64-12.6.68|                0         1.1 MB  nvidia
    cuda-cudart_linux-64-12.6.68|                0         216 KB  nvidia
    cuda-cuobjdump-12.6.68     |                0         292 KB  nvidia
    cuda-cupti-12.6.68         |                0         2.4 MB  nvidia
    cuda-cupti-dev-12.6.68     |                0         5.4 MB  nvidia
    cuda-cuxxfilt-12.6.68      |                0         288 KB  nvidia
    cuda-driver-dev-12.6.68    |                0          16 KB  nvidia
    cuda-driver-dev_linux-64-12.6.68|                0          33 KB  nvidia
    cuda-gdb-12.6.68           |                0         405 KB  nvidia
    cuda-libraries-12.5.1      |                0          16 KB  nvidia
    cuda-libraries-dev-12.5.1  |                0          16 KB  nvidia
    cuda-nsight-12.6.68        |                0       113.7 MB  nvidia
    cuda-nvcc-12.6.68          |                0          16 KB  nvidia
    cuda-nvcc-dev_linux-64-12.6.68|                0        13.4 MB  nvidia
    cuda-nvcc-impl-12.6.68     |                0          18 KB  nvidia
    cuda-nvcc-tools-12.6.68    |                0        26.7 MB  nvidia
    cuda-nvcc_linux-64-12.6.68 |                0          19 KB  nvidia
    cuda-nvdisasm-12.6.68      |                0        47.9 MB  nvidia
    cuda-nvml-dev-12.6.68      |                2         179 KB  nvidia
    cuda-nvprof-12.6.68        |                0         4.8 MB  nvidia
    cuda-nvprune-12.6.68       |                0          70 KB  nvidia
    cuda-nvrtc-12.6.68         |                0        20.2 MB  nvidia
    cuda-nvrtc-dev-12.6.68     |                0          27 KB  nvidia
    cuda-nvtx-12.6.68          |                0          29 KB  nvidia
    cuda-nvvm-dev_linux-64-12.6.68|                0          17 KB  nvidia
    cuda-nvvm-impl-12.6.68     |                0         8.9 MB  nvidia
    cuda-nvvm-tools-12.6.68    |                0        12.0 MB  nvidia
    cuda-nvvp-12.6.68          |                0       115.4 MB  nvidia
    cuda-opencl-12.6.68        |                0          26 KB  nvidia
    cuda-opencl-dev-12.6.68    |                0          87 KB  nvidia
    cuda-profiler-api-12.6.68  |                0          19 KB  nvidia
    cuda-runtime-12.5.1        |                0          16 KB  nvidia
    cuda-sanitizer-api-12.6.68 |                0        11.9 MB  nvidia
    cuda-toolkit-12.5.1        |                0          16 KB  nvidia
    cuda-tools-12.5.1          |                0          16 KB  nvidia
    cuda-version-12.6          |                3          16 KB  nvidia
    cuda-visual-tools-12.5.1   |                0          16 KB  nvidia
    dbus-1.13.18               |       hb2f20db_0         504 KB
    expat-2.6.3                |       h6a678d5_0         176 KB
    fontconfig-2.14.1          |       h55d465d_3         281 KB
    freetype-2.12.1            |       h4a9f257_0         626 KB
    gcc_impl_linux-64-11.2.0   |       h1234567_1        22.2 MB
    gcc_linux-64-11.2.0        |       h5c386dc_0          25 KB
    gds-tools-1.11.1.6         |                0        40.7 MB  nvidia
    glib-2.78.4                |       h6a678d5_0         508 KB
    glib-tools-2.78.4          |       h6a678d5_0         115 KB
    gmp-6.2.1                  |       h295c915_3         544 KB
    gxx_impl_linux-64-11.2.0   |       h1234567_1        10.6 MB
    gxx_linux-64-11.2.0        |       hc2dff05_0          24 KB
    icu-73.1                   |       h6a678d5_0        25.9 MB
    kernel-headers_linux-64-3.10.0|      h57e8cba_10         952 KB
    ld_impl_linux-64-2.38      |       h1181459_1         654 KB
    libcublas-12.6.1.4         |                0       337.1 MB  nvidia
    libcublas-dev-12.6.1.4     |                0          87 KB  nvidia
    libcufft-11.2.6.59         |                0       172.1 MB  nvidia
    libcufft-dev-11.2.6.59     |                0          29 KB  nvidia
    libcufile-1.11.1.6         |                0         1.0 MB  nvidia
    libcufile-dev-1.11.1.6     |                0          29 KB  nvidia
    libcurand-10.3.7.68        |                0        51.7 MB  nvidia
    libcurand-dev-10.3.7.68    |                0         396 KB  nvidia
    libcusolver-11.6.4.69      |                0       116.0 MB  nvidia
    libcusolver-dev-11.6.4.69  |                0          65 KB  nvidia
    libcusparse-12.5.3.3       |                0       187.8 MB  nvidia
    libcusparse-dev-12.5.3.3   |                0          50 KB  nvidia
    libffi-3.4.4               |       h6a678d5_1         141 KB
    libgcc-devel_linux-64-11.2.0|       h1234567_1         2.5 MB
    libgcc-ng-11.2.0           |       h1234567_1         5.3 MB
    libglib-2.78.4             |       hdc74915_0         1.5 MB
    libgomp-11.2.0             |       h1234567_1         474 KB
    libiconv-1.16              |       h5eee18b_3         759 KB
    libnpp-12.3.1.54           |                0       141.0 MB  nvidia
    libnpp-dev-12.3.1.54       |                0         593 KB  nvidia
    libnvfatbin-12.6.68        |                0         888 KB  nvidia
    libnvfatbin-dev-12.6.68    |                0          21 KB  nvidia
    libnvjitlink-12.6.68       |                0        17.1 MB  nvidia
    libnvjitlink-dev-12.6.68   |                0          20 KB  nvidia
    libnvjpeg-12.3.3.54        |                0         3.0 MB  nvidia
    libnvjpeg-dev-12.3.3.54    |                0          28 KB  nvidia
    libpng-1.6.39              |       h5eee18b_0         304 KB
    libstdcxx-devel_linux-64-11.2.0|       h1234567_1        14.6 MB
    libstdcxx-ng-11.2.0        |       h1234567_1         4.7 MB
    libuuid-1.41.5             |       h5eee18b_0          27 KB
    libxcb-1.15                |       h7f8727e_0         505 KB
    libxkbcommon-1.0.1         |       h097e994_2         590 KB
    libxml2-2.13.1             |       hfdd30dd_2         739 KB
    ncurses-6.4                |       h6a678d5_0         914 KB
    nsight-compute-2024.3.1.2  |                0       504.7 MB  nvidia
    nspr-4.35                  |       h6a678d5_0         244 KB
    nss-3.89.1                 |       h6a678d5_0         2.1 MB
    openssl-3.0.15             |       h5eee18b_0         5.2 MB
    pcre2-10.42                |       hebb0a14_1         1.3 MB
    pip-24.2                   |  py312h06a4308_0         2.8 MB
    python-3.12.4              |       h5148396_1        34.8 MB
    readline-8.2               |       h5eee18b_0         357 KB
    setuptools-72.1.0          |  py312h06a4308_0         2.9 MB
    sqlite-3.45.3              |       h5eee18b_0         1.2 MB
    sysroot_linux-64-2.17      |      h57e8cba_10        32.6 MB
    tk-8.6.14                  |       h39e8969_0         3.4 MB
    tzdata-2024a               |       h04d1e81_0         116 KB
    wheel-0.43.0               |  py312h06a4308_0         142 KB
    xz-5.4.6                   |       h5eee18b_1         643 KB
    zlib-1.2.13                |       h5eee18b_1         111 KB
    ------------------------------------------------------------
                                           Total:        2.10 GB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
  _sysroot_linux-64~ pkgs/main/noarch::_sysroot_linux-64_curr_repodata_hack-3-haa98f57_10
  binutils_impl_lin~ pkgs/main/linux-64::binutils_impl_linux-64-2.38-h2a08ee3_1
  binutils_linux-64  pkgs/main/linux-64::binutils_linux-64-2.38.0-hc2dff05_0
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6
  ca-certificates    pkgs/main/linux-64::ca-certificates-2024.7.2-h06a4308_0
  cuda               nvidia/linux-64::cuda-12.5.1-0
  cuda-cccl          nvidia/linux-64::cuda-cccl-12.6.37-0
  cuda-cccl_linux-64 nvidia/noarch::cuda-cccl_linux-64-12.6.37-0
  cuda-command-line~ nvidia/linux-64::cuda-command-line-tools-12.5.1-0
  cuda-compiler      nvidia/linux-64::cuda-compiler-12.5.1-0
  cuda-crt-dev_linu~ nvidia/noarch::cuda-crt-dev_linux-64-12.6.68-0
  cuda-crt-tools     nvidia/linux-64::cuda-crt-tools-12.6.68-0
  cuda-cudart        nvidia/linux-64::cuda-cudart-12.6.68-0
  cuda-cudart-dev    nvidia/linux-64::cuda-cudart-dev-12.6.68-0
  cuda-cudart-dev_l~ nvidia/noarch::cuda-cudart-dev_linux-64-12.6.68-0
  cuda-cudart-static nvidia/linux-64::cuda-cudart-static-12.6.68-0
  cuda-cudart-stati~ nvidia/noarch::cuda-cudart-static_linux-64-12.6.68-0
  cuda-cudart_linux~ nvidia/noarch::cuda-cudart_linux-64-12.6.68-0
  cuda-cuobjdump     nvidia/linux-64::cuda-cuobjdump-12.6.68-0
  cuda-cupti         nvidia/linux-64::cuda-cupti-12.6.68-0
  cuda-cupti-dev     nvidia/linux-64::cuda-cupti-dev-12.6.68-0
  cuda-cuxxfilt      nvidia/linux-64::cuda-cuxxfilt-12.6.68-0
  cuda-driver-dev    nvidia/linux-64::cuda-driver-dev-12.6.68-0
  cuda-driver-dev_l~ nvidia/noarch::cuda-driver-dev_linux-64-12.6.68-0
  cuda-gdb           nvidia/linux-64::cuda-gdb-12.6.68-0
  cuda-libraries     nvidia/linux-64::cuda-libraries-12.5.1-0
  cuda-libraries-dev nvidia/linux-64::cuda-libraries-dev-12.5.1-0
  cuda-nsight        nvidia/linux-64::cuda-nsight-12.6.68-0
  cuda-nvcc          nvidia/linux-64::cuda-nvcc-12.6.68-0
  cuda-nvcc-dev_lin~ nvidia/noarch::cuda-nvcc-dev_linux-64-12.6.68-0
  cuda-nvcc-impl     nvidia/linux-64::cuda-nvcc-impl-12.6.68-0
  cuda-nvcc-tools    nvidia/linux-64::cuda-nvcc-tools-12.6.68-0
  cuda-nvcc_linux-64 nvidia/linux-64::cuda-nvcc_linux-64-12.6.68-0
  cuda-nvdisasm      nvidia/linux-64::cuda-nvdisasm-12.6.68-0
  cuda-nvml-dev      nvidia/linux-64::cuda-nvml-dev-12.6.68-2
  cuda-nvprof        nvidia/linux-64::cuda-nvprof-12.6.68-0
  cuda-nvprune       nvidia/linux-64::cuda-nvprune-12.6.68-0
  cuda-nvrtc         nvidia/linux-64::cuda-nvrtc-12.6.68-0
  cuda-nvrtc-dev     nvidia/linux-64::cuda-nvrtc-dev-12.6.68-0
  cuda-nvtx          nvidia/linux-64::cuda-nvtx-12.6.68-0
  cuda-nvvm-dev_lin~ nvidia/noarch::cuda-nvvm-dev_linux-64-12.6.68-0
  cuda-nvvm-impl     nvidia/linux-64::cuda-nvvm-impl-12.6.68-0
  cuda-nvvm-tools    nvidia/linux-64::cuda-nvvm-tools-12.6.68-0
  cuda-nvvp          nvidia/linux-64::cuda-nvvp-12.6.68-0
  cuda-opencl        nvidia/linux-64::cuda-opencl-12.6.68-0
  cuda-opencl-dev    nvidia/linux-64::cuda-opencl-dev-12.6.68-0
  cuda-profiler-api  nvidia/linux-64::cuda-profiler-api-12.6.68-0
  cuda-runtime       nvidia/linux-64::cuda-runtime-12.5.1-0
  cuda-sanitizer-api nvidia/linux-64::cuda-sanitizer-api-12.6.68-0
  cuda-toolkit       nvidia/linux-64::cuda-toolkit-12.5.1-0
  cuda-tools         nvidia/linux-64::cuda-tools-12.5.1-0
  cuda-version       nvidia/noarch::cuda-version-12.6-3
  cuda-visual-tools  nvidia/linux-64::cuda-visual-tools-12.5.1-0
  dbus               pkgs/main/linux-64::dbus-1.13.18-hb2f20db_0
  expat              pkgs/main/linux-64::expat-2.6.3-h6a678d5_0
  fontconfig         pkgs/main/linux-64::fontconfig-2.14.1-h55d465d_3
  freetype           pkgs/main/linux-64::freetype-2.12.1-h4a9f257_0
  gcc_impl_linux-64  pkgs/main/linux-64::gcc_impl_linux-64-11.2.0-h1234567_1
  gcc_linux-64       pkgs/main/linux-64::gcc_linux-64-11.2.0-h5c386dc_0
  gds-tools          nvidia/linux-64::gds-tools-1.11.1.6-0
  glib               pkgs/main/linux-64::glib-2.78.4-h6a678d5_0
  glib-tools         pkgs/main/linux-64::glib-tools-2.78.4-h6a678d5_0
  gmp                pkgs/main/linux-64::gmp-6.2.1-h295c915_3
  gxx_impl_linux-64  pkgs/main/linux-64::gxx_impl_linux-64-11.2.0-h1234567_1
  gxx_linux-64       pkgs/main/linux-64::gxx_linux-64-11.2.0-hc2dff05_0
  icu                pkgs/main/linux-64::icu-73.1-h6a678d5_0
  kernel-headers_li~ pkgs/main/noarch::kernel-headers_linux-64-3.10.0-h57e8cba_10
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
  libcublas          nvidia/linux-64::libcublas-12.6.1.4-0
  libcublas-dev      nvidia/linux-64::libcublas-dev-12.6.1.4-0
  libcufft           nvidia/linux-64::libcufft-11.2.6.59-0
  libcufft-dev       nvidia/linux-64::libcufft-dev-11.2.6.59-0
  libcufile          nvidia/linux-64::libcufile-1.11.1.6-0
  libcufile-dev      nvidia/linux-64::libcufile-dev-1.11.1.6-0
  libcurand          nvidia/linux-64::libcurand-10.3.7.68-0
  libcurand-dev      nvidia/linux-64::libcurand-dev-10.3.7.68-0
  libcusolver        nvidia/linux-64::libcusolver-11.6.4.69-0
  libcusolver-dev    nvidia/linux-64::libcusolver-dev-11.6.4.69-0
  libcusparse        nvidia/linux-64::libcusparse-12.5.3.3-0
  libcusparse-dev    nvidia/linux-64::libcusparse-dev-12.5.3.3-0
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1
  libgcc-devel_linu~ pkgs/main/linux-64::libgcc-devel_linux-64-11.2.0-h1234567_1
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
  libglib            pkgs/main/linux-64::libglib-2.78.4-hdc74915_0
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
  libiconv           pkgs/main/linux-64::libiconv-1.16-h5eee18b_3
  libnpp             nvidia/linux-64::libnpp-12.3.1.54-0
  libnpp-dev         nvidia/linux-64::libnpp-dev-12.3.1.54-0
  libnvfatbin        nvidia/linux-64::libnvfatbin-12.6.68-0
  libnvfatbin-dev    nvidia/linux-64::libnvfatbin-dev-12.6.68-0
  libnvjitlink       nvidia/linux-64::libnvjitlink-12.6.68-0
  libnvjitlink-dev   nvidia/linux-64::libnvjitlink-dev-12.6.68-0
  libnvjpeg          nvidia/linux-64::libnvjpeg-12.3.3.54-0
  libnvjpeg-dev      nvidia/linux-64::libnvjpeg-dev-12.3.3.54-0
  libpng             pkgs/main/linux-64::libpng-1.6.39-h5eee18b_0
  libstdcxx-devel_l~ pkgs/main/linux-64::libstdcxx-devel_linux-64-11.2.0-h1234567_1
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
  libuuid            pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0
  libxcb             pkgs/main/linux-64::libxcb-1.15-h7f8727e_0
  libxkbcommon       pkgs/main/linux-64::libxkbcommon-1.0.1-h097e994_2
  libxml2            pkgs/main/linux-64::libxml2-2.13.1-hfdd30dd_2
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
  nsight-compute     nvidia/linux-64::nsight-compute-2024.3.1.2-0
  nspr               pkgs/main/linux-64::nspr-4.35-h6a678d5_0
  nss                pkgs/main/linux-64::nss-3.89.1-h6a678d5_0
  openssl            pkgs/main/linux-64::openssl-3.0.15-h5eee18b_0
  pcre2              pkgs/main/linux-64::pcre2-10.42-hebb0a14_1
  pip                pkgs/main/linux-64::pip-24.2-py312h06a4308_0
  python             pkgs/main/linux-64::python-3.12.4-h5148396_1
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0
  setuptools         pkgs/main/linux-64::setuptools-72.1.0-py312h06a4308_0
  sqlite             pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0
  sysroot_linux-64   pkgs/main/noarch::sysroot_linux-64-2.17-h57e8cba_10
  tk                 pkgs/main/linux-64::tk-8.6.14-h39e8969_0
  tzdata             pkgs/main/noarch::tzdata-2024a-h04d1e81_0
  wheel              pkgs/main/linux-64::wheel-0.43.0-py312h06a4308_0
  xz                 pkgs/main/linux-64::xz-5.4.6-h5eee18b_1
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1


Proceed ([y]/n)? y


Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate cuda12.5
#
# To deactivate an active environment, use
#
#     $ conda deactivate

$ ls -l miniforge3/envs/cuda12.5/
total 184
drwxrwxr-x  3 bsuh bsuh 12288 Sep  9 20:10 bin
drwxrwxr-x  2 bsuh bsuh  4096 Sep  9 20:10 cmake
drwxrwxr-x  2 bsuh bsuh  4096 Sep  9 20:10 compiler_compat
drwxrwxr-x  4 bsuh bsuh  4096 Sep  9 20:10 compute-sanitizer
drwxrwxr-x  2 bsuh bsuh 12288 Sep  9 20:10 conda-meta
drwxrwxr-x  6 bsuh bsuh  4096 Sep  9 20:10 etc
drwxrwxr-x  3 bsuh bsuh  4096 Sep  9 20:10 extras
drwxrwxr-x  3 bsuh bsuh  4096 Sep  9 20:10 gds
drwxrwxr-x 21 bsuh bsuh  4096 Sep  9 20:10 include
drwxrwxr-x 23 bsuh bsuh 20480 Sep  9 20:10 lib
drwxrwxr-x  4 bsuh bsuh  4096 Sep  9 20:10 libexec
drwxrwxr-x  7 bsuh bsuh  4096 Sep  9 20:10 libnvvp
-rw-rw-r--  2 bsuh bsuh 63021 Aug 14 22:25 LICENSE
drwxrwxr-x  4 bsuh bsuh  4096 Sep  9 20:10 man
drwxrwxr-x  9 bsuh bsuh  4096 Sep  9 20:10 nsight-compute-2024.3.1
drwxrwxr-x  2 bsuh bsuh  4096 Sep  9 20:10 nsightee_plugins
drwxrwxr-x  6 bsuh bsuh  4096 Sep  9 20:10 nvvm
drwxrwxr-x 20 bsuh bsuh  4096 Sep  9 20:10 share
drwxrwxr-x  3 bsuh bsuh  4096 Sep  9 20:10 ssl
drwxrwxr-x  3 bsuh bsuh  4096 Sep  9 20:10 targets
drwxrwxr-x  3 bsuh bsuh  4096 Sep  9 20:10 var
drwxrwxr-x  3 bsuh bsuh  4096 Sep  9 20:10 x86_64-conda_cos7-linux-gnu
drwxrwxr-x  7 bsuh bsuh  4096 Sep  9 20:10 x86_64-conda-linux-gnu


$ conda search --skip-flexible-search -c nvidia 'cuda[subdir=linux-64]=12.5'
Loading channels: done
# Name                       Version           Build  Channel
cuda                          12.5.0               0  nvidia
cuda                          12.5.1               0  nvidia

@morremeyer
Copy link

@jakirkham @billysuh7 We have upgraded the CDN cloning jobs over the last few weeks. On September 12th, we upgraded the nvidia channel to the newest version, which fully supports the .conda archive format.

With that, the existing .conda archives should be downloadable and any newly uploaded .conda archives will also be incorporated.

@jezdez
Copy link
Member

jezdez commented Sep 16, 2024

Closing as fixed.

@jezdez jezdez closed this as completed Sep 16, 2024
@github-project-automation github-project-automation bot moved this from 🏗️ In Progress to 🏁 Done in 🧭 Planning Sep 16, 2024
@jakirkham
Copy link
Member Author

Awesome thank you! 🙏

@morremeyer could you please check rapidsai as well?

@morremeyer
Copy link

@jakirkham rapidsai is currently not a CDN cloned channel, so it should not have been affected by this. I just checked and .conda packages are present in the repodata for the linux-64 subdir.

@jakirkham
Copy link
Member Author

@jakirkham rapidsai is currently not a CDN cloned channel, so it should not have been affected by this

Ok this is something we have discussed enabling before. Not sure where it got stuck, but let's complete this work. Raised issue: #1055

I just checked and .conda packages are present in the repodata for the linux-64 subdir.

Great, thanks! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
severity::2 critical; broken functionality with an unacceptably complex workaround source::partner created by or for an Anaconda, Inc. partner company
Projects
Archived in project
Development

No branches or pull requests

7 participants