You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using a new machine (Apple laptop), and I want to develop my Horovod routines with PyCharm, so I am trying to create an environment locally. All went well, including the PyTorch install. However, when trying to install Horovod locally using pip, things derail because I cannot build it locally. My install was run in a conda environment call to pip as this:
(ccai_lai_project) ricardo@MacBook-Air ccai_lai_project % pip install --no-cache-dir horovod
Collecting horovod
Downloading horovod-0.28.1.tar.gz (3.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.5/3.5 MB 15.9 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting cloudpickle (from horovod)
Downloading cloudpickle-2.2.1-py3-none-any.whl (25 kB)
Collecting psutil (from horovod)
Downloading psutil-5.9.5-cp38-abi3-macosx_11_0_arm64.whl (246 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.1/246.1 kB 27.3 MB/s eta 0:00:00
Collecting pyyaml (from horovod)
Obtaining dependency information for pyyaml from https://files.pythonhosted.org/packages/5b/07/10033a403b23405a8fc48975444463d3d10a5c2736b7eb2550b07b367429/PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl.metadata
Downloading PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl.metadata (2.1 kB)
Requirement already satisfied: packaging in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from horovod) (23.1)
Collecting cffi>=1.4.0 (from horovod)
Downloading cffi-1.15.1-cp310-cp310-macosx_11_0_arm64.whl (174 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.3/174.3 kB 26.1 MB/s eta 0:00:00
Collecting pycparser (from cffi>=1.4.0->horovod)
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 23.1 MB/s eta 0:00:00
Downloading PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl (169 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.3/169.3 kB 24.1 MB/s eta 0:00:00
Building wheels for collected packages: horovod
Building wheel for horovod (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [272 lines of output]
/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-11.0-arm64-cpython-310
creating build/lib.macosx-11.0-arm64-cpython-310/horovod
copying horovod/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
copying horovod/keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
copying horovod/keras/callbacks.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
copying horovod/keras/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/functions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/mpi_ops.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/compression.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/task_fn.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/gloo_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/launch.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/mpi_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/run_task.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/js_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/functions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/mpi_ops.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/compression.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/sync_batch_norm.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/optimizer.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
copying horovod/_keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
copying horovod/_keras/callbacks.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
copying horovod/_keras/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/process_sets.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/basics.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/exceptions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/functions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/mpi_ops.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/compression.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/sync_batch_norm.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/gradient_aggregation_eager.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/gradient_aggregation.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/data
copying horovod/data/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/data
copying horovod/data/data_loader_base.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/data
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/runner.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/adapter.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/strategy.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/ray_logger.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/utils.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/elastic_v2.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/runner.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/conf.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/gloo_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/mpi_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/streams.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/cache.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/threads.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/network.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/lsf.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/registration.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/rendezvous.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/discovery.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/constants.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/settings.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/driver.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/driver
copying horovod/runner/driver/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/driver
copying horovod/runner/driver/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/driver
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common
copying horovod/runner/common/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
copying horovod/runner/http/http_client.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
copying horovod/runner/http/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
copying horovod/runner/http/http_server.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/task
copying horovod/runner/task/task_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/task
copying horovod/runner/task/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/task
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/env.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/config_parser.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/timeout.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/hosts.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/safe_shell_exec.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/host_hash.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/secret.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/network.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/settings.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/codec.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/tiny_shell_exec.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/task_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/compute_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
copying horovod/torch/elastic/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
copying horovod/torch/elastic/sampler.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
copying horovod/torch/elastic/state.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
copying horovod/tensorflow/keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
copying horovod/tensorflow/keras/callbacks.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
copying horovod/tensorflow/keras/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
copying horovod/tensorflow/data/compute_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
copying horovod/tensorflow/data/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
copying horovod/tensorflow/data/compute_worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/tensorflow.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/optimizer.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/bare.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/legacy.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/rendezvous.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/mpirun_rsh.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/host_discovery.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/job_id.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/rsh.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/store.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/params.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/backend.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/_namedtuple_fix.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/constants.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/cache.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/serialization.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/tensorflow
copying horovod/spark/tensorflow/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/tensorflow
copying horovod/spark/tensorflow/compute_worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/tensorflow
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/task_info.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/mpirun_exec_fn.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/task_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/gloo_exec_fn.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/data_loaders
copying horovod/spark/data_loaders/pytorch_data_loaders.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/data_loaders
copying horovod/spark/data_loaders/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/data_loaders
running build_ext
Running CMake in build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo:
cmake /private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELWITHDEBINFO=/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/build/lib.macosx-11.0-arm64-cpython-310 -DPYTHON_EXECUTABLE:FILEPATH=/Users/ricardo/mambaforge/envs/ccai_lai_project/bin/python3.10
cmake --build . --config RelWithDebInfo -- -j8 VERBOSE=1
-- Could not find CCache. Consider installing CCache to speed up compilation.
-- The CXX compiler identification is unknown
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ - broken
CMake Error at /Users/ricardo/mambaforge/envs/ccai_lai_project/share/cmake-3.22/Modules/CMakeTestCXXCompiler.cmake:62 (message):
The C++ compiler
"/usr/bin/c++"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make -f Makefile cmTC_a2628/fast && xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:17 (project)
-- Configuring incomplete, errors occurred!
See also "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo/CMakeFiles/CMakeOutput.log".
See also "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/setup.py", line 213, in <module>
setup(name='horovod',
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 364, in run
self.run_command("build")
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 88, in run
_build_ext.run(self)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/setup.py", line 145, in build_extensions
subprocess.check_call(command, cwd=cmake_build_dir)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELWITHDEBINFO=/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-i9uswwlx/horovod_70204d1123c8490a86fd75beed664b60/build/lib.macosx-11.0-arm64-cpython-310', '-DPYTHON_EXECUTABLE:FILEPATH=/Users/ricardo/mambaforge/envs/ccai_lai_project/bin/python3.10']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for horovod
Running setup.py clean for horovod
Failed to build horovod
ERROR: Could not build wheels for horovod, which is required to install pyproject.toml-based projects
Any suggestion on how can I solve this issue? The idea is not running horovod in my machine, but just be able to have minimal IDE functionality, to I can then speed up debugging ( I am actually testing my code in an HPC cluster).
--- UPDATE --
I was able to evolve a bit, by installing cxx-compiler (https://stackoverflow.com/a/69485927/4743714), however, it seems that the lack of a NVIDIA card prevents me from compiling it. Any way of getting around this?
(ccai_lai_project) ricardo@ ccai_lai_project % mamba install -c conda-forge cxx-compiler
Looking for: ['cxx-compiler']
conda-forge/noarch 12.2MB @ 8.1MB/s 1.5s
conda-forge/osx-arm64 6.5MB @ 3.1MB/s 2.1s
Pinned packages:
- python 3.10.*
Transaction
Prefix: /Users/ricardo/mambaforge/envs/ccai_lai_project
All requested packages already installed
(ccai_lai_project) ricardo@ ccai_lai_project % pip install --no-cache-dir horovod[pytorch]
Collecting horovod[pytorch]
Downloading horovod-0.28.1.tar.gz (3.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.5/3.5 MB 12.0 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting cloudpickle (from horovod[pytorch])
Downloading cloudpickle-2.2.1-py3-none-any.whl (25 kB)
Collecting psutil (from horovod[pytorch])
Downloading psutil-5.9.5-cp38-abi3-macosx_11_0_arm64.whl (246 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.1/246.1 kB 25.6 MB/s eta 0:00:00
Collecting pyyaml (from horovod[pytorch])
Obtaining dependency information for pyyaml from https://files.pythonhosted.org/packages/5b/07/10033a403b23405a8fc48975444463d3d10a5c2736b7eb2550b07b367429/PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl.metadata
Downloading PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl.metadata (2.1 kB)
Requirement already satisfied: packaging in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from horovod[pytorch]) (23.1)
Collecting cffi>=1.4.0 (from horovod[pytorch])
Downloading cffi-1.15.1-cp310-cp310-macosx_11_0_arm64.whl (174 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.3/174.3 kB 24.4 MB/s eta 0:00:00
Requirement already satisfied: torch in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from horovod[pytorch]) (2.0.1)
Collecting pycparser (from cffi>=1.4.0->horovod[pytorch])
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 21.2 MB/s eta 0:00:00
Requirement already satisfied: filelock in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from torch->horovod[pytorch]) (3.12.3)
Requirement already satisfied: typing-extensions in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from torch->horovod[pytorch]) (4.7.1)
Requirement already satisfied: sympy in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from torch->horovod[pytorch]) (1.12)
Requirement already satisfied: networkx in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from torch->horovod[pytorch]) (3.1)
Requirement already satisfied: jinja2 in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from torch->horovod[pytorch]) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from jinja2->torch->horovod[pytorch]) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages (from sympy->torch->horovod[pytorch]) (1.3.0)
Downloading PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl (169 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.3/169.3 kB 6.0 MB/s eta 0:00:00
Building wheels for collected packages: horovod
Building wheel for horovod (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [282 lines of output]
/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-11.0-arm64-cpython-310
creating build/lib.macosx-11.0-arm64-cpython-310/horovod
copying horovod/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
copying horovod/keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
copying horovod/keras/callbacks.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
copying horovod/keras/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/functions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/mpi_ops.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/compression.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
copying horovod/mxnet/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/mxnet
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/task_fn.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/gloo_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/launch.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/mpi_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/run_task.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
copying horovod/runner/js_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/functions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/mpi_ops.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/compression.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/sync_batch_norm.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
copying horovod/torch/optimizer.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
copying horovod/_keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
copying horovod/_keras/callbacks.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
copying horovod/_keras/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/_keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/process_sets.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/basics.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
copying horovod/common/exceptions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/common
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/functions.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/mpi_ops.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/compression.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/sync_batch_norm.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/gradient_aggregation_eager.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
copying horovod/tensorflow/gradient_aggregation.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/data
copying horovod/data/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/data
copying horovod/data/data_loader_base.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/data
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/runner.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/adapter.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/strategy.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/ray_logger.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/utils.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
copying horovod/ray/elastic_v2.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/ray
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/runner.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/conf.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/gloo_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
copying horovod/spark/mpi_run.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/streams.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/cache.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/threads.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/network.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
copying horovod/runner/util/lsf.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/util
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/registration.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/rendezvous.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/discovery.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/constants.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/settings.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
copying horovod/runner/elastic/driver.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/elastic
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/driver
copying horovod/runner/driver/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/driver
copying horovod/runner/driver/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/driver
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common
copying horovod/runner/common/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
copying horovod/runner/http/http_client.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
copying horovod/runner/http/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
copying horovod/runner/http/http_server.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/http
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/task
copying horovod/runner/task/task_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/task
copying horovod/runner/task/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/task
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/env.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/config_parser.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/timeout.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/hosts.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/safe_shell_exec.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/host_hash.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/secret.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/network.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/settings.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/codec.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
copying horovod/runner/common/util/tiny_shell_exec.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/util
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/task_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/compute_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
copying horovod/runner/common/service/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/runner/common/service
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
copying horovod/torch/elastic/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
copying horovod/torch/elastic/sampler.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
copying horovod/torch/elastic/state.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/torch/elastic
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
copying horovod/tensorflow/keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
copying horovod/tensorflow/keras/callbacks.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
copying horovod/tensorflow/keras/elastic.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
copying horovod/tensorflow/data/compute_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
copying horovod/tensorflow/data/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
copying horovod/tensorflow/data/compute_worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/tensorflow/data
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/tensorflow.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/optimizer.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
copying horovod/spark/keras/bare.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/keras
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/legacy.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
copying horovod/spark/lightning/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/lightning
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/rendezvous.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/mpirun_rsh.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/driver_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/host_discovery.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/job_id.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
copying horovod/spark/driver/rsh.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/driver
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/remote.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
copying horovod/spark/torch/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/torch
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/store.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/params.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/backend.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/util.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/_namedtuple_fix.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/constants.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/cache.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/datamodule.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/estimator.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
copying horovod/spark/common/serialization.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/common
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/tensorflow
copying horovod/spark/tensorflow/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/tensorflow
copying horovod/spark/tensorflow/compute_worker.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/tensorflow
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/task_info.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/mpirun_exec_fn.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/task_service.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
copying horovod/spark/task/gloo_exec_fn.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/task
creating build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/data_loaders
copying horovod/spark/data_loaders/pytorch_data_loaders.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/data_loaders
copying horovod/spark/data_loaders/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/horovod/spark/data_loaders
running build_ext
Running CMake in build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo:
cmake /private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELWITHDEBINFO=/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad/build/lib.macosx-11.0-arm64-cpython-310 -DPYTHON_EXECUTABLE:FILEPATH=/Users/ricardo/mambaforge/envs/ccai_lai_project/bin/python3.10
cmake --build . --config RelWithDebInfo -- -j8 VERBOSE=1
-- Could not find CCache. Consider installing CCache to speed up compilation.
-- The CXX compiler identification is Clang 15.0.7
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/ricardo/mambaforge/envs/ccai_lai_project/bin/arm64-apple-darwin20.0.0-clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build architecture flags:
-- Using command /Users/ricardo/mambaforge/envs/ccai_lai_project/bin/python3.10
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
-- Could NOT find MPI (missing: MPI_CXX_FOUND)
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
-- Looking for a CUDA host compiler - /Users/ricardo/mambaforge/envs/ccai_lai_project/bin/arm64-apple-darwin20.0.0-clang++
-- Could not find nvcc, please set CUDAToolkit_ROOT.
-- Could NOT find NVTX (missing: NVTX_INCLUDE_DIR)
-- The C compiler identification is Clang 15.0.7
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Users/ricardo/mambaforge/envs/ccai_lai_project/bin/arm64-apple-darwin20.0.0-clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Gloo build as STATIC library
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
-- Checking for one of the modules 'libuv>=1.26'
CMake Error at /Users/ricardo/mambaforge/envs/ccai_lai_project/share/cmake-3.22/Modules/FindPkgConfig.cmake:890 (message):
None of the required 'libuv>=1.26' found
Call Stack (most recent call first):
third_party/gloo/cmake/Dependencies.cmake:81 (pkg_search_module)
third_party/gloo/CMakeLists.txt:111 (include)
CMake Error at third_party/gloo/cmake/Dependencies.cmake:88 (message):
Unable to find static libuv library in
Call Stack (most recent call first):
third_party/gloo/CMakeLists.txt:111 (include)
-- Configuring incomplete, errors occurred!
See also "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad/build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo/CMakeFiles/CMakeOutput.log".
See also "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad/build/temp.macosx-11.0-arm64-cpython-310/RelWithDebInfo/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad/setup.py", line 213, in <module>
setup(name='horovod',
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 364, in run
self.run_command("build")
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 88, in run
_build_ext.run(self)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad/setup.py", line 145, in build_extensions
subprocess.check_call(command, cwd=cmake_build_dir)
File "/Users/ricardo/mambaforge/envs/ccai_lai_project/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELWITHDEBINFO=/private/var/folders/kv/lgm4tw951sl0vd7ls_78gp540000gn/T/pip-install-m79p76f2/horovod_e2a6cc26e5844b0cb14bda1f7a56a3ad/build/lib.macosx-11.0-arm64-cpython-310', '-DPYTHON_EXECUTABLE:FILEPATH=/Users/ricardo/mambaforge/envs/ccai_lai_project/bin/python3.10']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for horovod
Running setup.py clean for horovod
Failed to build horovod
ERROR: Could not build wheels for horovod, which is required to install pyproject.toml-based projects
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi everyone!
I am using a new machine (Apple laptop), and I want to develop my Horovod routines with PyCharm, so I am trying to create an environment locally. All went well, including the PyTorch install. However, when trying to install Horovod locally using pip, things derail because I cannot build it locally. My install was run in a conda environment call to pip as this:
Any suggestion on how can I solve this issue? The idea is not running horovod in my machine, but just be able to have minimal IDE functionality, to I can then speed up debugging ( I am actually testing my code in an HPC cluster).
--- UPDATE --
I was able to evolve a bit, by installing cxx-compiler (https://stackoverflow.com/a/69485927/4743714), however, it seems that the lack of a NVIDIA card prevents me from compiling it. Any way of getting around this?
Beta Was this translation helpful? Give feedback.
All reactions