Wheel repair question

[ErichZimmer]

Hi all,

I am trying to perform wheel repairing on my first c++/python project. The dynamic libraries are built on build time and do not exist beforehand. Here is a quick folder layout.

project_folder
project_folder/extern (third party packages, some managed by vcpkg)
project_folder/openpiv_cxx(namespace python package)
project_folder/src (pure c++/c code)
project_folder/setup.py
project_folder/vcpkg.json (list of external dependencies)

Using scikit-build and cmake, this compiles into the following folder structure.

my_proj/_libs (contains the built runtime dependencies)
my_project/submodule1 (contains pybind11 extensions linked to dynamic libraries)
my_proj/set_path.py (adds _libs and dynamic libraries to path so it can be found)

However, when performing wheel repair, delocate can't find the libraries in the _libs folder. Even using

CIBW_REPAIR_WHEEL_COMMAND_MACOS: >
  DYLD_LIBRARY_PATH=$REPAIR_LIBRARY_PATH delocate-listdeps {wheel} &&
  DYLD_LIBRARY_PATH=$REPAIR_LIBRARY_PATH delocate-wheel --require-archs {delocate_archs} -w {dest_dir} {wheel}

and setting REPAIR_LIBRSRY_PATH, I still have issues because of locating the temporary dir in which the wheel is built. I saw this script in a previous answer, but I would like to stay away from bash scripts. Is there a way to locate the dynamic libraries built under _libs in CIBW, or should I simply use bash scripts to circumnavigate my current issue? Or should I simply restructure my project in a way CIBW can somehow work?

[ErichZimmer]

Here is a short copy of the log.

  + DYLD_LIBRARY_PATH=$LIBRARY_PATH delocate-listdeps /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/built_wheel/OpenPIV_cxx-0.3.2-cp38-cp38-macosx_10_14_x86_64.whl && DYLD_LIBRARY_PATH=$LIBRARY_PATH delocate-wheel --require-archs x86_64 -w /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/repaired_wheel /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/built_wheel/OpenPIV_cxx-0.3.2-cp38-cp38-macosx_10_14_x86_64.whl
  
  @rpath/libopenpivcore.dylib not found:
    Needed by: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpl72oe3y4/openpiv_cxx/process/_process_cpp.cpython-38-darwin.so
    Search path:
      
  
  @rpath/libopenpivcore.dylib not found:
    Needed by: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpl72oe3y4/openpiv_cxx/process/_process_cpp.cpython-38-darwin.so
    Search path:
      
  @rpath/libopenpivcore.dylib not found, requested by /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpl72oe3y4/openpiv_cxx/process/_process_cpp.cpython-38-darwin.so
  ERROR:delocate.libsana:
  @rpath/libopenpivcore.dylib not found:
    Needed by: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpu5f1xels/wheel/openpiv_cxx/process/_process_cpp.cpython-38-darwin.so
    Search path:
      
  ERROR:delocate.libsana:
  @rpath/libopenpivcore.dylib not found:
    Needed by: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpu5f1xels/wheel/openpiv_cxx/process/_process_cpp.cpython-38-darwin.so
    Search path:
      
  ERROR:delocate.libsana:@rpath/libopenpivcore.dylib not found, requested by /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpu5f1xels/wheel/openpiv_cxx/process/_process_cpp.cpython-38-darwin.so
  Traceback (most recent call last):
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/build/venv/bin/delocate-wheel", line 8, in <module>
      sys.exit(main())
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/build/venv/lib/python3.8/site-packages/delocate/cmd/delocate_wheel.py", line 128, in main
      copied = delocate_wheel(
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/build/venv/lib/python3.8/site-packages/delocate/delocating.py", line 643, in delocate_wheel
      copied_libs = delocate_path(
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/build/venv/lib/python3.8/site-packages/delocate/delocating.py", line 465, in delocate_path
      lib_dict = tree_libs_from_directory(
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/build/venv/lib/python3.8/site-packages/delocate/libsana.py", line 384, in tree_libs_from_directory
      return _tree_libs_from_libraries(
    File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-r_tepl4z/cp38-macosx_x86_64/build/venv/lib/python3.8/site-packages/delocate/libsana.py", line 325, in _tree_libs_from_libraries
      raise delocate.delocating.DelocationError(
  delocate.delocating.DelocationError: Could not find all dependencies.

[ErichZimmer]

To answer my question, restructuring my package worked. I followed what the maintainers of Freud did and installed the shared libraries directly into the sub-modules so CIBW can repair it.

[joerick]

Thanks for the follow up!