[feature] Normalizing path in generators

[KUGA2]

What is your suggestion?

Hi!

we export generator and package dirs (with deployer). The generator dir (find_package() *_data.cmake files ) contains very long paths:

e.G.:

set(grpc_PACKAGE_FOLDER_RELWITHDEBINFO "${CMAKE_CURRENT_LIST_DIR}/../../../Build/armv8-lin-gcc-RelWi-conan/conan_installed/host/grpc/1.54.3/armv8")

This causes problems in our Windows build. Windows has the ancient limitation of 260 characters in path name. This problem can be mitigated by registry (if your IT department plays ball) or by using a shorter base path. Sometimes both is not possible.
(As you can see, we already abbreviated RelWithDebInfo to fight for every byte here :( )

Could conan change the path so this error is less likely to happen?

a) Generate a shorter path here
In our case, this could be shortended to:

set(grpc_PACKAGE_FOLDER_RELWITHDEBINFO "${CMAKE_CURRENT_LIST_DIR}/../conan_installed/host/grpc/1.54.3/armv8")

b) Set it as before, but NORMALIZE it afterwards, before passing it to tools

cmake_path(SET grpc_PACKAGE_FOLDER_RELWITHDEBINFO NORMALIZE "${grpc_PACKAGE_FOLDER_RELWITHDEBINFO}")

(This is CMake 3.20, and I am not sure weather this is allowed here)

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Thanks for your suggestion @KUGA2

I am checking the generated code, and the paths that I am getting are of the form:

${CMAKE_CURRENT_LIST_DIR}/full_deploy/host/dep/0.1/Release

Not sure where the multiple ../../.. comes from.
Could you please provide details, a minimal conanfile and exact commands to reproduce and obtain such a path? Thanks!

[KUGA2]

We changed the build path, the generator path and the path where the dependencies are (usually in cache dir, but we got a custom deployer)

        self.folders.build = os.path.join(self.folders.source, "Build",
                                          self.settings.arch.value +                        # x86_64
                                          "-" + self.settings.os.value[:3].lower() +        # lin[ux]
                                          "-" + self.settings.compiler.value +              # gcc
                                          "-" + self.settings.build_type.value[:5] +        # Relea[se]
                                          ("-static" if not self.options.shared else "") +  # static
                                          special_build_suffix +                            # safety
                                          "-conan")                                         # conan
        self.folders.generators = os.path.join(self.folders.build, "generators")

And the deployer puts all dependencies into <self.folders.build>/conan_installed/<dep.ref.name>/<dep.ref.version>/<dep.info.settings.get_safe("arch")>:

def deploy(graph, output_folder):
    """
    Deploys to <build_folder>/conan_installed. Ignores output_folder variable.
    """
    # Just like conans "full_deploy" deployer this will not work with editable
    # packages.
    conanfile = graph.root.conanfile
    output_folder = os.path.join(conanfile.build_folder, "conan_installed")
    # Make path absolute. conanfile.build_folder is (usually) relative.
    output_folder = os.path.abspath(output_folder)
    conanfile.output.info(f"Custom DevPackage deployer to {output_folder}")

    for dep in conanfile.dependencies.values():
        if dep.package_folder is None:
            conanfile.output.info(f"No package folder for dependency {dep} - skipping")
            continue
        folder_name = os.path.join(dep.context, dep.ref.name, str(dep.ref.version))
        arch = dep.info.settings.get_safe("arch")
        if arch:
            folder_name = os.path.join(folder_name, arch)

        output_folder = os.path.abspath(output_folder)
        conanfile.output.info(f"Deploying dependency {dep} to subdirectory {folder_name}")
        _deploy_single(dep, conanfile, output_folder, folder_name)

I think, the path in the *_data.cmake file is generated like this (from observation, not from reading conan code):

• Go from file location (CMAKE_CURRENT_LIST_DIR) upwards to folder containing conanfile.py (or self.folders.source)
• Then add relative path from there to binaries (path in custom deployer)

This traversing ends up going up and down into nearly the same folder.

PS: If you are asking why we are doing this madness: We need to provide a full package (zip) to our internal customer (similar to what they got from us before) for a transition period before they can adopt out conan package. We need to export our binaries, our cmake descriptions and also 3rd party binaries and cmake descriptions (as in conan generators folder)

[memsharded]

Thanks for the feedback, it helps.

I am trying (as a draft and experiment, not guaranteed to move forward, lets see if there are corner cases or possible risks of breaking) #17945 trying to make those relative paths shorter.

[KUGA2]

Thank you!

Adding the normalize line in cmake is not an option? It seems easier to me (also it can remove all the ../ paths and save more chars). I tested it (manually adjusting data files) and it works.

[memsharded]

Adding the normalize line in cmake is not an option?

Conan works with CMake 3.15 as a base, and that would require CMake 3.20

[KUGA2]

Adding the normalize line in cmake is not an option?

Conan works with CMake 3.15 as a base, and that would require CMake 3.20

Thats a pitty :( I had a quick look and yes, 3.15 does not have a similar function.
You could use exexute_process(cmake -E chdir "${path_to_normalize}" cmake -P ${full_path_to_script}/print_pwd.cmake ... REDIRECT STDOUT path_to_normalize) to achieve the same (This should be correct on all Win+Lin at least. No Idea about other OS) . I am not sure that is better though...

[memsharded]

#17945 implementing shorter relative paths has been merged, it will be in next Conan 2.15 release, thanks for the feedback!

(The execute_process approach seemed more complicated, so I went via Python)