conan_provider.cmake and conan options?

[drmacdon]

The CMake conan_provider.cmake approach as pointed out in #635 does not invoke conan until the first find_package(), but CMake processes it just after the project() line. I was not clear on why conan was deferred until the first find_package()?

Today, when conan is invoked it includes the option setting in the conan_toolchain.cmake but those option settings are unavailable to the conan_provider.cmake path and the point at which find_package() is actually invoked typically means CMake options have already been initialized.

To avoid replicating conan options when using conan_provider.cmake, I ended up doing something like the below which shells out to conan graph info to get the current values set prior the regular CMake option setting.

But if conan was resolvable at the point that conan_provider.cmake was evaluated it could do something similar with far less dependencies and be more seamless enabling the CMake to remain entirely agnostic of conan.

cmake_minimum_required(VERSION 3.24 FATAL_ERROR)

execute_process(
    COMMAND just cmake-options ${CMAKE_BINARY_DIR}/conan-options.cmake
    WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)

project(testapp VERSION 1.0 LANGUAGES CXX C)

if (DEFINED CMAKE_PROJECT_TOP_LEVEL_INCLUDES AND EXISTS ${CMAKE_BINARY_DIR}/conan-options.cmake)
  include(${CMAKE_BINARY_DIR}/conan-options.cmake)
endif()

set(XYZ_OPTION None CACHE STRING "The XYZ OPTION")

Where the just cmake-options writes out the options via graph,

set unstable    # requires script from Just v1.32.0 to run nushell scripts

[script("nu"), private]
cmake-options OUTPUT_FILE="-":
	let options = (conan graph info "{{justfile_directory()}}" --format=json -vquiet | from json | get graph | get nodes | get "0" | get options | sort | transpose name value | 
		each { |it| 
			match $it.name {
				"fPIC" => $"set\(CMAKE_POSITION_INDEPENDENT_CODE ($it.value)\)"
				"shared" => $"set\(BUILD_SHARED_LIBS ($it.value)\)"
				_ => $"set\(($it.name | str upcase) \"($it.value)\"\)"
			}
		})
	if ("{{OUTPUT_FILE}}" == '-') {
	  $options
	} else {
	  "message(WARNING \"Loading cmake-options\")\n" | save -f "{{OUTPUT_FILE}}"
	  $options | save -a "{{OUTPUT_FILE}}"
	}

[memsharded]

Hi @drmacdon

Thanks for your feedback

The CMake conan_provider.cmake approach as pointed out in #635 does not invoke conan until the first find_package(), but CMake processes it just after the project() line. I was not clear on why conan was deferred until the first find_package()?

This is how the CMake dependency providers work: https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html#dependency-providers, CMake designed them as interceptors of find_package() calls. Conan just followed their design.

But if conan was resolvable at the point that conan_provider.cmake was evaluated it could do something similar with far less dependencies and be more seamless enabling the CMake to remain entirely agnostic of conan.

I partially understand the desire to be able to keep 1 single command like cmake ... and have everything integrated from there. But at some point, if the added complexity to do this is very high, it is simply not worth it. Other programming languages have had flows such as npm install to install dependencies, then gulp build to do the actual build with a different tools for long time.

Hiding the package and dependency resolution is not always a good thing, from our experience. It often just "delays" the issues, overloading the "build" engineers everytime something doesn't work because the "build is broken" without understanding if it is a dependency new version, if it is a compiler error somewhere, or what.

The aim of "remain agnostic of conan" recalls me from devs using some abstractions over git when migrating from subversion (to not have to learn the complexity of git). Our experience is that we have seen in general better results in organizations that accept Conan as another new tool in the developers flow and "toolbelt" (because it really is a new tool in the flow) than organizations trying to completely hide it as a build internal implementation detail.

I understand that this is not always possible, and this is why we provide this cmake-conan integration, with a special focus on being able to provide a transparent integration without modifying the CMakeLists.txt, and following the recommended practices of the CMake dependency providers. But this doesn't allow to provide a multi-stage bidirectional flow of information between Conan and CMake, so cases like what you are trying to implement will be quite challenging.

[Talkless]

and following the recommended practices of the CMake dependency providers.

And these dependency providers cannot take any more information from within main CMakeLists.txt? EVERYTHING has to be provided via (manually written/generated?) profiles, -D's. and even these profiles (and many) more must be set in advanced via -DCONAN_FOO_BAR=..., i.e. no way from main CMakeLists.txt?

In 1.x, I just:

• open CMakeLists.txt with QtCreator
• set some option()'s via QtCreator GUI
• cmake is executed by QtCreator, it passes all needed args to cmake
• conan_cmake_run called in main CMakeLists.txt, gets bunch of args via various ${} (like path to conan venv itself, options for packages, etc), which might be taken from some "repo-global" include(../../../../[vars|common|...].cmake) or whatever.

...and that's it.

If I switch from Release to Debug in QtCreator, all runs again, automatically, and it works beautifully (had to do some hacks when cross-compiling, or building mobile Qt application, but it's doable).

Now with 2.0, by reading various issues here, it seems this usability I had with 1.x will go down the drain..?

It seems I will have to pass -DCONAN_FOO... every time I open new (without existing CMakeLists.txt.user) Qt project, change/generate (where/when?) profile with package options, and make sure I sync options with cmake's option()'s manually, and do it for Debug and Release builds individually..?

Or am I missing something, @memsharded ?

Thanks.

[jcar87]

Thanks @Talkless
Could you provide a specific example with actual code, even if minimal?

open CMakeLists.txt with QtCreator
set some option()'s via QtCreator GUI

Do you mean, editing CMakeLists.txt as a text file, and adding option statements, or editing the CMake variables passed to the build like described here?

cmake is executed by QtCreator, it passes all needed args to cmake
conan_cmake_run called in main CMakeLists.txt, gets bunch of args via various ${} (like path to conan venv itself, options for packages, etc), which might be taken from some "repo-global" include(../../../../[vars|common|...].cmake) or whatever.

The variables listed in the docs for cmake-conan:

• https://github.com/conan-io/cmake-conan?tab=readme-ov-file#customizing-conan-profiles
• https://github.com/conan-io/cmake-conan?tab=readme-ov-file#customizing-the-invocation-of-conan-install

can technically be defined in CMakeLists.txt as long as they are defined before the call to project()

The same experience as before can be achieved, the recommended way in CMake is to use CMakePresets, which are also supported by Qt creator:
https://doc.qt.io/qtcreator/creator-build-settings-cmake-presets.html

CMake presets can live alongside CMakeLists.txt.

I strongly suggest opening a new issue so that we can document this better if necessary, as this issue seems unrelated (it expects variables defined by the CMaketoolchain generator, which was never possible with the Conan 1.x integration either)

[tkhyn]

@Talkless it looks like you have a very similar problem to the one I had when trying to migrate to conan 2.0 - and that's the reason why I'm replying today as I subscribed to this issue at the time! - which is the impossibility to pass options defined in cmake to conan 2 with the approach used by cmake-conan v2.

To circumvent that limitation I ended up creating cmake-conanfile which can inject cmake variables in conanfile.py files before conan is run. Maybe it can help you?

[Talkless]

Do you mean, editing CMakeLists.txt as a text file, and adding option statements, or editing the CMake variables passed to the build like described here?

@jcar87 I have bunch of "options()'s" in CMakeLists.txt:

And then I can conveniently change them in QtCreator "Build settings" GUI:

The variables listed in the docs for cmake-conan:

With Conan 1.x, I have just:

conan_cmake_run(
    BUILD missing
    BUILD_TYPE "${STY_CONAN_BUILD_TYPE}"
    CONANFILE conanfile.txt
    CONAN_COMMAND "${STY_CONAN_COMMAND}"
    INSTALL_FOLDER "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"
    OPTIONS qt-solutions-service:qtversion=${STY_QT_VERSION}
    PROFILE "${STY_PROFILE_WINDOWS_OR_LINUX}"

STY_QT_VERSION (commercial pre-built Qt, specified globally for all repo/branch), STY_CONAN_COMMAND (from branch'es venv), STY_PROFILE_WINDOWS_OR_LINUX (somewhere in repo root dir) - all is taken via some include().

With 2.0 I will have to list them in "Additiona CMake options" like this, manually, every time project is newly launched (without QtCreator's CMakeLists.txt.user, like in a new branch).

CMake presets can live alongside CMakeLists.txt.

Yeah maybe presets could help somehow, I'll have to research them.

I strongly suggest opening a new issue so that we can document this better if necessary, as this issue seems unrelated

I agree, new issue would be best, with some minimal example project. I'll see if I can find time for that.

Thanks.

[Talkless]

To circumvent that limitation I ended up creating cmake-conanfile which can inject cmake variables in conanfile.py files before conan is run. Maybe it can help you?

Thanks for noting.

It is complete alternative to cmake-conan? And it passes only the options, right..? Does not solve need to specify more, like for example, INSTALL_FOLDER "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"?

[tkhyn]

Does not solve need to specify more, like for example, INSTALL_FOLDER "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}"?

Conan 2's conan install does not support the --install-folder argument so specifying the INSTALL_FOLDER this way won't work. However my understanding is that you can now use the layout and generate methods in conanfile.py, and cmake-conanfile can be used to inject cmake variables in these methods if needed. Regarding other conan install arguments, cmake-conanfile does not support all of them yet, but they can certainly be added if someone needs them!