🚀 Replace vcpkg with conan #641

AnotherFoxGuy · 2023-09-24T14:28:32Z

This PR is still a work in progress, but the interface should built and run on both Windows and Linux

TODO

How to build on Windows with conan
Add the remote
conan remote add overte https://artifactory.overte.org/artifactory/api/conan/overte
Install the deps with conan
conan install . -b missing -pr:b=tools/conan-profiles/vs-19-release -pr=tools/conan-profiles/vs-19-release -of build
Generate the VS project files
cmake --preset conan-default
(Note: Currently only the Release build can be built)

How to build on Linux with conan
Let conan detect the build environment
conan profile detect
Add the remote
conan remote add overte https://artifactory.overte.org/artifactory/api/conan/overte
Install the deps with conan
conan install . -s build_type=Release -b missing -of build
Generate the make files
cmake --preset conan-default
Build the interface
cd build && make

JulianGro

A little comment on vhacd-util: As far as I know this is somewhat legacy. Our engine will automatically generate collision hulls and doesn't use vhacd-util for that. vhacd-util could be very useful for generating performant collision hulls for more complicated objects, but I don't think anyone does that. In fact, I have never even tried using it, and don't even know if it works.

JulianGro · 2023-09-30T11:49:22Z

conan-recipes/nvidia-texture-tools/all/conanfile.py

+        tc.variables["BUILD_TOOLS"] = "OFF"
+        tc.variables["USE_CUDA"] = "OFF"
+        if self.settings.os == "Linux":
+            tc.variables["CMAKE_CXX_FLAGS"] = "-march=msse3 -fPIC"


This would break on aarch64, which technically Overte can currently run on.

Apparently, the -march=msse3 just gets ignored on aarch64 without even throwing a warning.

cmake/macros/AutoScribeShader.cmake

cmake/macros/PackageLibrariesForDeployment.cmake

cmake/macros/TargetJson.cmake

CMakeLists.txt

conan-recipes/libnode/config.yml

JulianGro · 2023-09-30T12:20:27Z

conanfile.py

+        "qt*:qtxmlpatterns": "True",
+        "glad*:spec": "gl",
+        "glad*:gl_profile": "core",
+        "glad*:gl_version": "4.6",


Not sure if you know, but Overte supports multiple OpenGL versions. For example, it will fall back to OpenGL 4.1 if OpenGL 4.6 is not available. It can also be built for OpenGL ES 3.2.
I don't know how this part of the Conan code works though.

This is just the targeted version for glad, I just copied the settings from the vcpkg package

interface/CMakeLists.txt

JulianGro · 2023-09-30T12:25:00Z

tools/conan-profiles/vs-19-debug

@@ -0,0 +1,8 @@
+[settings]
+os=Windows
+arch=x86_64


We don't currently have any plans for an aarch64 Windows version, but would it make sense to have the profile name reflect the architecture?

JulianGro · 2023-09-30T12:35:03Z

tools/shadergen.py

@@ -217,7 +217,7 @@ def processCommand(line):
        executeSubprocess(scribeArgs)

        # Generate the un-optimized output
-        executeSubprocess([glslangExec, '-V110', '-o', upoptSpirvFile, unoptGlslFile])
+        executeSubprocess([glslangExec, '-V100', '-o', upoptSpirvFile, unoptGlslFile])


Could you explain what this change is doing? Looking it up online it seems like this switches to an older Visual Studio toolchain?

The glslangValidator from the conan package is too old to support that version, it needs to be updated

JulianGro · 2023-10-01T05:54:51Z

A comment on building this on Linux:
Since Conan is not available as a distribution package (at least not on Debian), but rather a pip package, the installation procedure is not that trivial.

# Install python3 virtual environment
sudo apt install python3-venv
# Create a virtual environment in the Overte codebase
python3 -m venv .venv
# Install pip dependencies
.venv/bin/pip3 install conan
# Now one can run conan
.venv/bin/conan profile detect

JulianGro · 2023-10-01T16:53:18Z

Am I understanding it right that Conan is supposed to only get dependencies of its own if the system doesn't already provide them?
If yes, we should probably always depend on a version range, from the oldest version that we know works, to the newest version that we know works. With libopus we would get version 1.3.1 from the system on Ubuntu 20.04.
Or does Conan "only" provide the tools for us to conditionally use system packages on our own?

AnotherFoxGuy · 2023-10-01T17:16:34Z

Am I understanding it right that Conan is supposed to only get dependencies of its own if the system doesn't already provide them?

That is really dependent on how the conan recipes are written.
The recipes provided by the conan-center are always using conan packages to make packaging easier and more reproducible.
But it is possible to script conan to install system packages, for example the xorg recipe just installs the appropriate system packages https://github.com/conan-io/conan-center-index/blob/master/recipes/xorg/all/conanfile.py#L26

JulianGro · 2023-10-02T08:30:31Z

It would be nice if you could put a README into the conan-recipes directory on how to consume those and how to update them.

It seems like I can build node for example using a command like

../../.venv/bin/conan create all/conanfile.py --version 18.17.1

JulianGro · 2023-10-02T21:41:03Z

Building libnode through conan fails on Ubuntu 20.04.
profile detect seems to have chosen C++14, while libnode seems to require C++17 or higher to build.
Log: https://bin.linux.pizza/?3ad49c3deaebf232#5NxEbsMUNagxtaMvnFJxm3dpWXTt2Ja5CJQrYio89wAR

JulianGro · 2023-10-07T18:33:22Z

Just tried on aarch64 postmarketOS. (I cannot select text or pipe it to file for some reason.)

It looks like it is trying to install gl through the Alpine Package manager, which is not a valid package name. Unfortunately, Conan Center is having issues, so I don't know how to take a look at the source code.

AnotherFoxGuy · 2023-10-07T18:58:14Z

I don't know how to take a look at the source code.

The source for the package is here:
https://github.com/conan-io/conan-center-index/blob/master/recipes/opengl/all/conanfile.py

JulianGro · 2023-10-07T20:14:48Z

Installing a bunch of packages fixed the gl issue.

conanfile.py

…stem OpenSSL package. (Crashpad depends on libcurl.)" This reverts commit dd51cb9.

JulianGro · 2025-03-07T10:06:34Z

conanfile.py

+        self.requires("artery-font-format/1.0.1")
+        self.requires("bullet3/3.25")
+        self.requires("cgltf/1.14@overte/stable")
+        # self.requires("crashpad/cci.20220219" ) # Broken


Since we barely use Crashpad, and it is a huge pain to build, I think we should just leave it disabled for now. We can take a look at it again once Conan is merged.

JulianGro · 2025-03-07T12:14:04Z

Building on Ubuntu 20.04 currently fails due to conan-io/conan-center-index#26806

AnotherFoxGuy force-pushed the conan branch from f10011e to c7a34e6 Compare September 24, 2023 15:02

This comment was marked as resolved.

Sign in to view

AnotherFoxGuy force-pushed the conan branch 2 times, most recently from 1f6bfe1 to 705a375 Compare September 27, 2023 11:55

AnotherFoxGuy marked this pull request as ready for review September 27, 2023 14:04

AnotherFoxGuy requested a review from JulianGro as a code owner September 27, 2023 14:04

JulianGro reviewed Sep 30, 2023

View reviewed changes

This comment was marked as resolved.

Sign in to view

This comment was marked as outdated.

Sign in to view

AnotherFoxGuy force-pushed the conan branch from 7c7d9d8 to e2fadc3 Compare October 2, 2023 21:18

JulianGro force-pushed the conan branch from 114c5d2 to 2bec5c7 Compare October 9, 2023 07:34

AnotherFoxGuy force-pushed the conan branch from 2bec5c7 to 752e0f2 Compare November 8, 2023 10:02

AnotherFoxGuy force-pushed the conan branch 3 times, most recently from becf99d to 8d93025 Compare November 24, 2023 09:16

daleglass mentioned this pull request Dec 4, 2023

Upgrade openexr #752

Merged

AnotherFoxGuy force-pushed the conan branch 2 times, most recently from e98cdb9 to c699930 Compare December 7, 2023 10:32

JulianGro reviewed Mar 7, 2025

View reviewed changes

conanfile.py Outdated Show resolved Hide resolved

JulianGro added 3 commits March 7, 2025 10:46

Reenable WebRTC on Windows, as it is kind of important.

7eb991c

Revert "Add hacked libcurl package which doesn't conflict with our sy…

63c4059

…stem OpenSSL package. (Crashpad depends on libcurl.)" This reverts commit dd51cb9.

Remove shaderc as it appears unused.

eb16e0e

JulianGro reviewed Mar 7, 2025

View reviewed changes

JulianGro added 4 commits March 7, 2025 11:18

Fix syntax error in PR builds GitHub Actions Workflow.

e4c56af

Work around OpenSSL/system sometimes not being found.

50750fe

Remove leftovers from Crashpad external_projects.

146b100

Build without Crashpad for now.

0ed0882

JulianGro and others added 10 commits March 7, 2025 13:34

Just use Ubuntu 22.04 instead of Ubuntu 20.04 on the CI for now.

e244c8e

Don't fail if Conan profile already exists.

92dd9fc

Add architecture to cache key.

b5c7ca6

Use system Qt on Ubuntu 22.04.

18dc20c

🐛 Fixed cache key

5d31f2e

Remove VCPKG references.

144cef7

Update libnode to allow building on Visual Studio 2022.

fe10d63

Get Qt from Conan on Windows.

d3139d2

Fix windeployqt not being found.

2f3dfee

Update build documentation for Windows.

aec1dab

This comment was marked as resolved.

Sign in to view

JulianGro added 2 commits March 20, 2025 00:23

Add missing include. Fixes building on Visual Studio 2022.

0b91b74

Remove VCPKG specific code.

b233d2b

This comment was marked as resolved.

Sign in to view

Try building on Windows 2022.

ee641aa

JulianGro force-pushed the conan branch 2 times, most recently from 6caebac to a2591ab Compare March 21, 2025 17:34

This comment was marked as resolved.

Sign in to view

Specify which build preset to use.

769ce03

JulianGro force-pushed the conan branch from a2591ab to 769ce03 Compare March 21, 2025 17:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚀 Replace vcpkg with conan #641

🚀 Replace vcpkg with conan #641

AnotherFoxGuy commented Sep 24, 2023 •

edited

Loading

This comment was marked as resolved.

JulianGro left a comment

JulianGro Sep 30, 2023

JulianGro Feb 20, 2025

JulianGro Sep 30, 2023

AnotherFoxGuy Sep 30, 2023

JulianGro Sep 30, 2023

JulianGro Sep 30, 2023

AnotherFoxGuy Sep 30, 2023

JulianGro commented Oct 1, 2023

This comment was marked as resolved.

This comment was marked as resolved.

JulianGro commented Oct 1, 2023

AnotherFoxGuy commented Oct 1, 2023

This comment was marked as resolved.

JulianGro commented Oct 2, 2023 •

edited

Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as outdated.

JulianGro commented Oct 2, 2023

JulianGro commented Oct 7, 2023

AnotherFoxGuy commented Oct 7, 2023

JulianGro commented Oct 7, 2023

JulianGro Mar 7, 2025

JulianGro commented Mar 7, 2025

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

🚀 Replace vcpkg with conan #641

Are you sure you want to change the base?

🚀 Replace vcpkg with conan #641

Conversation

AnotherFoxGuy commented Sep 24, 2023 • edited Loading

This comment was marked as resolved.

JulianGro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JulianGro commented Oct 1, 2023

This comment was marked as resolved.

This comment was marked as resolved.

JulianGro commented Oct 1, 2023

AnotherFoxGuy commented Oct 1, 2023

This comment was marked as resolved.

JulianGro commented Oct 2, 2023 • edited Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as outdated.

JulianGro commented Oct 2, 2023

JulianGro commented Oct 7, 2023

AnotherFoxGuy commented Oct 7, 2023

JulianGro commented Oct 7, 2023

Choose a reason for hiding this comment

JulianGro commented Mar 7, 2025

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

AnotherFoxGuy commented Sep 24, 2023 •

edited

Loading

JulianGro commented Oct 2, 2023 •

edited

Loading