Skip to content

Optimize build: Reduce compiler memory usage by introducing TypeList (Part 2) (backport #953) - #968

Open
mergify[bot] wants to merge 1 commit into
gz-physics7from
mergify/bp/gz-physics7/pr-953
Open

Optimize build: Reduce compiler memory usage by introducing TypeList (Part 2) (backport #953)#968
mergify[bot] wants to merge 1 commit into
gz-physics7from
mergify/bp/gz-physics7/pr-953

Conversation

@mergify

@mergify mergify Bot commented May 1, 2026

Copy link
Copy Markdown
Contributor

🎉 New feature

Needs #916

Summary

This commit introduces a lightweight TypeList to replace std::tuple for intermediate template metaprogramming operations (filtering, flattening, and deduping feature lists).

By using TypeList and C++17 fold expressions instead of recursive std::tuple instantiations, we drastically reduce compiler memory consumption (Max RSS) and build times. The public API retains std::tuple for backward compatibility, but internal operations are fully modernized.

Using GNU time, I measured a 33% peak RSS memory reduction on macOS and 68.9% reduction on Linux 🎉 (inside a VM with no swap).

Test it

Compare against main by building the code while measuring memory consumption

/usr/bin/time -v make gz-physics-dartsim-plugin

Results from Linux test

Before this change:

        Command being timed: "ninja -j1 gz-physics-dartsim-plugin"
        User time (seconds): 71.59
        System time (seconds): 7.72
        Percent of CPU this job got: 75%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 1:45.64
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 5472448
 

After this PR:

        Command being timed: "ninja -j1 gz-physics-dartsim-plugin"
        User time (seconds): 56.75
        System time (seconds): 6.98
        Percent of CPU this job got: 71%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 1:29.49
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1701576
 

Checklist

  • Signed all commits for DCO
  • Added a screen capture or video to the PR description that demonstrates the feature
  • Added tests
  • Added example and/or tutorial
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • Consider updating Python bindings (if the library has them)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • Updated Bazel files (if adding new files). Created an issue otherwise.
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers
  • Was GenAI used to generate this PR? If so, make sure to add "Generated-by" to your commits. (See this policy for more info.)

Generated-By: Gemini 3.0 Pro

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by and Generated-by messages.

Backports: If this is a backport, please use Rebase and Merge instead.


This is an automatic backport of pull request #917 done by Mergify.
This is an automatic backport of pull request #953 done by Mergify.

@mergify
mergify Bot requested review from azeey and scpeters as code owners May 1, 2026 15:12
@mergify mergify Bot added the conflicts label May 1, 2026
@mergify

mergify Bot commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

Cherry-pick of cc44170 has failed:

On branch mergify/bp/gz-physics7/pr-953
Your branch is up to date with 'origin/gz-physics7'.

You are currently cherry-picking commit cc44170.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   include/gz/physics/FeatureList.hh
	modified:   include/gz/physics/FindFeatures.hh
	modified:   include/gz/physics/RequestEngine.hh
	modified:   include/gz/physics/TemplateHelpers.hh
	modified:   include/gz/physics/detail/Entity.hh
	modified:   include/gz/physics/detail/FeatureList.hh
	modified:   include/gz/physics/detail/InspectFeatures.hh
	modified:   include/gz/physics/detail/Register.hh
	modified:   src/FeatureList_TEST.cc
	renamed:    src/FilterTuple_TEST.cc -> src/FilterTypeList_TEST.cc

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   include/gz/physics/detail/RegisterStatic.hh

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@github-actions github-actions Bot added the 🎵 harmonic Gazebo Harmonic label May 1, 2026
@azeey azeey changed the title Optimize build: Reduce compiler memory usage by introducing TypeList (Part 2) (backport #917) (backport #953) Optimize build: Reduce compiler memory usage by introducing TypeList (Part 2) (backport #953) May 1, 2026
@azeey
azeey force-pushed the mergify/bp/gz-physics7/pr-953 branch from 487735f to 56ba9c1 Compare May 1, 2026 17:10
@azeey

azeey commented May 1, 2026

Copy link
Copy Markdown
Contributor

Can I get a review on this since I fixed conflicts?

@github-project-automation github-project-automation Bot moved this from Inbox to In review in Core development May 1, 2026
…(Part 2) (#953)

* Optimize build: Reduce compiler memory usage by introducing TypeList (Part 2) (#917)

* Replace custom void_t with std::void_t

Replaced the legacy custom implementation of `void_t` in
`TemplateHelpers.hh` with `std::void_t` from `<type_traits>`,
fulfilling the existing TODO to migrate to standard C++17
metaprogramming types.

* Optimize InspectFeatures with C++17 fold expressions

Replaced the recursive template traversal in `InspectFeatures` with
flat C++17 fold expressions over the `FeatureTuple`. This significantly
reduces compile-time template instantiation depth and improves runtime
performance for feature verification.

Generated-By: Gemini 3.0 Pro
Signed-off-by: Addisu Z. Taddese <addisuzt@intrinsic.ai>

* Optimize compiler memory by introducing TypeList

This commit introduces a lightweight `TypeList` to replace `std::tuple`
for intermediate template metaprogramming operations (filtering, flattening,
and deduping feature lists).

By using `TypeList` and C++17 fold expressions instead of recursive
`std::tuple` instantiations, we drastically reduce compiler memory
consumption (Max RSS) and build times. The public API retains `std::tuple`
for backward compatibility, but internal operations are fully modernized.

* Tuple -> TypeList everywhere a TypeList expected rather than std::tuple
* Refactor intermediate metaprogramming to use TypeList instead of std::tuple

This change completes the transition from `std::tuple` to `TypeList` for
intermediate feature and entity metaprogramming operations. Utilizing `TypeList`
for internal metadata containers avoids the significant compiler memory overhead
associated with instantiating `std::tuple` and its associated logic.

The `FeatureList` API boundaries retain `std::tuple` to ensure backward
compatibility, but internal operations like `CombineLists`, `ExtractFeatures`,
and entity downcast verification now rely solely on `TypeList`. The helper
structs `ToTuple` and `TupleToTypeList` have also been moved to the `detail`
namespace to prevent global namespace pollution.

Generated-By: Gemini 3.0 Pro
Signed-off-by: Addisu Z. Taddese <addisuzt@intrinsic.ai>
(cherry picked from commit 1c4f296)

* Restore backward compatibility for UpcastIdentifiers

Reverted UpcastIdentifiers from TypeList to std::tuple in public macros
to maintain source compatibility. Added support for std::tuple to
TypeListContainsBase to handle both types.

Generated-By: Gemini 3.0 Pro
Signed-off-by: Addisu Z. Taddese <addisuzt@intrinsic.ai>

---------

Signed-off-by: Addisu Z. Taddese <addisuzt@intrinsic.ai>
Co-authored-by: Addisu Z. Taddese <addisuzt@intrinsic.ai>
Co-authored-by: Jose Luis Rivero <jrivero@honurobotics.com>
@luca-della-vedova
luca-della-vedova force-pushed the mergify/bp/gz-physics7/pr-953 branch from 56ba9c1 to 12f411b Compare July 1, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants