Improve VSAG's compatibility with macOS #1439

stuBirdFly · 2025-12-18T06:07:48Z

This MR introduces comprehensive macOS compatibility to the vsag project, enabling successful builds and tests on Apple Silicon (arm64) Macs.

🔧 Key Changes:
Fixed linker issues:
Replaced Linux-specific --whole-archive with macOS-compatible -force_load (or removed where unnecessary).
Resolved duplicate symbol errors from fmt by ensuring consistent static-library usage (removed accidental header-only instantiations).
Resolved missing LAPACKE dependency:
On macOS, linked against the system-provided Accelerate framework instead of liblapacke, which is not available by default.
Updated CMake logic to auto-select Accelerate on APPLE.
Improved compiler detection:
Updated Clang detection to include AppleClang (CMake ID is AppleClang, not Clang).
Ensured correct debug flags (-gdwarf-4) and warning suppressions work on both LLVM Clang and Apple Clang.
Fixed ExternalProject integration:
Adjusted HDF5 and other dependencies to avoid premature include_directories/link_directories that cause “search path not found” warnings on macOS.
Cleaned up duplicate library linking:
Removed redundant static library references (e.g., fmt, roaring, io) to eliminate linker warnings.

gemini-code-assist · 2025-12-18T06:13:17Z

Summary of Changes

Hello @stuBirdFly, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the VSAG project's compatibility with macOS, particularly for Apple Silicon. It addresses various build and linking issues specific to the macOS environment, ensuring that the project can be successfully compiled and tested on Apple's hardware. The changes involve adapting build configurations, resolving library dependencies, and updating type usage across the codebase to align with macOS conventions and prevent compilation errors.

Highlights

macOS Compatibility: Comprehensive changes have been implemented to ensure VSAG builds and tests successfully on macOS, including Apple Silicon (arm64) architectures.
Linker and Library Resolution: Linux-specific linker flags like --whole-archive have been replaced with macOS-compatible -force_load or removed. Duplicate symbol errors from fmt were resolved by ensuring consistent static-library usage. The missing LAPACKE dependency on macOS now links against the system-provided Accelerate framework.
CMake Logic and Compiler Detection: CMake logic has been updated to auto-select the Accelerate framework on Apple platforms and to correctly detect AppleClang. Conditional compilation and linking options were added for libc++ and libstdc++, and rpath handling was adjusted for macOS.
Type Consistency and External Projects: Numerous size_t types in DiskANN applications and headers have been updated to uint64_t for better consistency and compatibility. Adjustments were made to ExternalProject integration for HDF5 and other dependencies to prevent 'search path not found' warnings on macOS.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This is a comprehensive pull request that significantly improves macOS compatibility for the project. The systematic refactoring from size_t to uint64_t across the codebase is a major step towards better cross-platform type safety and is well-executed. The CMake build system updates and platform-specific API usage for macOS are also appreciated.

I have two main points of feedback on the CMakeLists.txt changes. My primary concern is the introduction of a hard dependency on Homebrew for finding libraries, which could make the build process less portable for developers using other setups. I've also noted a minor issue with a comment in Chinese that could be translated for consistency. Addressing these points would make this excellent contribution even more robust.

gemini-code-assist · 2025-12-18T06:17:10Z

CMakeLists.txt

+if (APPLE)
+    execute_process(
+        COMMAND brew --prefix gcc
+        OUTPUT_VARIABLE GCC_PREFIX
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET
+    )
+    if (GCC_PREFIX)
+        link_directories(${GCC_PREFIX}/lib/gcc/current)
+        message(STATUS "Added gfortran library path: ${GCC_PREFIX}/lib/gcc/current")
+    endif()
+
+    execute_process(
+        COMMAND brew --prefix libomp
+        OUTPUT_VARIABLE OMP_PREFIX
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET
+    )
+    if (OMP_PREFIX)
+        link_directories(${OMP_PREFIX}/lib)
+        include_directories(${OMP_PREFIX}/include)
+        message(STATUS "Added libomp library path: ${OMP_PREFIX}/lib")
+    endif()
+
+    execute_process(
+        COMMAND brew --prefix lapack
+        OUTPUT_VARIABLE LAPACK_PREFIX
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET
+    )
+    if (LAPACK_PREFIX)
+        link_directories(${LAPACK_PREFIX}/lib)
+        include_directories(${LAPACK_PREFIX}/include)
+        message(STATUS "Added LAPACK library path: ${LAPACK_PREFIX}/lib")
+    endif()
+endif()


This block introduces a hard dependency on Homebrew by using execute_process with brew. This can make the build process brittle for developers who use other package managers (like MacPorts) or have dependencies installed in different locations. Additionally, the use of link_directories is generally discouraged in modern CMake.

A more robust and portable approach would be to use find_package. For example:

find_package(OpenMP) can locate libomp.

find_package(LAPACK) will find the Accelerate framework on macOS, which aligns with the goal stated in the PR description.

This would make the build system more standard and less dependent on a specific local setup.

gemini-code-assist · 2025-12-18T06:17:10Z

CMakeLists.txt


-vsag_add_exe_linker_flag (-static-libstdc++)
-vsag_add_shared_linker_flag (-static-libstdc++)
+# Apple 平台使用系统 libc++，不要静态链接 libstdc++


For consistency with the rest of the comments in the project, which are in English, it would be great to translate this comment to English. This will improve maintainability for all contributors.

# On Apple platforms, use the system's libc++ instead of static linking libstdc++

codecov · 2025-12-18T09:15:07Z

Codecov Report

❌ Patch coverage is 96.26556% with 9 lines in your changes missing coverage. Please review.

@@            Coverage Diff             @@
##             main    #1439      +/-   ##
==========================================
+ Coverage   91.33%   91.35%   +0.02%     
==========================================
  Files         326      326              
  Lines       19121    19121              
==========================================
+ Hits        17464    17468       +4     
+ Misses       1657     1653       -4

Flag	Coverage Δ
cpp	`91.35% <96.26%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
common	`85.66% <100.00%> (ø)`
datacell	`93.59% <89.47%> (+0.13%)`	⬆️
index	`91.07% <94.96%> (+0.03%)`	⬆️
simd	`100.00% <ø> (ø)`

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1830648...5deef9a. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

LHT129 · 2026-01-04T07:32:08Z

src/io/mmap_io.cpp

+
+#ifdef __APPLE__
+        // macOS 没有 mremap，先取消旧映射再重新 mmap
+        munmap(this->start_, old_size);


wxyucs · 2026-01-04T07:07:36Z

extern/diskann/DiskANN/rust/diskann/src/utils/bit_vec_extension.rs


    #[test]
-    fn resize_test() {
+    fn reuint64_test() {


revert this change

wxyucs · 2026-01-04T07:07:48Z

extern/diskann/DiskANN/rust/diskann/src/utils/kmeans.rs

 /// Run Lloyds until max_reps or stopping criterion
 /// If you pass NULL for closest_docs and closest_center, it will NOT return
 /// the results, else it will assume appropriate allocation as closest_docs =
-/// new vec<usize> [num_centers], and closest_center = new size_t[num_points]


revert this change

wxyucs · 2026-01-04T07:08:09Z

extern/diskann/DiskANN/rust/vector/distance.c

 }

-float distance_compare_avx512f_f16(const unsigned char *vec1, const unsigned char *vec2, size_t size)
+float distance_compare_avx512f_f16(const unsigned char *vec1, const unsigned char *vec2, uint64_t size)


revert changes in this file

wxyucs · 2026-01-04T07:08:53Z

extern/diskann/DiskANN/src/restapi/search_wrapper.cpp

    : BaseSearch(tagsFile)
 {
-    size_t dimensions, total_points = 0;
+    uint64_t dimensions, total_points = 0;


revert changes in the restapi/ directory

Changing the type will cause a build failure.

wxyucs · 2026-01-04T07:09:09Z

extern/diskann/DiskANN/src/restapi/server.cpp

        auto best_tags = results[0].tags_enabled() ? new std::string[K] : nullptr;

        auto numsearchers = _multi_searcher.size();
-        std::vector<size_t> pos(numsearchers, 0);


revert changes in the restapi/ directory

wxyucs · 2026-01-04T07:25:14Z

src/storage/stream_reader.h

 private:
    StreamReader* const reader_impl_{nullptr};
    vsag::Allocator* allocator_;
    char* buffer_{nullptr};    // Stores the cached content


align the comment

wxyucs · 2026-01-04T07:30:46Z

tests/CMakeLists.txt

+        target_link_libraries (unittests
+                        PRIVATE
+                        Catch2::Catch2 
+                        "-Wl,--whole-archive"


this may cause the unittests file to be large?

Mainly because macOS doesn't support --whole-archive.

wxyucs · 2026-01-04T07:31:41Z

tests/test_diskann.cpp

 // limitations under the License.

-#include <fmt/format-inl.h>
+#include <fmt/format.h>


what is the reason to use the header file fmt/format.h

The macOS linker is stricter—inline functions included in multiple .cpp files will cause duplicate symbol errors.

wxyucs · 2026-01-04T07:32:21Z

CMakeLists.txt


-vsag_add_exe_linker_flag (-static-libstdc++)
-vsag_add_shared_linker_flag (-static-libstdc++)
+# Apple 平台使用系统 libc++，不要静态链接 libstdc++


comment in English

wxyucs · 2026-01-04T07:39:02Z

CMakeLists.txt

+# Add gfortran, OpenMP, and LAPACK library paths for macOS
+if (APPLE)
+    execute_process(
+        COMMAND brew --prefix gcc
+        OUTPUT_VARIABLE GCC_PREFIX
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET
+    )
+    if (GCC_PREFIX)
+        link_directories(${GCC_PREFIX}/lib/gcc/current)
+        message(STATUS "Added gfortran library path: ${GCC_PREFIX}/lib/gcc/current")
+    endif()
+
+    execute_process(
+        COMMAND brew --prefix libomp
+        OUTPUT_VARIABLE OMP_PREFIX
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET
+    )
+    if (OMP_PREFIX)
+        link_directories(${OMP_PREFIX}/lib)
+        include_directories(${OMP_PREFIX}/include)
+        message(STATUS "Added libomp library path: ${OMP_PREFIX}/lib")
+    endif()
+
+    execute_process(
+        COMMAND brew --prefix lapack
+        OUTPUT_VARIABLE LAPACK_PREFIX
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET
+    )
+    if (LAPACK_PREFIX)
+        link_directories(${LAPACK_PREFIX}/lib)
+        include_directories(${LAPACK_PREFIX}/include)
+        message(STATUS "Added LAPACK library path: ${LAPACK_PREFIX}/lib")
+    endif()
+endif()


this part should move to cmake/ directory

wxyucs · 2026-01-04T07:40:40Z

@stuBirdFly please follow this guide to fix the DCO check error: https://github.com/antgroup/vsag/blob/main/CONTRIBUTING.md#developer-certificate-of-origin-dco

Signed-off-by: stuBirdFly <[email protected]>

stuBirdFly requested review from LHT129, ShawnShawnYou, inabao, jiaweizone and wxyucs as code owners December 18, 2025 06:07

pull-request-size bot added the size/XXL label Dec 18, 2025

mergify bot added module/example module/thirdparty module/api module/attr module/datacell module/index module/simd module/tools module/testing labels Dec 18, 2025

gemini-code-assist bot reviewed Dec 18, 2025

View reviewed changes

wxyucs self-assigned this Dec 18, 2025

wxyucs added the version/0.18 label Dec 18, 2025

LHT129 mentioned this pull request Dec 25, 2025

refactor(size_t): use uint64_t instead size_t #1451

Closed

LHT129 reviewed Jan 4, 2026

View reviewed changes

wxyucs reviewed Jan 4, 2026

View reviewed changes

stuBirdFly force-pushed the compat_mac branch 3 times, most recently from a0a21cb to 76f0d62 Compare January 8, 2026 02:35

Improve VSAG's compatibility with macOS

06c6b06

Signed-off-by: stuBirdFly <[email protected]>

stuBirdFly force-pushed the compat_mac branch from 76f0d62 to 06c6b06 Compare January 8, 2026 06:11

merge master branch

5deef9a

Signed-off-by: stuBirdFly <[email protected]>

Improve VSAG's compatibility with macOS #1439

Are you sure you want to change the base?

Improve VSAG's compatibility with macOS #1439

Uh oh!

Conversation

stuBirdFly commented Dec 18, 2025

Uh oh!

gemini-code-assist bot commented Dec 18, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wxyucs commented Jan 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

codecov bot commented Dec 18, 2025 •

edited

Loading