Skip to content

Build Arrow with support for mimalloc and jemalloc #540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 7, 2025
Merged

Build Arrow with support for mimalloc and jemalloc #540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a78ceef
Build Arrow with support for mimalloc and jemalloc
adamreeve Jul 7, 2025
38e0c25
Don't use jemalloc on Windows or arm64
adamreeve Jul 7, 2025
3c3ac21
Don't build mimalloc when cross-compiling to arm64 as this doesn't work
adamreeve Jul 7, 2025
503702f
Fix MemoryPool tests
adamreeve Jul 7, 2025
e5ef505
Bump version
adamreeve Jul 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions csharp.test/TestMemoryPool.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ public static void TestDefaultMemoryPool()
var pool = MemoryPool.GetDefaultMemoryPool();

Assert.AreEqual(0, pool.BytesAllocated);
Assert.Greater(pool.MaxMemory, 0);
Copy link
Contributor Author

@adamreeve adamreeve Jul 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was incorrect even with the system allocator and only passed when other tests had been run previously and allocated memory.

Assert.AreEqual("system", pool.BackendName);
Assert.IsNotEmpty(pool.BackendName);

using (var buffer = new ResizableBuffer())
{
Expand Down
2 changes: 1 addition & 1 deletion csharp/ParquetSharp.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<NoWarn>1591;</NoWarn>
<VersionPrefix>20.0.0-beta1</VersionPrefix>
<VersionPrefix>20.0.0-beta2</VersionPrefix>
<Company>G-Research</Company>
<Authors>G-Research</Authors>
<Product>ParquetSharp</Product>
Expand Down
18 changes: 15 additions & 3 deletions vcpkg.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,26 @@
"$schema": "https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json",
"name": "parquetsharp",
"version-string": "undefined",
"builtin-baseline": "d9ccd77bb554e67137f3f754a2e2f11f4188c82c",
"builtin-baseline": "e08b7bd89ae162f8579df2f8d39a1ae94107c8fd",
Copy link
Contributor Author

@adamreeve adamreeve Jul 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed to bump the baseline to pick up this fix that allows using mimalloc on non-Windows: microsoft/vcpkg#46299

"dependencies": [
"arrow"
{
"name": "arrow",
"features": [
{
"name": "mimalloc",
"platform": "linux | !arm64"
},
{
"name": "jemalloc",
"platform": "!windows&!arm64"
}
]
}
],
"overrides": [
{
"name": "arrow",
"version": "20.0.0"
"version": "20.0.0#1"
}
]
}