nix: windows build #6167
nix: windows build #6167
Conversation
.devops/nix/package.nix
Outdated
| mv $out/bin/main${executableSuffix} $out/bin/llama | ||
| mv $out/bin/server${executableSuffix} $out/bin/llama-server |
There was a problem hiding this comment.
This has been brought up before, it's really awkward we maintain a custom intsallPhase: the cmakelists need to be updated
(out of scope for this PR)
There was a problem hiding this comment.
❯ nix run nixpkgs#wine64 -- "$(nix build github:ggerganov/llama.cpp/553aafca25ed6aa3721b3e0bfa121eed3f2b7b58#windows --print-out-paths)"/bin/llama-bench.exe
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
0084:fixme:hid:handle_IRP_MN_QUERY_ID Unhandled type 00000005
| model | size | params | backend | threads | test |
t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ---------: | ---------- | ----
-----------: |
main: error: failed to load model 'models/7B/ggml-model-q4_0.gguf'
I guess this works
SomeoneSerge
added
the
nix
flake.nix
Outdated
| windows = config.legacyPackages.llamaPackages.llama-cpp.override { | ||
| stdenv = pkgs.pkgsCross.mingwW64.stdenv; | ||
| }; |
There was a problem hiding this comment.
Oh wait! Actually, I'm surprised this worked. I never touched windows, but shouldn't we be taking the other dependencies from pkgsCross as well?
There was a problem hiding this comment.
Which would look something like (pkgsCross.mingwW64.callPackage .devops/nix/scope.nix { }).llama-cpp
There was a problem hiding this comment.
From my experimentation the only parts that are needed to cross compile for Windows is the MinGW executable and the libraries it comes with which is all in stdenv. I did consider using pkgsCross.callPackage instead of overwriting stdenv but wouldn't that mean that we cannot compile for windows and CUDA/ROCM at the same time, since we use pkgsCuda.callPackage and pkgsRocm.callPackage respectively?
There was a problem hiding this comment.
but wouldn't that mean that we cannot compile for windows and CUDA/ROCM since we use pkgsCuda.callPackage and pkgsRocm.callPackage respectively?
Being able to construct a configuration with cuda+rocm and exposing a flake output for that configuration are two different things. The flake outputs are there mostly for show. The actual "useful" program is the package.nix (and the scope.nix) which are functions that you can call from arbitrary instances of Nixpkgs.
Those can be consumed e.g. as overlays
There was a problem hiding this comment.
But also linking the build platform's (linux's) CUDA libraries to a mingw executable would obviously fail
There was a problem hiding this comment.
Yes I've tried doing this while using pkgs.pkgsCross.mingwW64.callPackage but I'm getting problems with packages that complain about the platform now. This will take some time to get working in the right way but I'm on it.
There was a problem hiding this comment.
(I'm also surprised we didn't have to use makeScopeWithSplicing)
There was a problem hiding this comment.
Alright after quite a bit of testing it seems that the best solution is to override stdenv - I've tested the binaries on a Windows system and they all seems to run correctly 👍
I think this should be ready to merge. Let me know if you think anything is missing.
There was a problem hiding this comment.
Hi, check out the patches I just pushed. The only thing that was broken in the cross-compilation set up was python
There was a problem hiding this comment.
You did it! I tried to use pkgs.pkgsCross.mingwW64.callPackage but I kept getting that python error - which you've fixed now. I can even compile for Vulkan and it works on our Windows machine.
initial nix build for windows using zig mingwW64 build removes nix zig windows build removes nix zig windows build removed unnessesary glibc.static removed unnessesary import of pkgs in nix fixed missing trailing newline on non-windows nix builds overriding stdenv when building for crosscompiling to windows in nix better variables when crosscompiling windows in nix cross compile windows on macos removed trailing whitespace remove unnessesary overwrite of "CMAKE_SYSTEM_NAME" in nix windows build nix: keep file extension when copying result files during cross compile for windows nix: better checking for file extensions when using MinGW nix: using hostPlatform instead of targetPlatform when cross compiling for Windows using hostPlatform.extensions.executable to extract executable format
- The generic /usr/bin/env shebangs are good enough - Python deps are provisioned in the devShells - We need to be able to leave python out at least on windows (currently breaks eval)
Take all dependencies from the cross stage, rather tha only stdenv
SomeoneSerge
force-pushed
the
nix-windows
branch
from
97b2410
to
23577d0
Compare
|
@hutli I squashed and force-pushed your commits because there were some git conflicts with the master branch. I imagine you still have the commits with your original experiments (zig&c) locally if that matters |
Amazing, thank you! Yes, I have it locally and I'll save it just in case, thank you for the heads up 👍 |
.devops/nix/package.nix
Outdated
| @@ -200,7 +200,7 @@ effectiveStdenv.mkDerivation ( | |||
| ++ optionals useMpi [ mpi ] | |||
| ++ optionals useOpenCL [ clblast ] | |||
| ++ optionals useRocm rocmBuildInputs | |||
| ++ optionals useBlas [ blas ] | |||
| ++ optionals (useBlas && builtins.elem effectiveStdenv.hostPlatform.system blas.meta.platforms) [ blas ] | |||
There was a problem hiding this comment.
I think this is lib.meta.availableOn but first say, does this really break in .#windows?
.devops/nix/package.nix
Outdated
| @@ -200,7 +200,7 @@ effectiveStdenv.mkDerivation ( | |||
| ++ optionals useMpi [ mpi ] | |||
| ++ optionals useOpenCL [ clblast ] | |||
| ++ optionals useRocm rocmBuildInputs | |||
| ++ optionals useBlas [ blas ] | |||
| ++ optionals (useBlas && blas.meta.available) [ blas ] | |||
There was a problem hiding this comment.
Also I think we should instead && this in
llama.cpp/.devops/nix/package.nix
Lines 21 to 27 in ae31660
Rationale: user-supplied useBlas should have the priority, but we want to choose a different default for useBlas in the cross-compilation setting
|
The failure is from #6346. Wfm locally, also the eval job succeeds both on mac and ubuntu, so I'm guessing this is ready |
Fixes correct file extensien when using pkgsCross stdenv for crossCompiling for windows using mingwW64 inside Nix.