Build Clojurescript web application

[storvik]

I'm currently working on deploying my Clojurescript web application using Nix and stumbled upon this project when looking around. As this is my first Clojure(script) project and I haven't used Java before everything is new to me atm, and adding the complexity of Nix on top of that does not seem to make things easier.

Typical Clojurescript project consists of some npm dependencies and some Clojure dependencies. When developing I usually run a REPL, Clojure fetches dependencies and everything is smoothly put together with shadow-cljs. In order to build the release version of the webapp shadow-cljs is invoked (before tailwind compiles the css). This works perfectly well in my Nix shell. However when trying to do the same inside mkDerivation it fails because of missing network connection.

This project seems to aim for a solution to this specific problem, but I cannot figure out how to use it or if it's really possible. I need to package my dependencies from my deps.edn and then use a custom jdk / Clojure which knows where to find my dependencies when invoking shadow-cljs. Do you think it's doable?

Thank you so much for any help, and for contributing to the Nix community!

[jlesquembre]

In theory, it should be possible, to combine clj-nix with node2nix, yarn2nix, or any other 2nix for the node ecosystem. Probably some changes will be required, not sure about it.
Right now I'm focused on JVM Clojure. I don't discard adding support for Clojurescript, but I don't have any concrete plans for it. While it would be for sure nice to support Clojurescript, it's not a priority for me.

[kenranunderscore]

@storvik I'm currently using clj-nix for something similar to this scenario, although I haven't tried out a REPL or similar yet. There is some work required, but the idea is the following:

• Use node2nix to generate nix lockfiles from an existing package-lock.json
• Use clj-nix to lock down your Clojure + ClojureScript dependencies in deps-lock.json
• In mkDerivation you may now use mk-deps-cache { lockfile = ./deps-lock.json; } to get the CLJ(S) dependencies without network access, and the node2nix-generated files to get the same for npm.
• Symlink these to your $(pwd) and then do the following:

export HOME=$(pwd)
export JAVA_TOOL_OPTIONS="-Duser.home=$HOME"
${clj-builder} --patch-git-sha $(pwd)
clj -T:build <something-you-want-to-do>

I think that in order for this to work it's necessary to invoke shadow-cljs directly from within your build.clj instead of via npx, but I'm not 100% sure about this. @neshtea might know more about this.

[neshtea]

@storvik I'm basically doing what @kenranunderscore described above. My goal was to not have to call any Javascript programs (npm, yarn, ...) during the build. shadow-cljs works quite well when used as a library during a build with tools.build, like this

(defn build-cljs
  "Buids a single cljs application with shadow-cljs.
  The `task` is a task for shadow-cljs.
  The `target` is a target key in the `shadow-cljs.edn` file."
  [{:keys [task target]}]
  (let [basis (b/create-basis {:project "deps.edn"
                               :aliases [:cljs]})
        cmds  (b/java-command {:basis     basis
                               :main      'clojure.main
                               :main-args ["-m" "shadow.cljs.devtools.cli" (str task) (str target)]})]
    (b/process cmds)))

with a deps.edn like this

{:deps
 {de.active-group/active-clojure {:mvn/version "0.41.0"}}

 :paths
 ["src"]

 :aliases
 {:build
  {:deps
   {io.github.clojure/tools.build
    {:git/tag "v0.8.2"
     :git/sha "ba1a2bf421838802e7bdefc541b41f57582e53b6"}}
   :ns-default build}

  :cljs
  {:extra-deps
   {thheller/shadow-cljs    {:mvn/version "2.19.0"}
    de.active-group/reacl-c {:mvn/version "0.10.8"}}
   :extra-paths
   ["test"]}}}

and a shadow-cljs.edn like this

{:deps {:aliases [:cljs]}

 :builds
 {:app
  {:target     :browser
   :output-dir "resources/public/search-client/js"
   :asset-path "/search-client/js"
   :modules    {:main {:entries [your.app.core]}}}}}

We use node2nix to lock the JavaScript dependencies from a package-lock.json and link that into the build before the clojure build is executed, like this:

# with-node-modules.nix Given some derivation `drv` and a
# `nodeDefault`, fetch the JavaScript deps and link them into `drv`
# at the start of the build.
{ pkgs, drv, nodeDefault}:
let
  # the default.nix is the result of calling node2nix
  nodeDependencies = (pkgs.callPackage nodeDefault {
    inherit pkgs;
    inherit (pkgs) nodejs;
  }).nodeDependencies;
in drv.overrideAttrs (old:
  old // {
    buildPhase = ''
      ln -s ${nodeDependencies}/lib/node_modules ./node_modules
    '' + old.buildPhase;
  })

When you run your build (in this case, something like clojure -T:build build-cljs :task release :target app), you instantiate shadow-cljs with all your dependencies already on the classpath. To use clj-nix in it's current state, you only need to wrap it in a dummy jar in which you copy the generated JavaScript and extract it later in you build when you need it.

This is the pipeline I've put together in the last few weeks and I'm sure we could quite easily wrap this in a nix function to make it easily available (if @jlesquembre is okay with it, of course).

[jlesquembre]

This is the pipeline I've put together in the last few weeks and I'm sure we could quite easily wrap this in a nix function to make it easily available (if @jlesquembre is okay with it, of course).

Ok with it, I'd like to add such a nix function ;)

@neshtea Thanks for sharing your approach. I don't have too much time, but I'll try to create a dummy shadow-cljs to play with it.

In the meantime, a PR for it is very welcome

[neshtea]

@jlesquembre

In the meantime, a PR for it is very welcome

If we want to try something like my setup, it would mean to include node2nix in clj-nix. Would this be a problem for you? The other option would be to make a new flake specifically for ClojureScript and base that on clj-nix. Both have pros and cons, and I'm not sure which way is better.

[jlesquembre]

@neshtea IMO it is ok to add a dependency to node2nix. node2nix doesn't provide a flake yet, but it's included in nixpkgs, we could use the nixpkgs version since anyways we already depend on it.

The other option would be to make a new flake specifically for ClojureScript and base that on clj-nix

I don't like that option too much. While it could have some advantages, I think splitting it into 2 flakes will potentially make it more confusing to the users. I prefer to have it in one flake. One of my goals is to make clj-nix easy to use.

One last thought, to support cljs we must add a dependency to one of the npm lockers (node2nix, yarn2nix, npmlock2nix, ...), I don't think there is any other solution. Ideally, we should be agnostic to the npm locker, but I fear that it's not possible, or, in the best case, it will be a lot of extra work. For that reason, I think we should just pick one. node2nix locks mature, I'll go with it.

[storvik]

Thank you all for pointing me in the right direction. Managed to build my project using the helper function mk-deps-cache. However I'm not using node2nix, but settled on nix-npm-buildpackage.

You can decide if you want to close this issue or keep it open to further for further discussion.

[johannesloetzsch]

@jlesquembre Thanks for this wonderful project :) In past I used mvn2nix, but clj-nix is much nicer 👍

You wrote

node2nix doesn't provide a flake yet

Actually there is one: https://github.com/svanderburg/node2nix/blob/master/flake.nix

I had been building clojurescript and nodejs projects with node2nix and yarn2nix. Till now some manual effort and boilerplate is required.
I would love to create better tooling for easy building of shadow-cljs projects. When I attempted to do so last time, I realized that the uberjar of shadow-cljs is not correctly building. It could be easily fixed, but I wasn't able to convince thheller to accept the fix: thheller/shadow-cljs#823

Would be nice to package shadow-cljs with clj-nix and include it in nixpkgs. A first quick trial resulted in the error:

"Maven repo not found"

I could try to debug it, but maybe one of you knows how to solve it already?

[bendlas]

This would be easier if clj-nix could be hooked into other builds: #69