Replies: 33 comments 96 replies
-
Interceptor model twist by @helins: https://gist.github.com/helins/de21872849d4149282dbcfeec49a41bf Interceptor model by @stevebuik: (defn run!
[{:keys [verbose]} steps]
(let [all-results (atom {})]
(doseq [step steps]
(let [{:keys [action success?]} step
result (action @all-results)]
(println)
(println ">>" (:name step))
(when verbose
(pprint result))
(if (success? result)
(swap! all-results assoc (:name step) result)
(throw (ex-info "action failed" result)))))
(println)
(println "Success!")))
(utils/run! options
[{:name :get-tag
:action utils/step-get-git-tag
:success? (fn [result]
(not (nil? (:latest-commit-tag result))))}
{:name :get-sha
:action utils/step-get-git-sha
:success? (constantly true)}
{:name :compress-main-js
:action (fn [_]
(utils/compress-file! "./public/client-js" "main.js"))
:success? (fn [result]
(> (:size-compressed result) 0))}
{:name :deploy-editor-js
:action (fn [results]
(let [sha (get-in results [:get-sha :sha])]
(utils/deploy-compressed-asset-to-s3!
{:region bucket-region
:content-type "application/javascript"}
"main.js.gz"
(str "/editor/main." sha ".js"))))
:success? (fn [result]
(nil? (:Error result)))}
]) |
Beta Was this translation helpful? Give feedback.
-
Mach took the idea of make / Makefile and made it into a ClojurreScript version: |
Beta Was this translation helpful? Give feedback.
-
Of course we are reaching the realm of personal opinion but what does the "code" version truly provide over the EDN "declarative" approach which allows for calling custom BB scripts? |
Beta Was this translation helpful? Give feedback.
-
The thing is, there is no such thing as a "general-purpose" build tool / task runner. It pays out to think about what are 95% of use-cases and target that instead of trying to be unnecessarily perfect.
By providing a way of running tasks in a sequence with a fail fast mechanism (usually, building anything should stop if anything fails), we are already covering much ground. Anything needing more complex logic can be encapsulated in a BB script (or whatever) anyways which would do heavy work, call several functions, maintain some state in-between... Otherwise, tasks should be fairly opaque units of work.
What would set apart a BB-based task runner from anything else and all those Make alternatives? It is essentially Clojure, it can speak EDN, and EDN is perfect for describing a build process. It is searchable, CI-friendly while remaining human-friendly. It could rather easily build an EDN report listing everything that happened, pinpointing the exact moment of failure. When calling a BB script task, that task could itself return EDN akin to a rich exit code, adding valuable information to that report. That is a life worth living. Even when calling a bash job, we could specify EDN data to select depending on the exit code.
Such ideas remain purely declarative and in my opinion already provide much more than common options. In other words, I believe there are more exciting things to do in an "EDN approach" before dreaming too big with a "code approach".
|
Beta Was this translation helpful? Give feedback.
-
Currently this is possible in the bb.edn branch: :never-failing-shell
{:task/type :babashka
;; we are executing a shell task, yet we are continuing despite the error code
:args [-e (-> (do (deref (babashka.process/process ["ls" "foobar"] {:inherit true}))
(deref (babashka.process/process ["echo" "done"] {:inherit true}))
nil))]}
:print-hello {:task/type :babashka
:args [-e (println :hello)]}
:always-failing-shell
{:task/type :babashka
;; we are executing a shell task, yet we are continueing despite the error code
:args [-e (babashka.process/check (babashka.process/process ["ls" "foobar"] {:inherit true}))]}
:composed-never-failing {:task/type :babashka
:args [:do :never-failing-shell
:__ :print-hello]}
:composed-failing {:task/type :babashka
:args [:do :always-failing-shell
:__ :print-hello]} So Machfile-like execution is simply done using $ BABASHKA_EDN=test-resources/bb.edn bb :composed-never-failing
ls: foobar: No such file or directory
done
:hello $ BABASHKA_EDN=test-resources/bb.edn bb :composed-failing
ls: foobar: No such file or directory
----- Error --------------------------------------------------------------------
Type: clojure.lang.ExceptionInfo
Message:
Location: <expr>:1:1
----- Context ------------------------------------------------------------------
1: (babashka.process/check (babashka.process/process ["ls" "foobar"] {:inherit true}))
^---
----- Stack trace --------------------------------------------------------------
user - <expr>:1:1 |
Beta Was this translation helpful? Give feedback.
-
Hmmm, improvising as well I would prefer something more decoupled. Basic tasks without any dependencies between them: :find-java
{:task/type :shell
:args ["which" "java"]}
:print-java-version
{:task/type :shell
:args [:path "-version"]}
;; Notice the `:path` keyword "input" And now, a task which is actually a runner: {:task/type :runner
:tasks [:find-java
[:print-java-version {:pipe {:find-java :path}}]]} A task always output data to the context, the key being its name. Doing so, tasks are declared in an entirely decoupled fashion. We couple them only when composing them. We could even reuse And we can easily compose several |
Beta Was this translation helpful? Give feedback.
-
I guess we can start with the uncontroversial bits of |
Beta Was this translation helpful? Give feedback.
-
Interesting blog post by JUXT on how they are using bb + make: |
Beta Was this translation helpful? Give feedback.
-
I updated #756 with a problem statement and a proposed solution. Feedback is welcome here. |
Beta Was this translation helpful? Give feedback.
-
Awesome feature! |
Beta Was this translation helpful? Give feedback.
-
@borkdude I just read your updates to #756. A pleasure to read - so succinct and clear! I don't know if you need to answer these questions in your overview, but I wondered (with my naive reader hat on):
And to capture some highlights of our brainstorming session (not suggestions for initial release):
|
Beta Was this translation helpful? Give feedback.
-
(UX issue) Better feedback when user provides an unexisting task
Because of the use of a colon I would expect
|
Beta Was this translation helpful? Give feedback.
-
(UX issue) Better feedback when user provides invalid task format{:tasks {:foo [:task/type :babashka
:task/description "adf"
:task/args [-e (println "Hi")]]}} When listing tasks
When invoking
Expectation would be to get some feedback on the invalid format here. Maybe a data validation error ("not a map"). Especially in the beginning this might be confusing. |
Beta Was this translation helpful? Give feedback.
-
Alternative to the use of
|
Beta Was this translation helpful? Give feedback.
-
One other issue: You write a lib, like an http server and you write a bb.edn for it, so you can easily invoke it with bb. I'm considering something like https://shadow-cljs.github.io/docs/UsersGuide.html#deps-edn So libs can have a deps.edn + a bb.edn runner which merges in the settings from deps.edn automatically. |
Beta Was this translation helpful? Give feedback.
-
Hiccup/malli style syntax, less verbose, maybe better: {:vars {:ci #eval (System/getenv "CI")
:artifact "out/foo.jar"
,}
:tasks
{:test [:do
[:shell "clojure" "-M:outdated:nop:1.8" "--exclude=clojure/brew-install"]
[:shell "clojure" "-M:dev:1.9:test"]
[:shell "clojure" "-M:dev:test"]]
:clean [:shell {:pre (do (println "Going to clean up") true)
:post true} ;; always return true, regardless if the shell command succeeded
"rm" "-rf" ".cpcache" "target"]
:deploy [:do
{:depends [:test :clean]}
[:shell {:pre (not #ref [:vars :ci])} "echo" "Deploying"]
[:shell "clojure" -X:deploy :installer :remote :artifact #ref [:vars :artifact]]]}} |
Beta Was this translation helpful? Give feedback.
-
Rewrote the code in the bb.edn branch and documentation in #756 to hiccup-style syntax. {:paths ["test-resources/bb-edn"]
:deps {medley/medley {:mvn/version "1.3.0"}}
:tasks {:shell1 ^{:description "Echo hello"}
[:shell "echo" "hello"]
:shell2 ^{:description "Echo bye"}
[:shell "echo" "bye"]
:medley ^{:description "Show off medley"}
[:babashka -e (do (require (quote [medley.core :as m]))
(m/index-by :id [{:id 1} {:id 2}]))]}
,} This is especially convenient for inline tasks: [:do
[:shell "clojure" "-M:outdated:nop:1.8" "--exclude=clojure/brew-install"]
[:shell "clojure" "-M:dev:1.9:test"]
[:shell "clojure" "-M:dev:test"]] |
Beta Was this translation helpful? Give feedback.
-
Ok, I am excited to be playing with current SNAPSHOT. Do you want small nits and observations as I see them at this point?
|
Beta Was this translation helpful? Give feedback.
-
A benefit of bb tasks that I am seeing is that I am thinking more about how will I make an operation clear and usable for its audience. And... loving not having to |
Beta Was this translation helpful? Give feedback.
-
Observation: As I move each script to a task I find I am deleting:
Because that's not the way the operation is invoked anymore. |
Beta Was this translation helpful? Give feedback.
-
Not sure if this is a result of current bb 3.1 work, but I am moving from: (defn main [args] ...)
(main *comand-line-args*) To: (defn -main [& args] ...) Which is more of what I am used to from Clojure, which I like. |
Beta Was this translation helpful? Give feedback.
-
Very small observation: not sure if you want to align help like lein does:
I think it makes things easier to read. I am finding as I work through rewrite-clj tasks I am usually using the Here's my current tasks:
What I'd prefer, maybe, is alignment like so:
Or maybe like:
Just ideas as they come to me. |
Beta Was this translation helpful? Give feedback.
-
I've completed my first pass through rewrite-clj with bb 3.1. I found myself making use of qualifiers on tasks to group them:
Work saved here, here's the commit. I was thinking of replacing script/ci_tests.clj entirely with tasks, might give that a try tomorrow. |
Beta Was this translation helpful? Give feedback.
-
Trying out today's 3.1 SNAPSHOT, much more readable > bb :tasks
The following tasks are available:
:import-vars - [gencode|check]
:lint
:outdated - report on outdated Clojure and npm dependencies
:ci/release - <args>
:ci/unit-tests
:doc/api-diffs - generate diff docs for rewrite-clj* APIs
:doc/cljdoc - <args> - preview what docs will look like on cljdoc
:doc/update-readme - honor our contributors in the README
:test/clj - [<args>]
:test/clj-watch - [<kaocha args>]
:test/cljs - [<args>] - run tests under ClojureScript
:test/cljs-shadow - run unit tests under shadow cljs
:test/cljs-watch - watch cljs tests with figwheel main
:test/coverage - generate code coverage reports for Clojure tests
:test/doc - test doc code blocks
:test/libs - [run|outdated] - verify that libs using rewrite-clj* work with current rewrite-clj
:test/native - run rewrite-clj and tests after compiled to GraalVM native image
:test/native-sci - interpret rewrite-clj tests via sci on a rewrite-clj native image
:test/native-sci-sanity - JVM only sanity test for :test/native-sci
Run bb :help <task> to view help of a specific task. Here are my notes from another experimentation run:
Today's changes are in this commit as I continue my experiments in this branch. Other observation:
|
Beta Was this translation helpful? Give feedback.
-
Another random bb tasks thought: I will likely rename my rewrite-clj This is merely an observation and something I have no issue with. |
Beta Was this translation helpful? Give feedback.
-
@lread Some updates:
{:tasks {:clean {:description "Foo" :task [:shell "rm -rf target"]}}} So, a top level map with
Let me know what you think. |
Beta Was this translation helpful? Give feedback.
-
I was able to port antq's Makefile to bb.edn's new task runner: |
Beta Was this translation helpful? Give feedback.
-
Trying out today's bb v3.1 SNAPSHOT (bb binary dated 4:39 PM).
Random bb tasks thoughts:
Today's changes are in this commit as I continue my experiments in this branch. |
Beta Was this translation helpful? Give feedback.
-
One thing I've been thinking about in the context of a task runner is the issue of scope. You might want to run tasks in ssh, or a container, or even a specific directory. |
Beta Was this translation helpful? Give feedback.
-
Now ported the tasks to normal functions: https://github.com/borkdude/antq/blob/bb.edn/script/tasks.clj The code is longer, but more flexible, so that may be the less controversial way to go.
Tasks can be discovered using (maybe warrants a
Tasks can be invoked using:
Docs:
This all works with the code that's now on master. Let's continue in https://github.com/babashka/babashka/discussions/765 for feedback on bb.edn. |
Beta Was this translation helpful? Give feedback.
-
Let's post some ideas here about issue #756 and I will extract the relevant info into the issue.
/cc @stevebuik @helins
Beta Was this translation helpful? Give feedback.
All reactions