How to use d3 force simulation?

[Folcon]

Firstly I'd like to say this library is great!

I'm just not sure how to hook in something like the d3 force simulation?

I've been trying to use the example code to get an idea of how this works, but I'm pretty stumped.
The state sim appears to initialise properly, but normally you use the tick to set the values of cx and cy values of the circles, but I've been trying to not directly mutate the dom. My current approach is to try and swap the values in the nodes/edges within the simulation?

(defn ->dots [did-mount?]
  (let [state (atom {:simulation
                     (.. js/d3
                        forceSimulation
                        (force "link" (.. js/d3 forceLink (id #(.id %)) (distance 120) (strength 1)))
                        (force "charge" (.. js/d3 forceManyBody (strength #(identity -30))))
                        (force "center" (.. js/d3 (forceCenter (/ 200 2) (/ 200 2))))
                      )})
    ]
    (.log js/console "d3:" (get @state :simulation))
    (fn [node ratom]
      (when did-mount?
        (set! (.-nodes (get @state :simulation)) (prepare-dataset ratom))
        (.on (get @state :simulation) "tick" #(.log js/console "TICK:" % node))
        (.. (get @state :simulation) (alpha 1) (alphaTarget 0) restart))
      (.log js/console "out:" node ratom)
        (rid3-> node
          {:cx 100
          :cy 100
          :r  50
          :sim #(.stringify js/JSON (.-nodes (get @state :simulation)))
          :data (fn [d] (.stringify js/JSON d))
          :title (fn [d] (or (gobj/get d "label") (gobj/getKeys d)))}))))

(defn prepare-dataset [ratom]
  (-> @ratom
    (#(map (fn [m] (into {} (for [[k v] m] [(str k) v]))) %))
      clj->js))

[rid3/viz
            {:id    "some-id"
            :ratom display
            :svg   {:did-mount (fn [node ratom]
                                  (rid3-> node
                                          {:width  400
                                           :height 400
                                           :style  {:background-color "white"}}))}
            :pieces [
              {:kind      :elem-with-data
               :class     "dots"
               :tag       "circle"
               :prepare-dataset prepare-dataset
               :did-mount  (->dots true)
               :did-update (->dots false)}
              ]
            }]

[Folcon]

PS: Here's the blocks link I mentioned which gives an example of this: https://bl.ocks.org/mbostock/4062045

[Folcon]

Ok, final thing for today:
Changing the below code

(force "link" (.. js/d3 forceLink (id #(.id %)) (distance 120) (strength 1)))

to

(force "link" (.. js/d3 forceLink (id #(do (.log js/console "NODES:" %) (gobj/get % ":db/id"))) (distance 120) (strength 1)))

Prints out multiple nodes with values for:

index: 0
vx: 0
vy: 0
x: 100
y: 100

alongside their other attrs, so it works. I'm just not sure what happens to them, as I've not worked out where they're stored/modified by the force-simulation.

Turning the tick function from:

(.on (get @state :simulation) "tick" #(.log js/console "TICK:" % node))

to

(.on (get @state :simulation) "tick" (fn [l] (.log js/console "TICK:" l node (get @state :simulation)) (.attr node "transform" (fn [d] (.log js/console "IN TRANSFORM:" d) (str "translate(" (.-x d) "," (.-y d) ")" )))))

gives me

TICK: undefined Selection {_groups: Array(1), _parents: Array(1), _enter: Array(1), _exit: Array(1)} {tick: ƒ, restart: ƒ, stop: ƒ, nodes: ƒ, alpha: ƒ, …}

and

IN TRANSFORM: {:person/name: "Test Name"}

Without the index, vx, vy, x and y, which is puzzling as that corresponds to:

function ticked() {
...

node
        .attr("cx", function(d) { return d.x; })

...
}

So I would expect to see the x, y... values being set. Perhaps I'm not correctly updating them?

[gadfly361]

@Folcon It looks like, with the current implementation of rid3, the only way to do this is with a :raw piece. I will noodle on how to update rid3 to better accommodate this use case.

Here is a working example though:
(based on this)

(ns folcon.core
  (:require
   [reagent.core :as reagent]
   [goog.object :as gobj]
   [rid3.core :as rid3 :refer [rid3->]]
   ))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Vars

(defonce app-state
  (reagent/atom {}))


(def nodes
  [{ :id "mammal" :group 0 :label "Mammals" :level 1 }
   { :id "dog" :group 0 :label "Dogs" :level 2 }
   { :id "cat" :group 0 :label "Cats" :level 2 }
   { :id "fox" :group 0 :label "Foxes" :level 2 }
   { :id "elk" :group 0 :label "Elk" :level 2 }
   { :id "insect" :group 1 :label "Insects" :level 1 }
   { :id "ant" :group 1 :label "Ants" :level 2 }
   { :id "bee" :group 1 :label "Bees" :level 2 }
   { :id "fish" :group 2 :label "Fish" :level 1 }
   { :id "carp" :group 2 :label "Carp" :level 2 }
   { :id "pike" :group 2 :label "Pikes" :level 2 }
   ])

(def links
  [{ :target "mammal" :source "dog" :strength 0.7 }
   { :target "mammal" :source "cat" :strength 0.7 }
   { :target "mammal" :source "fox" :strength 0.7 }
   { :target "mammal" :source "elk" :strength 0.7 }
   { :target "insect" :source "ant" :strength 0.7 }
   { :target "insect" :source "bee" :strength 0.7 }
   { :target "fish" :source "carp" :strength 0.7 }
   { :target "fish" :source "pike" :strength 0.7 }
   { :target "cat" :source "elk" :strength 0.1 }
   { :target "carp" :source "ant" :strength 0.1 }
   { :target "elk" :source "bee" :strength 0.1 }
   { :target "dog" :source "cat" :strength 0.1 }
   { :target "fox" :source "ant" :strength 0.1 }
   { :target "pike" :source "cat" :strength 0.1 }
   ])



;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Page


(def width 960)
(def height 600)

(def link-force
  (-> js/d3
      .forceLink
      (.id (fn [link]
             (gobj/get link "id")))
      (.strength (fn [link]
                   (gobj/get link "strength")))))

(def simulation
  (-> js/d3
      .forceSimulation
      (.force "link" link-force)
      (.force "charge"
              (.strength (js/d3.forceManyBody)
                         -120))
      (.force "center"
              (js/d3.forceCenter
               (/ width 2)
               (/ height 2)))))


(defn get-node-color [node]
  (let [level (gobj/get node "level")]
    (if (= 1 level)
      "red"
      "grey")))



(defn viz []
  (let [ratom (reagent/atom {:dataset {:nodes nodes
                                       :links links}})]
    (fn []
      (let []
        [rid3/viz
         {:id    "my-viz"
          :ratom ratom
          :svg   {:did-mount (fn [node ratom]
                               (rid3-> node
                                       {:width  width
                                        :height height
                                        }))}
          :pieces
          [{:kind      :raw
            :did-mount (fn [ratom]
                         (let [nodes      (-> @ratom
                                                :dataset
                                                :nodes
                                                clj->js)
                               links (-> @ratom
                                         :dataset
                                         :links
                                         clj->js)

                               linkElements (-> js/d3
                                                (.select "#my-viz svg .rid3-main-container")
                                                (.append "g")
                                                (.attr "class" "links")
                                                (.selectAll "line")
                                                (.data links)
                                                .enter
                                                (.append "line"))

                               nodeElements (-> js/d3
                                                (.select "#my-viz svg .rid3-main-container")
                                                (.append "g")
                                                (.attr "class" "nodes")
                                                (.selectAll "circle")
                                                (.data nodes)
                                                .enter
                                                (.append "circle"))

                               textElements (-> js/d3
                                                (.select "#my-viz svg .rid3-main-container")
                                                (.append "g")
                                                (.attr "class" "texts")
                                                (.selectAll "text")
                                                (.data nodes)
                                                .enter
                                                (.append "text"))]

                           (rid3-> linkElements
                                   {:stroke-width 1
                                    :stroke "rgba(50, 50, 50, 0.2)"})

                           (rid3-> nodeElements
                                   {:r    10
                                    :fill get-node-color})

                           (rid3-> textElements
                                   {:font-size 15
                                    :dx        15
                                    :dy        4}
                                   (.text (fn [node]
                                            (gobj/get node "label"))))

                           (-> simulation
                               (.nodes nodes)
                               (.on "tick" (fn []
                                             (-> nodeElements
                                                 (.attr "cx" (fn [node]
                                                               (gobj/get node "x")))
                                                 (.attr "cy" (fn [node]
                                                               (gobj/get node "y"))))

                                             (-> textElements
                                                 (.attr "x" (fn [node]
                                                              (gobj/get node "x")))
                                                 (.attr "y" (fn [node]
                                                              (gobj/get node "y"))))

                                             (-> linkElements
                                                 (.attr "x1" (fn [link]
                                                               (aget link "source" "x")))
                                                 (.attr "y1" (fn [link]
                                                               (aget link "source" "y")))
                                                 (.attr "x2" (fn [link]
                                                               (aget link "target" "x")))
                                                 (.attr "y2" (fn [link]
                                                               (aget link "target" "y")))))))

                           ;; needs to be after .on
                           (-> simulation
                               (.force "link")
                               (.links links))

                           ))
            }
           ]}]))))



(defn page [ratom]
  [:div
   [viz]
   ])



;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Initialize App

(defn dev-setup []
  (when ^boolean js/goog.DEBUG
    (enable-console-print!)
    (println "dev mode")
    ))

(defn reload []
  (reagent/render [page app-state]
                  (.getElementById js/document "app")))

(defn ^:export main []
  (dev-setup)
  (reload))

[Folcon]

@gadfly361 Thanks for this, I haven't vanished off the face of the earth ;)... Just been a bit busy and I won't be able to take a proper look over this until next weekend. I'll give it a proper go and see where I get =)...

Would it be useful to put this into the docs?

[Folcon]

So I've finally had a chance to experiment with this again.

I'm putting together a more complete re-frame example, with things like on-click and drag/drop.

I'm not sure if this is desired behaviour, but if you do this, then the graph never updates though the dispatch event triggers and the subscription has been updated.

The only way to change this I've noticed so far is to set :did-update in addition to :did-mount, however doing that definitely draws in the new values, but doesn't do anything to the old ones, so you get a slowly filling svg of nodes.

(defn d3-mouse-pos []
  ((.. js/d3 -mouse) (-> js/d3 .-event .-currentTarget)))

(defn display-graph-inner [graph-sub]
  (let [graph-name (gensym "display-graph")
        width 960 height 600
        resolution 20 r 15
        bounding-box {:min-x 10 :max-x (- width 10) :min-y 10 :max-y (- height 10)}
        make-node (fn [[x y]]
                    (let [c (count (:nodes @graph-sub))
                          _ (.log js/console "make-node" x y)]
                      {:id c :label c :x x :y y}))]
    (fn [graph-sub]
      [rid3/viz
       {:id graph-name
        :ratom graph-sub
        :svg   {:did-mount (fn [node ratom]
                            (rid3-> node
                                   {:width  width
                                         :height height
                                         :oncontextmenu "return false"
                                         :viewBox (str 0 " " 0 " " width " " height)
                                         :pointer-events :all}))}
        :pieces
        [{:kind :raw
          :did-mount
          (fn [ratom]
              (let [nodes (-> @ratom
                              :nodes
                              clj->js)
                    links (-> @ratom
                              :edges
                              clj->js)
                    _ (.log js/console "nodes::" nodes "\nedges::" links)

                    click-handler (fn [] (.log js/console "click" (js->clj (d3-mouse-pos)))
                                    (re-frame/dispatch [:graph/add-node (make-node (js->clj (d3-mouse-pos)))])
                                    (.log js/console "click" @graph-sub))
                    container (-> js/d3
                                  (.select (str "#" graph-name " svg"))
                                  (.on "click" click-handler))
                    _ (.log js/console "container::" container)
                    {:keys [min-x max-x min-y max-y]} bounding-box
                    nodeElements (-> js/d3
                                     (.select (str "#" graph-name " svg .rid3-main-container"))
                                     (.append "g")
                                     (.attr "class" "nodes")
                                     (.selectAll "circle")
                                     (.data nodes)
                                     .enter
                                     (.append "circle"))
                    textElements (-> js/d3
                                     (.select (str "#" graph-name " svg .rid3-main-container"))
                                     (.append "g")
                                     (.attr "class" "texts")
                                     (.selectAll "text")
                                     (.data nodes)
                                     .enter
                                     (.append "text"))
                    round-to-nearest (fn [n resolution]
                                         (-> n
                                           (/ resolution)
                                           (Math/round)
                                           (* resolution)))
                    round-to-grid (fn [pos k]
                                    (-> pos
                                        (max (condp = k
                                               :x min-x
                                               :y min-y))
                                        (min (condp = k
                                               :x max-x
                                               :y max-y))
                                        (round-to-nearest resolution)))
                    get-in-bounds (fn [k n]
                                    ;; k is :x or :y n is the node
                                    (-> n
                                      (gobj/get (name k))
                                      (round-to-grid k)))]

                (rid3-> nodeElements
                  {:r 10 :fill "green"
                   :cx (fn [d] (get-in-bounds :x d))
                   :cy (fn [d] (get-in-bounds :y d))})
                (rid3-> textElements
                        {:font-size 15
                         :dx        15
                         :dy        4
                         :x (fn [d] (get-in-bounds :x d))
                         :y (fn [d] (get-in-bounds :y d))}
                        (.text (fn [node]
                                 (or (gobj/get node "label") (gobj/getKeys node)))))))}]}])))

(defn display-graph [sub]
  (let [graph (re-frame/subscribe sub)]
    [display-graph-inner graph]))

[:div [display-graph [:graph/show]]]

Would you like me to tweak this so that it can be added to the docs?

[Folcon]

I've tried a couple of variants such as:

(defn d3-mouse-pos []
  ((.. js/d3 -mouse) (-> js/d3 .-event .-currentTarget)))

(defn display-graph-inner [graph-sub]
  (let [graph-name (gensym "display-graph")
        width 960 height 600
        resolution 20 r 15
        bounding-box {:min-x 10 :max-x (- width 10) :min-y 10 :max-y (- height 10)}
        {:keys [min-x max-x min-y max-y]} bounding-box
        make-node (fn [[x y]]
                    (let [c (count (:nodes @graph-sub))
                          _ (.log js/console "make-node" x y)]
                      {:id c :label c :x x :y y}))
        round-to-nearest (fn [n resolution]
                           (-> n
                             (/ resolution)
                             (Math/round)
                             (* resolution)))
        round-to-grid (fn [pos k]
                        (-> pos
                          (max (condp = k)
                              :x min-x
                              :y min-y)
                          (min (condp = k)
                              :x max-x
                              :y max-y)
                          (round-to-nearest resolution)))
        get-in-bounds (fn [k n]
                        ;; k is :x or :y n is the node
                        (-> n
                          (gobj/get (name k))
                          (round-to-grid k)))]
    (fn [graph-sub]
      [rid3/viz
       {:id graph-name
        :ratom graph-sub
        :svg   {:did-mount (fn [node ratom]
                             (rid3-> node
                               {:width  width
                                :height height
                                :oncontextmenu "return false"
                                :viewBox (str 0 " " 0 " " width " " height)
                                :pointer-events :all}))}
        :pieces
        [{:kind :raw
          :did-mount
          (fn [ratom]
              (let [nodes (-> @ratom
                              :nodes
                              clj->js)
                    links (-> @ratom
                              :edges
                              clj->js)
                    _ (.log js/console "nodes::" nodes "\nedges::" links)

                    click-handler (fn [] (.log js/console "click" (js->clj (d3-mouse-pos)))
                                    (re-frame/dispatch [:graph/add-node (make-node (js->clj (d3-mouse-pos)))])
                                    (.log js/console "click" @graph-sub))
                    container (-> js/d3
                                  (.select (str "#" graph-name " svg"))
                                  (.on "click" click-handler))
                    _ (.log js/console "container::" container)
                    node-refs (-> js/d3
                                (.select (str "#" graph-name " svg .rid3-main-container"))
                                (.append "g")
                                (.attr "class" "nodes")
                                (.selectAll "circle")
                                (.data nodes))]
                (rid3-> node-refs
                        (#(do (.log js/console "node-refs::" (js-keys %)) %))
                        .exit
                        .remove)
                (rid3-> node-refs
                  .enter
                  (.append "circle"
                    {:id (fn [d] (gobj/get d "id"))
                     :r 10 :fill "green"
                     :cx (fn [d] (get-in-bounds :x d))
                     :cy (fn [d] (get-in-bounds :y d))}))))}]}])))

(defn display-graph [sub]
  (let [graph (re-frame/subscribe sub)]
    [display-graph-inner graph]))

[:div [display-graph [:graph/show]]]

I think I'm going to start src diving to see what I'm missing >_<...

[Folcon]

Ok, I think that works =)...

(defn d3-mouse-pos []
  ((.. js/d3 -mouse) (-> js/d3 .-event .-currentTarget)))

(defn display-graph-inner [graph-sub]
  (let [graph-name (gensym "display-graph")
        width 960 height 600
        resolution 20 r 15
        bounding-box {:min-x 10 :max-x (- width 10) :min-y 10 :max-y (- height 10)}
        {:keys [min-x max-x min-y max-y]} bounding-box
        make-node (fn [[x y]]
                    (let [c (count (:nodes @graph-sub))]
                      {:id c :label c :x x :y y}))

        round-to-nearest (fn [n resolution]
                           (-> n
                             (/ resolution)
                             (Math/round)
                             (* resolution)))
        round-to-grid (fn [pos k]
                        (-> pos
                            (max (condp = k
                                   :x min-x
                                   :y min-y))
                            (min (condp = k
                                   :x max-x
                                   :y max-y))
                            (round-to-nearest resolution)))
        get-in-bounds (fn [k n]
                        ;; k is :x or :y n is the node
                        (-> n
                            (gobj/get (name k))
                            (round-to-grid k)))
        translate (fn [left top]
                    (str "translate("
                         (or left 0)
                         ","
                         (or top 0)
                         ")"))
        click-handler (fn [] (.log js/console "click" (js->clj (d3-mouse-pos)))
                        (re-frame/dispatch [:graph/add-node (make-node (js->clj (d3-mouse-pos)))]))
        mount-graph (fn [ratom]
                      (let [nodes (-> @ratom
                                      :nodes
                                      clj->js)
                            links (-> @ratom
                                      :edges
                                      clj->js)
                            container (-> js/d3
                                          (.select (str "#" graph-name " svg"))
                                          (.on "click" click-handler))
                            node-refs (-> js/d3
                                          (.select (str "#" graph-name " svg .rid3-main-container"))
                                          (.append "g")
                                          (.attr "class" "nodes")
                                          (.selectAll "circle")
                                          (.data nodes))
                            text-refs (-> js/d3
                                          (.select (str "#" graph-name " svg .rid3-main-container"))
                                          (.append "g")
                                          (.attr "class" "texts")
                                          (.selectAll "text")
                                          (.data nodes))]
                        (rid3-> node-refs
                          .enter
                          (.append "circle")
                          {:id (fn [d] (gobj/get d "id"))
                           :r 10 :fill "green"
                           :cx (fn [d] (get-in-bounds :x d))
                           :cy (fn [d] (get-in-bounds :y d))})
                        (rid3-> text-refs
                          .enter
                          (.append "text")
                          {:id (fn [d] (gobj/get d "id"))
                           :font-size 15
                           :dx        15
                           :dy        4
                           :x (fn [d] (get-in-bounds :x d))
                           :y (fn [d] (get-in-bounds :y d))}
                          (.text (fn [node]
                                   (or (gobj/get node "label") (gobj/getKeys node)))))))

        update-graph (fn [ratom]
                       (let [nodes (-> @ratom
                                       :nodes
                                       clj->js)
                             links (-> @ratom
                                       :edges
                                       clj->js)
                             node-refs (-> js/d3
                                           (.select (str "#" graph-name " svg .rid3-main-container"))
                                           (.selectAll "circle")
                                           (.data nodes))
                             text-refs (-> js/d3
                                           (.select (str "#" graph-name " svg .rid3-main-container"))
                                           (.selectAll "text")
                                           (.data nodes))]
                         
                           (rid3-> node-refs
                             .exit
                             .remove)
                           (rid3-> text-refs
                             .exit
                             .remove)
                           (rid3-> node-refs
                             .enter
                             (.append "circle")
                             {:id (fn [d] (gobj/get d "id"))
                              :r 10 :fill "green"
                              :cx (fn [d] (get-in-bounds :x d))
                              :cy (fn [d] (get-in-bounds :y d))})
                           (rid3-> text-refs
                             .enter
                             (.append "text")
                             {:id (fn [d] (gobj/get d "id"))
                              :font-size 15
                              :dx        15
                              :dy        4
                              :x (fn [d] (get-in-bounds :x d))
                              :y (fn [d] (get-in-bounds :y d))}
                             (.text (fn [node]
                                      (or (gobj/get node "label") (gobj/getKeys node)))))))]
    (fn [graph-sub]
      [rid3/viz
       {:id graph-name
        :ratom graph-sub
        :svg   {:did-mount (fn [node ratom]
                            (rid3-> node
                                   {:width  width
                                    :height height
                                    :oncontextmenu "return false"
                                    :viewBox (str 0 " " 0 " " width " " height)
                                    :pointer-events :all}))}
        :pieces
        [{:kind :raw
          :did-mount   mount-graph
          :did-update  update-graph}]}])))

[:div [display-graph [:graph/show]]]

@gadfly361 If you'd like me to turn this into an example I can do that =)...
Should hopefully give someone a more intermediate jumping off point to build more complex viz with :raw

[escherize]

@Folcon Hey I'd love to see this as an example! I'll be working on something similar soon and would love to benefit from your blood sweat and tears!

[Folcon]

Hey @escherize, what things would you like me to cover? :)... Also I'm doing most of this in re-frame, I can leave that out to make it more generic, or would that kind of thing be useful?

[escherize]

Hey @Folcon! Ideally I want to build a component that I can pass (or subscribe to) a data structure of a graph, and have it enter, exit, and update accordingly. Eventually I will want to use re-frame, but maybe it's outside the scope of a rid3 example, don't ya think?

…

On Thu, Nov 15, 2018 at 6:04 PM Folcon ***@***.***> wrote: Hey @escherize <https://github.com/escherize>, what things would you like me to cover? :)... Also I'm doing most of this in re-frame, I can leave that out to make it more generic, or would that kind of thing be useful? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAU8-DuHbD3VoDD12CbWuCU6pKf3DUcTks5uvgEfgaJpZM4UST73> .

[gadfly361]

@Folcon Thanks for following up on this! I haven't had time to dive through your latest example, but a functioning example would be great for the repo! And in vanilla reagent would be ideal :)

[prook]

d3-force example

This is a rewrite of Force-Directed Graph example to cljs and vanilla Reagent.

All you need is lein new figwheel-main rid3-force -- --reagent, add

  [rid3 "0.2.1-1"]
  [cljsjs/d3 "4.3.0-4"]

to dependencies in project.clj, add

  [rid3.core :as rid3 :refer [rid3->]]
  [cljsjs.d3]

to rid3-force.core requires, put the component and miserables.edn there as well, use [viz (r/atom miserables)] to render the component, finally do lein fig:build and you're all set.

(defn viz
  [ratom]
  (let [{:keys [links nodes]} @ratom
        width 950
        height 800
        nodes-group "nodes"
        node-tag "circle"
        links-group "links"
        link-tag "line"
        component-id "rid3-force-demo"
        links (clj->js links)
        nodes (clj->js nodes)
        nodes-sel (volatile! nil)
        links-sel (volatile! nil)
        sim (doto (js/d3.forceSimulation nodes)
              (.force "link" (-> (js/d3.forceLink links)
                                 (.id #(.-index %))))
              (.force "charge" (js/d3.forceManyBody))
              (.force "center" (js/d3.forceCenter (/ width 2) (/ height 2)))
              (.on "tick" (fn []
                            (when-let [s @links-sel]
                              (rid3-> s
                                      {:x1 #(.. % -source -x)
                                       :y1 #(.. % -source -y)
                                       :x2 #(.. % -target -x)
                                       :y2 #(.. % -target -y)}))
                            (when-let [s @nodes-sel]
                              (rid3-> s
                                      {:cx #(.-x %)}
                                      {:cy #(.-y %)})))))
        color (js/d3.scaleOrdinal (.-schemeCategory10 js/d3))
        drag (-> (js/d3.drag)
                 (.on "start" (fn started
                                [_d _ _]
                                (if (-> js/d3 .-event .-active zero?)
                                  (doto sim
                                    (.alphaTarget 0.3)
                                    (.restart)))))
                 (.on "drag" (fn dragged
                               [d _ _]
                               (let [event (.-event js/d3)]
                                 (set! (.-fx d) (.-x event))
                                 (set! (.-fy d) (.-y event)))))
                 (.on "end" (fn ended
                              [d _ _]
                              (if (-> js/d3 .-event .-active zero?)
                                (.alphaTarget sim 0))
                              (set! (.-fx d) nil)
                              (set! (.-fy d) nil))))]
    [rid3/viz {:id     component-id
               :ratom  ratom
               :svg    {:did-mount (fn [svg _ratom]
                                     (rid3-> svg
                                             {:width   width
                                              :height  height
                                              :viewBox #js [0 0 width height]}))}
               :pieces [{:kind            :elem-with-data
                         :class           links-group
                         :tag             link-tag
                         :prepare-dataset (fn [_ratom] links)
                         :did-mount       (fn [sel _ratom]
                                            (vreset! links-sel sel)
                                            (rid3-> sel
                                                    {:stroke         "#999"
                                                     :stroke-opacity 0.6
                                                     :stroke-width   #(-> (.-value %)
                                                                          js/Math.sqrt)}))}
                        {:kind            :elem-with-data
                         :class           nodes-group
                         :tag             node-tag
                         :prepare-dataset (fn [_ratom] nodes)
                         :did-mount       (fn [sel _ratom]
                                            (vreset! nodes-sel sel)
                                            (rid3-> sel
                                                    {:stroke       "#fff"
                                                     :stroke-width 1.5
                                                     :r            5
                                                     :fill         #(color (.-group %))}
                                                    (.call drag)))}]}]))

Notes and thoughts

• Unless I'm missing something, this is pretty much an exact rewrite of the original example.
• One thing I'm not too happy about is the use of nodes-sel and links-sel volatiles (set in :did-mount of their respective pieces). This is to avoid re-selecting the nodes and links each simulation tick (which could, possibly, be quite a performance hit -- haven't benchmarked this, though). Could this be done differently, without volatiles?
• rid3 ratom infrastructure is superfluous in this example, and I suspect this could be the case in general. What does this feature bring to the table, that cannot be done by other means (like the let block here)?

[gadfly361]

@prook

Thank you for your example, this was very helpful! I was able to reproduce a working force simulation from it.

Regarding the ratom, you are correct that, in the example, it is technically superfluous. However, the example currently only works with an initial dataset. If the dataset were to get updated, the visualization wont re-render and properly show the new dataset.

The secret sauce of rid3's ratom is here. It will cause a re-render when any of its data changes.

I made some tweaks to your example to show how you can make the force simulation re-render when the dataset changes (see below).

I want to call out a few things:

• I used a form-2 component, so the ratom would survive a re-render. (Note: in the in original example, it gets recreated on every re-render ... which means the data wouldn't persist)
• i am using a ratom and a helper-atom. Anything in the ratom will trigger a rerender when it changes. Anything in the helper-atom won't.
• I made a miserables2 dataset, which is the same as the original, with some links deleted from the end.

(def width 950)
(def height 800)
(def color (js/d3.scaleOrdinal (.-schemeCategory10 js/d3)))

(defn create-sim [links nodes helper-atom]
    (doto (js/d3.forceSimulation nodes)
      (.force "link" (-> (js/d3.forceLink links)
                         (.id #(.-index %))))
      (.force "charge" (js/d3.forceManyBody))
      (.force "center" (js/d3.forceCenter (/ width 2) (/ height 2)))
      (.on "tick" (fn []
                    (when-let [s (:links-sel @helper-atom)]
                      (rid3-> s
                              {:x1 #(.. % -source -x)
                               :y1 #(.. % -source -y)
                               :x2 #(.. % -target -x)
                               :y2 #(.. % -target -y)}))
                    (when-let [s (:nodes-sel @helper-atom)]
                      (rid3-> s
                              {:cx #(.-x %)}
                              {:cy #(.-y %)}))))))

(defn create-drag [ratom]
  (let [sim (:sim @ratom)]
    (-> (js/d3.drag)
        (.on "start" (fn started
                       [_d _ _]
                       (if (-> js/d3 .-event .-active zero?)
                         (doto sim
                           (.alphaTarget 0.3)
                           (.restart)))))
        (.on "drag" (fn dragged
                      [d _ _]
                      (let [event (.-event js/d3)]
                        (set! (.-fx d) (.-x event))
                        (set! (.-fy d) (.-y event)))))
        (.on "end" (fn ended
                     [d _ _]
                     (if (-> js/d3 .-event .-active zero?)
                       (.alphaTarget sim 0))
                     (set! (.-fx d) nil)
                     (set! (.-fy d) nil))))))

(defn update-ratom [ratom helper-atom data]
  (let [{:keys [links nodes]} data
        links (clj->js links)
        nodes (clj->js nodes)
        {:keys [sim]} @ratom]
    (swap! ratom assoc
           :sim (create-sim links nodes helper-atom)
           :links links
           :nodes nodes)))


(defn viz []
  ;; using a form-2 component so the ratom and helper-atom survive rerenders
  (let [
        ;; when the `ratom` gets updated, it will trigger a rerender bc of this line:
        ;; https://github.com/gadfly361/rid3/blob/8cee683f797c214106339d9f2a4a2b0708dc1ddf/src/main/rid3/viz.cljs#L12
        ratom (reagent/atom
               {:sim nil
                :links nil
                :nodes nil})

        ;; needs to be in separate atom to prevent performance issues
        ;; note: when the helper-atom gets updated, it doesn't cause a rerender
        helper-atom (atom {:links-sel nil
                           :nodes-sel nil})]
    (fn [] ;; need an inner fn to be a form-2 component
      [:div
       [:button
        {:on-click (fn []
                     (update-ratom ratom helper-atom data/miserables))}
        "Dataset 1"]

       [:button
        {:on-click (fn []
                     (update-ratom ratom helper-atom data/miserables2))}
        "Dataset 2"]

       [rid3/viz {:id     "rid3-force-demo"
                 :ratom  ratom
                 :svg    {:did-mount (fn [svg ratom]
                                       (rid3-> svg
                                               {:width   width
                                                :height  height
                                                :viewBox #js [0 0 width height]})
                                       (update-ratom ratom helper-atom data/miserables))

                          ;; override the did-update fall-back to did-mount
                          ;; if you don't, you'll observe performance issues because it'll keep updating the ratom and keep rerendering
                          :did-update (fn [_ _] )
                          }
                 :pieces [{:kind            :elem-with-data
                           :class           "links"
                           :tag             "line"
                           ;; the data should be derived from the ratom, otherwise it may not cause a rerender when / if the data changes
                           :prepare-dataset (fn [ratom]
                                              (:links @ratom))
                           :did-mount       (fn [sel ratom]
                                              (swap! helper-atom assoc :links-sel sel)
                                              (rid3-> sel
                                                      {:stroke         "#999"
                                                       :stroke-opacity 0.6
                                                       :stroke-width   #(-> (.-value %)
                                                                            js/Math.sqrt)}))}
                          {:kind            :elem-with-data
                           :class           "nodes"
                           :tag             "circle"
                           ;; the data should be derived from the ratom, otherwise it may not cause a rerender when / if the data changes
                           :prepare-dataset (fn [ratom]
                                              (:nodes @ratom))
                           :did-mount       (fn [sel ratom]
                                              (swap! helper-atom assoc :nodes-sel sel)
                                              (rid3-> sel
                                                      {:stroke       "#fff"
                                                       :stroke-width 1.5
                                                       :r            5
                                                       :fill         #(color (.-group %))}
                                                      (.call (create-drag ratom))))}]}]
       ])))

[prook]

Sorry, I should have been explicit about intentionally not handling data change: My goal was to keep the example to a bare minimum, and to actually avoid opening "the can of update" (just yet).

You see, handling update is hard.

In your example, for instance, the update is handled by update-ratom, which in turn calls create-sim. This means a new simulation is created and run on each data update, leaving the old one(s) to linger, to carry on with their heavy number crunching. On frequent data updates, the leaked sims will pile up, quickly bringing the whole app to a halt. We need to reuse the sim instead, update its nodes, links, and/or possibly other things.

Once we got that going, another problem appears. You see, when you update the data, it's as if the simulation started all over from scratch, with the nodes "exploding" from the origin on each update. I'd rather see the existing nodes to retain their position and velocity (or their fixed position, too!), while entering nodes join in nicely.

I was able to do the sim state carryover as well, but I think that goes far beyond the scope of a minimal example. Also, I'm not very happy with any of the solutions I came up with so far. They are all too much of a spaghetti code, too many moving parts with unintuitive dependencies.

I'll try and whip up another example that would tackle all this stuff as good as possible.

[prook]

So this is where I'm at right now: proper (?) handling of data updates in D3 simulation and rid3. Let's look at important bits.

create-sim creates and configures the simulation. Note that no nodes nor links (no data in general) are needed here. Also, the simulation is stopped (as there's nothing to simulate yet). Also note when-let guards in the tick function. They are crucial, and will be explained later.

(defn create-sim
  [d3-vars]
  (let [{:keys [width height]} @d3-vars]
    (doto (js/d3.forceSimulation)
      (.stop)
      (.force "link" (-> (js/d3.forceLink) (.id #(.-index %))))
      (.force "charge" (js/d3.forceManyBody))
      (.force "center" (js/d3.forceCenter (/ width 2) (/ height 2)))
      (.on "tick" (fn tick []
                    (when-let [s (:links-sel @d3-vars)]
                      (rid3-> s
                              {:x1 #(.. % -source -x)
                               :y1 #(.. % -source -y)
                               :x2 #(.. % -target -x)
                               :y2 #(.. % -target -y)}))
                    (when-let [s (:nodes-sel @d3-vars)]
                      (rid3-> s
                              {:cx #(.-x %)}
                              {:cy #(.-y %)})))))))

In create-drag, our drag object is prepared. The drag handlers mutate both the DOM and the simulation. Ewwww.

(defn create-drag
  [sim]
  (-> (js/d3.drag)
      (.on "start" (fn started
                     [_d _ _]
                     (if (-> js/d3 .-event .-active zero?)
                       (doto sim
                         (.alphaTarget 0.3)
                         (.restart)))))
      (.on "drag" (fn dragged
                    [d _ _]
                    (let [event (.-event js/d3)]
                      (set! (.-fx d) (.-x event))
                      (set! (.-fy d) (.-y event)))))
      (.on "end" (fn ended
                   [d _ _]
                   (if (-> js/d3 .-event .-active zero?)
                     (.alphaTarget sim 0))
                   (set! (.-fx d) nil)
                   (set! (.-fy d) nil)))))

Now, merge-nodes is something I've been pondering for a long time. Its purpose is to carry over position (x, y), velocity (vx,vy), or fixed position (fx, fy) of each node from previous simulation state to the new set nodes.

First we index original nodes by id function -- id takes a node (be it original or new) and returns a unique identifier (database id, natural key...). Next, based on that index, we run through the new nodes and look for original ones to carry over their state.

This code is kind of creepy mix of clj map and native js arrays, and I hate it. Ewwwww, again. I'd love to have this hidden away in some library (rid3-force? :) which would somehow plug this in automagically, without it being seen.

(defn merge-nodes
  [orig new id]
  (let [orig-map (into {} (map-indexed (fn [i n] [(id n) i]) orig))]
    (doseq [n new]
      (when-let [old (aget orig (orig-map (id n)))]
        (when-let [x (.-x old)] (set! (.-x n) x))
        (when-let [y (.-y old)] (set! (.-y n) y))
        (when-let [vx (.-vx old)] (set! (.-vx n) vx))
        (when-let [vy (.-vy old)] (set! (.-vy n) vy))
        (when-let [fx (.-fx old)] (set! (.-fx n) fx))
        (when-let [fy (.-fy old)] (set! (.-fy n) fy))))
    new))

Having merge-nodes at hand, update-sim! is actually quite simple. We pull old nodes from sim, carry their state over to new nodes, update and restart the simulation.

(defn update-sim! [sim alpha {:keys [links nodes]}]
  (let [old-nodes (.nodes sim)
        new-nodes (merge-nodes old-nodes nodes #(.-name %))]
    (doto sim
      (.nodes new-nodes)
      (-> (.force "link") (.links links))
      (.alpha alpha)
      (.restart))))

Now, let's put it all together in a level 2 component.

Note that d3-vars is a plain atom, not reagent/atom. This is intentional as we don't want to trigger component updates when modifying the atom. It took me a moment to realize that this is actually ok, that we actually want to allow d3 do its shenanigans without reagent noticing.

(Also I'm noticing a developing pattern here. It started as volatiles for :links-sel and :nodes-sel, then @gadfly361 came up with helper-atom, then, until a few moments ago, I called the atom viz-state, and then it hit me: d3-vars! This frames the atom's scope quite well, doesn't it?)

One thing I dislike (and which probably points out a flaw in this whole approach) is that :svg mount/update hooks are -- logically -- called before :pieces hooks. This means the simulation is (re)started in :svg hooks, probably slips in a few "blind" ticks before :pieces set their respective :*-sel... If it wasn't for those when-lets in tick function up in create-sim, the component would blow up on the first render. Kind of messy.

(defn viz
  [ratom]
  (let [d3-vars (atom {:width 950
                         :height 800
                         :links-sel nil
                         :nodes-sel nil})
        sim (create-sim d3-vars)
        drag (create-drag sim)
        color (js/d3.scaleOrdinal (.-schemeCategory10 js/d3))]
    (fn [ratom]
      [rid3/viz {:id     "rid3-force-demo"
                 :ratom  ratom
                 :svg    {:did-mount  (fn [svg ratom]
                                        (let [{:keys [width height]} @d3-vars]
                                          (rid3-> svg
                                                  {:width   width
                                                   :height  height
                                                   :viewBox #js [0 0 width height]}))
                                        (update-sim! sim 1 @ratom))
                          :did-update (fn [svg ratom]
                                        (update-sim! sim 0.3 @ratom))}
                 :pieces [{:kind            :elem-with-data
                           :class           "links"
                           :tag             "line"
                           :prepare-dataset (fn [ratom] (:links @ratom))
                           :did-mount       (fn [sel _ratom]
                                              (swap! d3-vars assoc :links-sel sel)
                                              (rid3-> sel
                                                      {:stroke         "#999"
                                                       :stroke-opacity 0.6
                                                       :stroke-width   #(-> (.-value %)
                                                                            js/Math.sqrt)}))}
                          {:kind            :elem-with-data
                           :class           "nodes"
                           :tag             "circle"
                           :prepare-dataset (fn [ratom] (:nodes @ratom))
                           :did-mount       (fn [sel _ratom]
                                              (swap! d3-vars assoc :nodes-sel sel)
                                              (rid3-> sel
                                                      {:stroke       "#fff"
                                                       :stroke-width 1.5
                                                       :r            5
                                                       :fill         #(color (.-group %))}
                                                      (.call drag)))}]}])))

And now for one last trick. As seen above, links and nodes are required in different places in js native form. So, to prevent repeated clj->js calls, we transform app-state using (reagent/track prechew app-state) before it enters the component.

(defn prechew
  [app-state]
  (-> @app-state
      (update :nodes clj->js)
      (update :links clj->js)))

(defn demo
  []
  [:div
   [:button {:on-click #(reset! app-state (miserables-rand-links))} "Randomize links"]
   [viz (reagent/track prechew app-state)]])

...and that's it.

The good thing is that it works. The bad thing is that this "basic" example is quite complex (at least my cognitive load is at its limits when dealing with it).

[gadfly361]

@prook

Reusing the sim is a great idea! I tested out your example locally and it worked great for me.

The concept of your 'prechew' never occurred to me, and I like it a lot 🙌

I like the name d3-vars a lot too. I think we should add it as an argument to rid3/viz as :d3-vars. Then we can expand the function signature of did-mount, did-update and prepare-dataset to include d3-vars.

I think I will try to make the above change to rid3 in the next week or two (unless you have a strong preference against adding d3-vars to the rid3/viz api).

Regarding something like rid3-force as a sister library, I think that could work well. Alternatively, it could be added to rid3 itself ... if you are interested in being a co-maintainer of rid3, I'd be happy to invite you to the repo :)

[kovasap]

Hey I stumbled across this just now when working on my own project. The example in #10 (comment) also works for me so far (with some errors when clicking on nodes, but those might be my fault?). I'm wondering if this code ever got upstreamed into the core rid3 codebase - I don't want to be working off of this example if there is a newer better way to accomplish this already in the library!

[gadfly361]

@kovasap Hey 👋 thanks for asking! This never made it in to rid3, so the above is still the best recommendation we have. As you work through this, please feel free to drop any thoughts or improvements here :)

[kovasap]

Ok thanks for the quick response! After my initial experimentation there are three things I still am not sure how to do:

1. Add links to nodes in the graph (so that when I click on them I'm taken to another web page).
2. Add an "on hover" feature to the nodes so that when I hover over them I get a text box (or arbitrary html). UPDATE: I got this working in kovasap/reddit-tree@7266292.
3. Figure out why the dragging functionality is broken for me (see my linked error). Currently, nothing happens when I try to drag nodes except these errors appear in the console.

I'm very new to cljs, reagent, and rid3, so any pointers on how to accomplish these things (or where to read about how to do them) would be much appreciated!

[kovasap]

Specifically for dragging, when I add these print statements I can see that the event variable is nil:

(defn create-drag
  [sim]
  (-> (js/d3.drag)
      (.on "start" (fn started
                     [_d _ _]
                     (if (-> js/d3 .-event .-active zero?)
                       (doto sim
                         (.alphaTarget 0.3)
                         (.restart)))))
      (.on "drag" (fn dragged
                    [d _ _]
                    (let [event (.-event js/d3)]
                      (prn "d" d)  ;; ADDED
                      (prn "event" event)  ;; ADDED
                      (set! (.-fx d) (.-x event))
                      (set! (.-fy d) (.-y event)))))
      (.on "end" (fn ended
                   [d _ _]
                   (if (-> js/d3 .-event .-active zero?)
                     (.alphaTarget sim 0))
                   (set! (.-fx d) nil)
                   (set! (.-fy d) nil)))))

Any ideas what that might mean?

[kovasap]

I'm also trying to figure out where exactly to use the fx and fy attributes to set nodes that meet certain conditions in fixed positions from the start of the simulation (and keep them there as if they were dragged). I'm trying to visualize a tree and I want the root node to always be at the top of the screen. I'll keep messing with this question, but if anyone knows the best place for this logic, I'd be happy to hear it!

[kovasap]

@prook Any ideas for #10 (comment)?

[prook]

Hi @kovasap. I'm not sure I'll be much help here -- I had to shift my focus elsewhere, and haven't returned back to this since. But let me invest 10 minutes to investigate.

Quick peek at a somewhat recent d3 example tells me the event is passed to event callbacks as the first parameter. The global d3.event has probably been removed in newer versions of D3. The drag handlers should probably look something like this:

(defn create-drag
  [sim]
  (-> (js/d3.drag)
      (.on "start" (fn started
                     [event d _]
                     (when (-> event .-active zero?)
                       (-> sim
                           (.alphaTarget 0.3)
                           (.restart)))
                     (set! (.-fx d) (.-x d))
                     (set! (.-fy d) (.-y d))))
      (.on "drag" (fn dragged
                    [event d _]
                    (set! (.-fx d) (.-x event))
                    (set! (.-fy d) (.-y event))))
      (.on "end" (fn ended
                   [event d _]
                   (when (-> event .-active zero?)
                     (-> sim
                         (.alphaTarget 0)))
                   (set! (.-fx d) nil)
                   (set! (.-fy d) nil)))))

I haven't tested nor attempted to run this, but it should work -- it's a verbatim transcription of that js example to cljs. :)

HTH

[prook]

Also, let me warn you -- in the most friendly way -- about biting off more than you can chew. This is a bit off-topic, but definitely something I wish I knew two years ago.

If you're new to Clojure, CLJS and Reagent, I'd recommend to study that first, and leave the monsters for later. You see:

• D3 is a great library, but incredibly huge, a world in itself.
• D3 and React/Reagent don't mesh well.
• JS interop is hard and ugly.
• Reagent too is a great library, but pretty much barebones. It's unnecessarily hard to do a project using only Reagent.

It's a mess unsuitable for baby steps. I've been in a position similar, if not identical, to yours. I struggled, I was overwhelmed, and made no real progress in the end.

From my own experience, I'd recommend to take a look at re-frame -- it's a library built upon Reagent. It saves you from re-inventing the wheel when trying to manage your app's state. But most importantly, it has exceptionaly good documentation, which transcends re-frame itself, and makes you go AHA! about Clojure, Reagent, React, functional programming, immutability, testing, about programming in general. It's an afternoon worth of reading at most, and is time spent much better than fumbling about with CLJS/Reagent/JS interop/D3 for weeks.

[kovasap]

#10 (comment) works perfectly! Thanks for looking into the issue!

#10 (comment) makes a lot of sense - I've started to realize this as I've worked on my project. I actually started working through a re-frame tutorial for making a d3 graph (like your code does). I stopped because it seemed to me like the library was just adding another layer of complexity it would be better for me to tackle later. Maybe now is the time to take another look. I will at least for sure read the linked documentation.

Thanks for the code fix and the advice, it's much appreciated!

[kovasap]

Hey I've gotten my project into a state I'm fairly happy with (all the issues I raised here have been fixed). You can see it at https://kovasap.github.io/reddit-tree.html. Thanks for all the help and support!

Posting here in part to also help others trying to do something similar : ).

One very strange issue I have yet to resolve is that when I build my app with npx shadow-cljs release app the node dragging functionality breaks. There are no errors in the console and when i hold down my mouse button on the nodes the graph simulation seems to be running (nodes will readjust), but I just cannot actually move the nodes. Running a npx shadow-cljs compile app results in perfectly functional behavior. I probably wont dig further into this issue here, but thought I'd mention it for completeness.