pipe behaves unintuitively with by

[AlexBaranosky]

Do to the way pipe macro expands into a series of let bindings, something like this will not work as you'd hope/expect:

(pipe <>
        (by :host <>)
        (moving-event-window 2 <>)
        (adjust (fn [events]
                  (when-let [e (folds/sum events)]
                    (assoc e
                      :state (if (even? (:metric e))
                               "even"
                               "odd"))))
                (fn [& args] (println ""))
                #(prn "EVENT:" %)
                (changed-state {:init "even"} #(prn "CHANGED:" %))))

;; macros expands to
(clojure.core/let [<> (adjust
                        (fn [events]
                          (when-let [e (folds/sum events)]
                            (assoc
                              e
                              :state
                              (if (even? (:metric e)) "even" "odd"))))
                        (fn [& args] (println ""))
                        #(prn "EVENT:" %)
                        (changed-state
                          {:init "even"}
                          #(prn "CHANGED:" %)))
                   <> (moving-event-window 2 <>) ;; <-- doh! it's fully evaluating here! BAD.
                   <> (by :host <>)]
  <>)

One solution is re-implement pipe to always expand to something like:

(by :host
               (moving-event-window 2
                                    (adjust
                                     (fn [events]
                                       (when-let [e (folds/sum events)]
                                         (assoc
                                             e
                                           :state
                                           (if (even? (:metric e)) "even" "odd"))))
                                     (fn [& args] (println ""))
                                     #(prn "EVENT:" %)
                                     (changed-state
                                      {:init "even"}
                                      #(prn "CHANGED:" %)))))

Swiss-arrows' -<> is similar to pipe except you can't choose the marker ... and it isn't in reverse order .... but the code could be borrowed and tweaked to get it to macro expand like the above.

[AlexBaranosky]

Is anyone reading these issues? I can write a PR for this if I know it'll get merged... otherwise won't waste my time.

[jamtur01]

@AlexBaranosky Please feel free to submit a PR but I think @pyr is best equipped to judge if this is actually an issue.

[pyr]

Hey,

Thanks for the nudge. The issues are read but summer sees less OSS involvment on my side :-)
So pipe and by are a bit antithetical. All your events will flow to the next pipe and this is intended, since doing otherwise would prevent the canonical use case which is jumping out of split.

I'm not sure what you mean by "full evaluated" in the above gist.

Moving event-window by itself works, if you skip by (a loose take on what you were writing):

(test-stream (pipe - (moving-event-window 2 -) 
                               (smap folds/sum -)
                               (split (even? metric) (with :state :even -)
                                                               (with :state :odd -))
        [{:metric 1} {:metric 2} {:metric 3}]
        [{:metric 1 :state :odd} {:metric 3 :state :odd} {:metric 5 :state :even}])

This returns true as expected.
As soon as you add (by :host) to the mix, it cannot work.