Clojure - Go Block Best Practices

General advice

It’s very tempting to do the following to send a message without waiting for a reply:

(go (>! c 42))

Although go blocks are cheap, they aren’t completely free. Thus it’s recommended to use

(async/put! c 42)

go just ends up calling put! eventually anyway, so there really isn’t a downside.

Also, if the code is being called inside a callback and you want to respect back-pressure, it’s fairly easy to use a recursive function along with put! to respect back-pressure.

(defn http-call
  "Makes an async call to a web browser"
  [url callback] ...)


(def urls [url1 url2 url3])

(defn load-urls
  "Spools the results of loading several urls onto a channel.
   does this without creating temporary channels or go blocks"
  [urls out-c]
  (http-call
    (first urls)
    (fn [response]
      (put! out-c response (fn [_] (load-urls (next urls) out-c))))))

(load-urls urls response-chan)

In this example we have some nice clean interop code that allows us to start working with channels in our app, without creating tons of channels or gos only to dispose of them shortly after they’re created.

Unsupported constructs and other limitations in go blocks

The go macro stops translating at function creation boundaries. This means the following code will fail to compile, or may just throw a runtime error stating that <! was used outside of a go block:

(go (let [my-fn (fn [] (<! c))] (my-fn)))

This is one thing to remember since many Clojure constructs create functions inside macros. The following are examples of code that will not work as one would expect:

(go (map <! some-chan))
(go (for [x xs]
      (<! x)))

However, other Clojure constructs, such as doseq do not allocate closures internally:

; This works just fine
(go (doseq [c cs]
      (println (<! c)))

Unfortunately, currently there isn’t a good way to know if a given macro will work as expected inside a go block unless one either looks at the source, or tests the code generated by the macro.

Why is this?

The best explanation for "why does go block translation stop at function creation?" basically comes down to a question of types. Examine the following snippet:

(map str [1 2 3])

We can easily see that this produces a seq of strings since the output type of str is a string. So what is the return type of async/<!? In the context of a go block it is an object taken from a channel. But the go block has to translate that to a parking call to async/put!. The return type of async/<! should really be thought of as something akin to Async<Object> or Promise<Object>. Thus the result of (map async/<! chans) is something like "a seq of pending channel operations" which makes no sense at all.

In short, the go macro can’t do these operations without some serious work. Other languages such as Erjang, allow for such constructs via translating all code in the entire JVM. This is something we’d like to avoid in core.async, as it complicates things and causes the logic of one library to infect the code of an entire JVM. So we’re left with the practical compromise, translation stops when it sees a (fn [] …).