Clojure - Ahead-of-time Compilation and Class Generation

Compiling

To compile a lib, use the compile function, and supply the namespace name as a symbol. For some namespace my.domain.lib, defined in my/domain/lib.clj, in the classpath, the following should occur:

A loader classfile will be produced in my/domain/lib__init.class, under *compile-path*, which must be in the classpath
A set of classfiles will be produced, one per fn in the namespace, with names such as my/domain/lib$fnname__1234.class
For each gen-class:
- A stub classfile will be produced with the specified name

Compiler options

The Clojure compiler can be controlled via the use of several compiler flags. At runtime these are stored in the dynamic var clojure.core/*compiler-options*, which is a map with the following optional keyword keys:

:disable-locals-clearing (boolean)
:elide-meta (vector of keywords)
:direct-linking (boolean)

These compiler options can be changed in a dynamic binding around a call to the compile function to change the compiler behavior.

Alternately, compiler options can also be set via the Java system properties at startup:

-Dclojure.compiler.disable-locals-clearing=true
-Dclojure.compiler.elide-meta="[:doc :file :line :added]"
-Dclojure.compiler.direct-linking=true

See below for more info on each of these options.

Locals clearing

By default, the Clojure compiler produces code that eagerly clears GC references to local bindings. However, when using a debugger locals will appear as nulls, which makes debugging difficult. Setting disable-locals-clearing=true will prevent locals clearing. It is not recommended to disable locals clearing for production compilation.

Elide meta

Var meta (docstrings, file and line info, etc) will be compiled into strings in the constant pool of the compiled classes. To decrease class size and make classloading faster, meta can be elided. This option takes a vector of meta keywords that should be removed - some common ones include :doc, :file, :line, and :added. Note that eliding meta may make certain features inoperable (for example, doc cannot return docstrings if they have been elided).

Direct linking

Normally, invoking a function will cause a var to be dereferenced to find the function instance implementing it, then invoking that function. This indirection via the var is one of the ways that Clojure provides a dynamic runtime environment. However, it has long been observed that the majority of function invocations in a production environment are never redefined in this way, incurring unnecessary redirection.

Direct linking can be used to replace this indirection with a direct static invocation of the function instead. This will result in faster var invocation. Additionally, the compiler can remove unused vars from class initialization and direct linking will make many more vars unused. Typically this results in smaller class sizes and faster startup times.

One consequence of direct linking is that var redefinitions will not be seen by code that has been compiled with direct linking (because direct linking avoids dereferencing the var). Vars marked as ^:dynamic will never be direct linked. If you wish to mark a var as supporting redefinition (but not dynamic), mark it with ^:redef to avoid direct linking.

As of Clojure 1.8, the Clojure core library itself is compiled with direct linking.

Runtime

Classes generated by Clojure are highly dynamic. In particular, note that no method bodies or other implementation details are specified in gen-class - it specifies only a signature, and the class that it generates is only a stub. This stub class defers all implementation to functions defined in the implementing namespace. At runtime, a call to some method foo of the generated class will find the current value of the var implementing.namespace/prefixfoo and call it. If the var is not bound or nil, it will call the superclass method, or if an interface method, generate an UnsupportedOperationException.

gen-class Examples

In the simplest case, an empty :gen-class is supplied, and the compiled class has only main, which is implemented by defining -main in the namespace. The file should be saved in src/clojure/examples/hello.clj:

(ns clojure.examples.hello
    (:gen-class))

(defn -main
  [greetee]
  (println (str "Hello " greetee "!")))

To compile, first ensure the target output directory classes exists:

mkdir classes

Then compile to generate the classes as follows:

(compile 'clojure.examples.hello)

And can be run like an ordinary Java app like so (be sure to include the output classes directory):

java -cp ./classes:clojure.jar clojure.examples.hello Fred
Hello Fred!

Here’s an example using both a more involved :gen-class, and stand-alone calls to gen-class and gen-interface. In this case we are creating classes we intend to create instances of. The clojure.examples.instance class will implement java.util.Iterator, a particularly nasty interface, in that it requires the implementation to be stateful. This class is going to take a String in its constructor and implement the Iterator interface in terms of delivering the characters from the string. The :init clause names the constructor function. The :constructors clause is a map of constructor signature to superclass constructor signature. In this case, the superclass defaults to Object, whose constructor takes no arguments. This object will have state, called state, and a main so we can test it.

:init functions (-init in this case) are unusual, in that they always return a vector, the first element of which is a vector of arguments for the superclass constructor - since our superclass takes no args, this vector is empty. The second element of the vector is the state for the instance. Since we are going to have to mutate the state (and the state is always final) we’ll use a ref to a map containing the string and the current index.

hasNext and next are implementations of methods in the Iterator interface. While the methods take no args, the implementation functions for instance methods will always take an additional first arg corresponding to the object the method is called upon, called by convention 'this' here. Note how the state can be obtained using an ordinary Java field access.

The gen-interface call will create an interface called clojure.examples.IBar, with a single method bar.

The stand-alone gen-class call will generate another named class, clojure.examples.impl, whose implementing namespace will default to the current namespace. It implements clojure.examples.IBar. The :prefix option causes the implementation of methods to bind to functions beginning with "impl-" rather than the default "-". The :methods option defines a new method foo not present in any superclass/interfaces.

Note in main how an instances of the classes can be created, and methods called, using ordinary Java interop. Using it would be similarly ordinary from Java.

(ns clojure.examples.instance
    (:gen-class
     :implements [java.util.Iterator]
     :init init
     :constructors {[String] []}
     :state state))

(defn -init [s]
  [[] (ref {:s s :index 0})])

(defn -hasNext [this]
  (let [{:keys [s index]} @(.state this)]
    (< index (count s))))

(defn -next [this]
  (let [{:keys [s index]} @(.state this)
        ch (.charAt s index)]
    (dosync (alter (.state this) assoc :index (inc index)))
    ch))

(gen-interface
 :name clojure.examples.IBar
 :methods [[bar [] String]])

(gen-class
 :name clojure.examples.impl
 :implements [clojure.examples.IBar]
 :prefix "impl-"
 :methods [[foo [] String]])

(defn impl-foo [this]
  (str (class this)))

(defn impl-bar [this]
  (str "I " (if (instance? clojure.examples.IBar this)
              "am"
              "am not")
       " an IBar"))

(defn -main [s]
  (let [x (new clojure.examples.instance s)
        y (new clojure.examples.impl)]
    (while (.hasNext x)
      (println (.next x)))
    (println (.foo y))
    (println (.bar y))))

Compile as above, and run like an ordinary Java app:

java -cp ./classes:clojure.jar clojure.examples.instance asdf
a
s
d
f
class clojure.examples.impl
I am an IBar