Skip to end of metadata
Go to start of metadata

Also known as CLJ-322

Why do we AOT compile?

  • Interop
    • To consume a Java API
      • Concrete derivation (gen-class)
      • Naming/packaging conventions
    • To expose an API to Java code
      • deftype/record/protocol/interface
    • Deployment environments
      • Special classloaders, e.g. OSGi, Eclipse
  • Application delivery
    • Main method (gen-class)
      • Mitigated by "-m" option to clojure.main
    • Bytecode post-processing
      • Android/Dalvik
    • Faster startup
      • Only an issue in constrained environments like Android
    • Hide source code from consumers
  • Syntax check & reflection warnings
    • Doesn't require writing files to disk
    • But still a common use case for invoking the compiler
    • Some tools support emitting .class files to temp dir
      • e.g. Mark Derricutt's clojure-maven-plugin

What do we want to AOT-compile?

  • Just a few things
    • For the Interop case
    • Or to shorten build times
    • Not currently supported
  • Everything
    • For the "Application delivery", "Syntax check", and "reflection warnings" cases
    • Currently supported by clojure.core/compile

Current behavior

  • Why is AOT-compilation transitive?
    • Clojure does not support separate compilation
      • All symbols must resolved at compile time
    • compile-files is a boolean
  • The effects of transitive compilation when AOT-comipiling a namespace "A"
    • When namespace A loads namespace B within the same project
      • Namespace B is AOT-compiled and emitted
    • When namespace A loads library C from external JAR
      • Library C's namespaces are AOT-compiled and emitted alongside A's
  • Work-arounds
    • Build tools: filter unwanted .class files from JAR
      • Ant, Maven, scripting
      • Clojure itself does this, e.g. "slim" JAR
      • Tedious
      • Not commonly used
      • Wastes time during builds to AOT-compile things which will be deleted
    • Write Java
      • Fairly common

Proposed solutions

Finer-grained control of the compilation process

  • A. Global flag for transitive/non-transitive compilation
    • Chas Emerick's patch
      • New system property "clojure.compiler.transitive"
      • When true (default), preserves current behavior
      • When false, only namespaces give as arguments to `compile` are emitted
    • Doesn't handle classes created by deftype/record/etc.
  • B. Specify exactly which .class files should be emitted
    • Stuart Sierra's patch
      • Specified as a Set of Strings, but could be any function, e.g. regex
  • C. Global flag for compiling "interop" forms
    • Is there a universal definition of what an "interop" form is?
      • gen-class, obviously
      • defprotocol?
      • deftype/defrecord?
    • Does not require maintaining list of .class files to emit
    • Could provide a convenient defaults for the common case
    • Macros like "deftype" can opt in with metadata
  • D. Metadata to control AOT-compilation
    • Permits fine-grained control
    • In-line with source code
    • How to support multiple AOT-compilation strategies in a single build
      • E.g. dev build only AOT-compiles interop forms, production AOT-compiles everything

Separate writing class files from compilation

  • E. Compiler holds references to generated bytecode
    • Not currently implemented in any patch
    • After compilation, call function to emit .class files
    • Will use lots of memory!
      • Or will it? The JVM already has this stuff in memory
      • Must disable in production!
    • Brings us closer to a Smalltalk-like "program image"
Labels:
  1. Nov 29, 2011

    Trying to think about how *compile-interop-forms* might work, I came up with the following:

    Option One

    The *compile-files* Var can be one of:

    • false (default, do not emit anything)
    • true (emit everything)
    • :interop (only emit per rules below)
      • gen-class: always emit
      • defprotocol/record/type/interface: always emit
      • Namespace or Var: never emit

    Pros:

    • existing tools that bind *compile-files* true do not break
    • common case (interop) is easy

    Cons:

    • Not flexible enough?

    Option Two

    The *compile-files* Var can be one of:

    • false (default, do not emit anything)
    • true (emit everything)
    • :emit-file (only emit per rules below)
      • gen-class: always emit
      • defprotocol/record/type/interface: when symbol has ^:emit-file true metadata
      • Namespace or Var: when symbol has ^:emit-file true metadata

    Pros:

    • existing tools that bind *compile-files* true do not break

    Cons:

    • ^:emit-file is redundant?
    • harder to implement
    • Not flexible enough?