Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Clojure today exists at a balance point between performance and dynamic flexibility. Some users would be willing to sacrifice dynamic features to gain more performance. Other users would be willing to sacrifice performance to gain better development error messages or debugging. The problem we aim to solve is to alter the Clojure build such that we can produce alternate versions of Clojure that are useful to smaller audiences while retaining Clojure as it is today.

Example builds we may want to produce (examples, not finalized):

  • "dev" build
    • More detailed error messages that would affect performance if included in default build
    • Skip locals clearing or other features to aid in debugging
    • Other features that may aid in development but affect performance
  • "dyn" build
    • Similar to what you get now with Clojure 1.6
  • "prod" build
    • Direct linking (static vars are inlined to be static fn calls) to gain greater perf
    • Trim metadata (omit docstrings for speed and space savings)
  • "experimental" build 
    • A place to provide experimental compiler features that we want users to evaluate for the default build

Constraints 

Clojure users:

  • Clojure users Build profile artifacts should be able to depend on and use special builds in a way that is natural for their environment available in public Maven repositories for use with existing build systems (Leiningen, Maven, etc)Specialized Clojure builds
    • Different artifacts:  [org.clojure/clojure-foo "1.6.0"]

Build:

    • Different qualifiers: [org.clojure/clojure "1.6.0-foo"]
    • Different classifiers: [org.clojure/clojure "1.6.0" :classifier "foo"]
  • Some features may require further compiler flags or metadata to activate even if using the specialized build
    • For example, you may need to supply ^:static metadata on a function to get access to this feature
    • Ideally, use of the special feature should be simply ignored if not using the specialized build (or a warning should direct you)

Build:

  • Produce multiple output jars each time we build a version (snapshot or release)
  • The jars will contain different versions of the classes that make up Clojure to deliver different code

Compiler:

  • Specialized build flags or env variables will need to be set during the build for each variant
Integration / release infrastructure:
  • Hudson CI server must be able to build and test custom builds as part of the dependency matrix
  • Hudson CI server must be able to perform releases for all of the variants

Compiler:

  • The compiler will need a standard way to determine which variant is active to make decisions during compilation.

Linking Choices

When a ns is compiled, for every call it can make a different linking decision:
  • For other namespaces 
  • For vars in current namespace
  • Perhaps overrides on specific functions - to leave some functions dynamically patchable 

 

Usage Scenarios

Clojure dyn (AOT)

  • Same as now, can redef calls to core Clojure fns both externally and internally
  • Metadata available

Clojure prod (AOT)

  • Clojure code is compiled with direct linking
  • Internal calls from one Clojure core function to another cannot be redefined (due to AOT)
    • So, no "monkeypatching" Clojure core internal calls
  • External code can redef calls to core Clojure fns (but only applies to those external calls)
  • Some metadata is trimmed (docstrings) to save space and loading time
    • Things like "doc" would not work
Application dyn
  • Same as now (but use of Clojure core depends on which version you use as above)
Application prod (source) 
  • As source is loaded and compiled, internal references are statically linked
  • Internal calls to application functions can be redefed?
Application prod (AOT)
  • Application code is compiled with direct linking
  • Application calls to Clojure functions is compiled with direct linking
  • Can redef in scope but all existing internal calls cannot be changed

 

Research

Maven coordinate options

  • Different artifactId
    • Releasing different artifacts would probably require running the build multiple times - there are probably some complexities in the CI and release process as a result.
  • Different qualifier
    • Qualifiers are really designed to give you more information about the purpose of a released version (alpha, beta, GA, build number), so this would be departing from standard Maven expectations.
  • Different classifier
    • Classifiers are designed for this use case - they allow you to create variants of the same group/artifact/version specialized for a particular environment (JDK variants is one example given)
    • Lein and Maven can both specify a classifier - in lein with [org.clojure/clojure "1.6.0" :classifier "foo"]
    • It seems good that the main build and variant share Maven co-ords - there is then a standard way to get from main artifact to (for example) the dev variant for tooling.
    • How-to in Maven
    • How-to in Ant
      • Define property that can be set to activate a particular variant
      • Set that property when running the build

Maven custom assembly

  • The Maven assembly plugin allows for arbitrary customization of the artifacts created by the build. However, creating multiple artifacts with different Maven coordinates will likely require multiple builds to produce each artifact for deployment. 

Custom script (ant, make, etc)