Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Goals

We need a convenient, uniform way to build Clojure and contrib projects. Problems are

  • Team will work across many projects and will not want to lean a different approach to building each one.
  • Projects will need to work with a variety of tools: Hudson, leiningen, maven, and IDEs at a minimum. (Note that these are to some degree overlapping.)
  • Users will want an easy way to find released versions.
  • Users will want sufficiently fine granularity of releases to get things they need, without a ton of stuff they don't.
  • Team will need automation: one step to trigger an entire push/build/release cycle when necessary.
  • Clojure/core will probably want to package multiple libs into easy-to-consume distributions.

Near-term Process

  • Clojure and all contrib projects will be built using Maven 2
  • All POMs will inherit from pom.baseline or pom.contrib
  • Only Hudson will deploy releases or SNAPSHOTs
  • All builds must be configured for JDK 1.5 in Hudson
    • Snag: Current clojure-maven-plugin requires 1.6. Can still use "toolchains" to build with 1.5; need to configure in POMs.
    • Mark Derricut is preparing a version of clojure-maven-plugin targeting 1.5
  • Stable and SNAPSHOT releases will be deployed to Sonatype's OSS repository
  • Stable releases will be initiated by the M2 Release Plugin
  • Contrib projects will produce source-only JARs whenever possible
  • ZIP distributions will be generated by Maven 2 and manually uploaded to clojure.org

Possible Future Development

...

The situation

  • Multi-module clojure-contrib in one repository has not worked well
    • Build is complicated and slow
    • Still hard for authors to work on their libs
  • We will have more projects under the org.clojure umbrella
    • New contrib libraries: nrepl, unify, finger-tree
    • More on the way
  • Current Clojure release process involves manual steps

The goals, in order of priority

  • For users
    • 1. Find release versions
    • 2. IDE / tool support
    • 3. Get just the stuff they need, without stuff they don't
    • 4. Easy-to-use distributions containing multiple libraries known to work together
  • For developers
    • 1. Build / release automation
    • 2. Unified build / release process across projects
    • 3. IDE / tool support

The plan

Clojure, the language

  • patch CLJ-681has been applied
    • Keeps the Ant build the way it is now
      • Developers can continue to use Ant for local builds
    • Maven controls the build/test/release process on Hudson
      • Real pom.xml
      • Declares Sonatype's oss-parent POM as a parent
        • Enables releases to public open-source repositories managed by Sonatype
      • Calls out to Ant for Clojure-specific compile and test phases
  • SNAPSHOT releases automatically deployed by hudson
  • Numbered releases controlled by Hudson/Maven
    • Only Clojure language committers may initiate a release

New contrib projects

  • To promote new contrib projects, see Guidelines for Clojure Contrib committers
  • To release, see How to Make Releases
  • Git repositories
    • New contrib libs get their own repos under the Clojure github organization
    • Each lib gets one or more "owners" who have commit access
    • Library authors still use JIRA + patches to accept contributions
    • All contributors must have a CA
  • JIRA
    • Each contrib library/project will get its own JIRA
  • Maven
    • Contrib projects must be built with Maven 2
    • All contrib POMs must declare org.clojure:pom.contribas a parent
      • Inherits from Sonatype's OSS POM
      • Defines release process (OSS snapshots + staging)
      • Set dependency on latest Clojure release (1.3.0-alpha5)
        • May be overridden on a per-project basis
        • Support building with a local Clojure JAR
      • Inherits from pom.oss-deploy
      • Sets license to EPL
      • Configures Java 1.5 as a target
      • Sets common plugin/build configurations
      • Uses com.theoryinpractise:clojure-maven-plugin
      • Defaults to source-only JAR, override to AOT-compile only when necessary

"Classic" / old clojure-contrib

  • The old monolithic build structure is deprecated
  • The interim multi-module build structure is deprecated
    • Module releases up to 1.3.0-alpha4 are available in the build.clojure.org release repository
      • These are source-only JARs wherever possible
      • They should be compatible with any version of Clojure
    • The multi-module build does not work with Clojure 1.3.0-alpha5 and later
  • Libraries with community demand and willing maintainers will become "new" contrib projects as above
  • The "classic" clojure-contrib JIRAis deprecated
    • Important issues should be reposted in the new per-project JIRAs

Work to be done

  • DONE Clean up / finalize pom.contrib
    • Replace 3-level inheritance (pom.oss-deploy -> pom.baseline -> pom.contrib) with just pom.contrib
  • DONE Finalize contrib project Hudson config
  • DONE Figure out how to give Hudson permissions to contrib authors (Stuart Sierra & Aaron Bedra)
  • DONE Create new Hudson job for finger-tree
  • DONE Figure out how to manage many similar Hudson configurations (Chas Emerick & Stuart Sierra)
  • TODO verify Sonatype release goals do the right thing in the presence of multiple staging repositories
  • TODO Get older releases into Maven Central?
  • TODO Redirect repositories at build.clojure.org to OSS Sonatype
    • Assuming we roll all existing releases into central, we only need to set up redirects for /snapshots

Possible future development

  • Improve the Maven build process; see Clojure Maven Plugin
  • Better Clojure-from-Java interface (see CLJ-452)
  • Aggregate projects that package many contrib libraries in one distribution
  • Redirect build.clojure.org/snapshots and /releases to Central or Sonatype repository
  • Testing harness to build/test many Clojure libraries with a locally-built Clojure JAR
  • Making the build process better
    • Improve clojure.lang.Compile
      • Investigate why compilation sometimes pauses for a long time (minutes) at the end
      • Investigate if core needs explicit compilation order
    • Investigate Maven 3
    • Investigate Polyglot Maven
    • Documentation: how do I do X, Y, or Z?
    • Write more plugins
    • Developing other non-Maven tools
      • Will require a lot of work to match what Maven can do
      • Not a Clojure/core priority