Skip to end of metadata
Go to start of metadata

The situation

  • Multi-module clojure-contrib in one repository has not worked well
    • Build is complicated and slow
    • Still hard for authors to work on their libs
  • We will have more projects under the org.clojure umbrella
    • New contrib libraries: nrepl, unify, finger-tree
    • More on the way
  • Current Clojure release process involves manual steps

The goals, in order of priority

  • For users
    • 1. Find release versions
    • 2. IDE / tool support
    • 3. Get just the stuff they need, without stuff they don't
    • 4. Easy-to-use distributions containing multiple libraries known to work together
  • For developers
    • 1. Build / release automation
    • 2. Unified build / release process across projects
    • 3. IDE / tool support

The plan

Clojure, the language

  • patch CLJ-681has been applied
    • Keeps the Ant build the way it is now
      • Developers can continue to use Ant for local builds
    • Maven controls the build/test/release process on Hudson
      • Real pom.xml
      • Declares Sonatype's oss-parent POM as a parent
        • Enables releases to public open-source repositories managed by Sonatype
      • Calls out to Ant for Clojure-specific compile and test phases
  • SNAPSHOT releases automatically deployed by hudson
  • Numbered releases controlled by Hudson/Maven
    • Only Clojure language committers may initiate a release

New contrib projects

  • To promote new contrib projects, see Guidelines for Clojure Contrib committers
  • To release, see How to Make Releases
  • Git repositories
    • New contrib libs get their own repos under the Clojure github organization
    • Each lib gets one or more "owners" who have commit access
    • Library authors still use JIRA + patches to accept contributions
    • All contributors must have a CA
  • JIRA
    • Each contrib library/project will get its own JIRA
  • Maven
    • Contrib projects must be built with Maven 2
    • All contrib POMs must declare org.clojure:pom.contribas a parent
      • Inherits from Sonatype's OSS POM
      • Defines release process (OSS snapshots + staging)
      • Set dependency on latest Clojure release (1.3.0-alpha5)
        • May be overridden on a per-project basis
        • Support building with a local Clojure JAR
      • Inherits from pom.oss-deploy
      • Sets license to EPL
      • Configures Java 1.5 as a target
      • Sets common plugin/build configurations
      • Uses com.theoryinpractise:clojure-maven-plugin
      • Defaults to source-only JAR, override to AOT-compile only when necessary

"Classic" / old clojure-contrib

  • The old monolithic build structure is deprecated
  • The interim multi-module build structure is deprecated
    • Module releases up to 1.3.0-alpha4 are available in the build.clojure.org release repository
      • These are source-only JARs wherever possible
      • They should be compatible with any version of Clojure
    • The multi-module build does not work with Clojure 1.3.0-alpha5 and later
  • Libraries with community demand and willing maintainers will become "new" contrib projects as above
  • The "classic" clojure-contrib JIRAis deprecated
    • Important issues should be reposted in the new per-project JIRAs

Work to be done

  • DONE Clean up / finalize pom.contrib
    • Replace 3-level inheritance (pom.oss-deploy -> pom.baseline -> pom.contrib) with just pom.contrib
  • DONE Finalize contrib project Hudson config
  • DONE Figure out how to give Hudson permissions to contrib authors (Stuart Sierra & Aaron Bedra)
  • DONE Create new Hudson job for finger-tree
  • DONE Figure out how to manage many similar Hudson configurations (Chas Emerick & Stuart Sierra)
  • TODO verify Sonatype release goals do the right thing in the presence of multiple staging repositories
  • TODO Get older releases into Maven Central?
  • TODO Redirect repositories at build.clojure.org to OSS Sonatype
    • Assuming we roll all existing releases into central, we only need to set up redirects for /snapshots

Possible future development

  • Improve the Maven build process; see Clojure Maven Plugin
  • Better Clojure-from-Java interface (see CLJ-452)
  • Aggregate projects that package many contrib libraries in one distribution
  • Testing harness to build/test many Clojure libraries with a locally-built Clojure JAR
  • Making the build process better
    • Improve clojure.lang.Compile
      • Investigate why compilation sometimes pauses for a long time (minutes) at the end
      • Investigate if core needs explicit compilation order
    • Investigate Maven 3
    • Investigate Polyglot Maven
    • Documentation: how do I do X, Y, or Z?
    • Write more plugins
    • Developing other non-Maven tools
      • Will require a lot of work to match what Maven can do
      • Not a Clojure/core priority
Labels:
  1. Nov 29, 2010

    If the maven release invocation fails, are the local commits on the build server (the pom changes) backed out automatically?

    1. Nov 29, 2010

      Not by any means I could see, which is why:

      1. Our "root" parent POM configures the release plugin to not push those changesets and tags upstream; we do this ourselves in the new hudson jobs, via a post-mvn "extra" build step (which is invoked only on build success, after all deployments).
      2. Just to be safe, every one of our hudson jobs should wipe out the local workspace at the start of each build (see the option behind the "Advanced" SCM config button, for e.g. build.poms). Bit of a nuclear option, but will guarantee that no tags from prior failed release builds slip into future pushes. (Yes, there's a git clean option, and the hudson git plugin should reset the wd, but I'm happier being safe rather than sorry.)

      Unfortunate that #1 is necessary, but that's a consequence of the defaults for the release plugin having been conceived of and implemented with non-D VCSs in mind.