The situation
- Multi-module clojure-contrib in one repository has not worked well
- Build is complicated and slow
- Still hard for authors to work on their libs
- We will have more projects under the org.clojure umbrella
- New contrib libraries: nrepl, unify, finger-tree
- More on the way
- Current Clojure release process involves manual steps
The goals, in order of priority
- For users
- 1. Find release versions
- 2. IDE / tool support
- 3. Get just the stuff they need, without stuff they don't
- 4. Easy-to-use distributions containing multiple libraries known to work together
- For developers
- 1. Build / release automation
- 2. Unified build / release process across projects
- 3. IDE / tool support
The plan
Clojure, the language
- patch CLJ-681 has been applied
- Keeps the Ant build the way it is now
- Developers can continue to use Ant for local builds
- Maven controls the build/test/release process on Hudson
- Real pom.xml
- Declares Sonatype's oss-parent POM as a parent
- Enables releases to public open-source repositories managed by Sonatype
- Calls out to Ant for Clojure-specific compile and test phases
- Temporary until we have better plugins, see Clojure Maven Plugin and Why Maven?
- Keeps the Ant build the way it is now
- SNAPSHOT releases automatically deployed by hudson
- Numbered releases controlled by Hudson/Maven
- Only Clojure language committers may initiate a release
New contrib projects
- Git repositories
- New contrib libs get their own repos under the Clojure github organization
- Each lib gets one or more "owners" who have commit access
- Library authors still use JIRA + patches to accept contributions
- All contributors must have a CA
- JIRA
- Each contrib library/project will get its own JIRA
- Maven
- Contrib projects must be built with Maven 2
- All contrib POMs must declare "org.clojure:pom.contrib" as a parent
- Inherits from Sonatype's OSS POM
- Defines release process (OSS snapshots + staging)
- Set dependency on latest Clojure release (1.3.0-alpha5)
- May be overridden on a per-project basis
- Support building with a local Clojure JAR
- Inherits from pom.oss-deploy
- Sets license to EPL
- Configures Java 1.5 as a target
- Sets common plugin/build configurations
- Uses com.theoryinpractise:clojure-maven-plugin
- Temporary until we have better plugins, see Clojure Maven Plugin and Why Maven?
- Defaults to source-only JAR, override to AOT-compile only when necessary
- Procedure for SNAPSHOT releases
- Push to github
- Hudson builds automatically
- Automatically deploys to Sonatype OSS SNAPSHOT repo
- Procedure for numbered releases
- Project owners have permission in Hudson to do a release of their projects
- Project owners must follow release numbering policy
- Use Semantic Versioning: major.minor.patch
- Core team must approve first 1.0.0 release
- In Hudson, click "Perform Maven Release" link
- Specify a release version number and the next development version number
- In Hudson's local checkout of the project:
- modifies pom.xml with the release version number, commits, tags with version
- modifies pom.xml with the dev. version number, commits to master
- checks out the tagged release, builds & deploys to Sonatype OSS
- Creates a numbered "staging" repository
- Automatically closes and promotes the staging repo to Sonatype "releases"
- Sonatype automatically syncs to Maven Central
- After a successful release; tags and new master are automatically pushed to Github
"Classic" / old clojure-contrib
- Libraries with community demand and willing maintainers will become "new" contrib projects as above
- Current 1.3.0-alpha* releases will continue to parallel Clojure 1.3.0-alpha* releases
- 1.3.0 final will be the last release of "classic" clojure-contrib
- Most modules are already released as source-only JARs
- can still use w/ new Clojure versions, assuming source compatibility
- Only four AOT-compiled modules remain
- condition
- fnmap
- jmx
- repl-ln
- Most modules are already released as source-only JARs
- The "classic" clojure-contrib JIRA will be abandoned
- Important issues should be reposted in the new per-project JIRAs
Work to be done
- TODO Clean up / finalize pom.contrib
- Replace 3-level inheritance (pom.oss-deploy -> pom.baseline -> pom.contrib) with just pom.contrib
- TODO Finalize contrib project Hudson config
- TODO Figure out how to give Hudson permissions to contrib authors (Stuart Sierra & Aaron Bedra)
- TODO Create new Hudson job for finger-tree
- TODO Figure out how to manage many similar Hudson configurations (Chas Emerick & Stuart Sierra)
- TODO verify Sonatype release goals do the right thing in the presence of multiple staging repositories
- TODO Get older releases into Maven Central?
- TODO Redirect repositories at build.clojure.org to OSS Sonatype
- Assuming we roll all existing releases into central, we only need to set up redirects for /snapshots
Possible future development
- Improve the Maven build process; see Clojure Maven Plugin
- Better Clojure-from-Java interface (see CLJ-452)
- Aggregate projects that package many contrib libraries in one distribution
- Testing harness to build/test many Clojure libraries with a locally-built Clojure JAR
- Making the build process better
- Improve clojure.lang.Compile
- Investigate why compilation sometimes pauses for a long time (minutes) at the end
- Investigate if core needs explicit compilation order
- Investigate Maven 3
- Investigate Polyglot Maven
- Documentation: how do I do X, Y, or Z?
- Write more plugins
- Not hard
- Possible in Clojure; see clojure-mojo-example
- More portable than shell scripts
- Developing other non-Maven tools
- Will require a lot of work to match what Maven can do
- Not a Clojure/core priority
- Improve clojure.lang.Compile
Labels: