We need a convenient, uniform way to build contrib projects. Problems are
- Team will work across many projects and will not want to lean a different approach to building each one.
- Projects will need to work with a variety of tools: Hudson, leiningen, maven, and IDEs at a minimum. (Note that these are to some degree overlapping.)
- Users will want an easy way to find released versions.
- Users will want sufficiently fine granularity of releases to get things they need, without a ton of stuff they don't.
- Team will need automation: one step to trigger an entire push/build/release cycle when necessary.
- Clojure/core will probably want to package multiple libs into easy-to-consume distributions.
Here is a possible direction. Please shred this and come up with something better!
- Treat the pom.xml as the data of record. All the other tools listed above are capable of producing/consuming this as necessary.
- Register all projects with build.clojure.org to get CI and snapshot builds.
- Configure build.clojure.org to cut releases when it sees a commit with version metadata.
- As an example, the clojure build currently uses boolean flag name "interim", when this is false, Clojure could cut a release.
- Should automate both release and follow-on commit that resets the "interim" flag.
- Document everything and verify that a new contrib author can get started easily.
- Is the build box a good maven repos, or should something different be used for releases?
- Does whatever solution we choose imply any feature requests for leiningen?
- What about Clojars? The proposal above assumes (I think correctly) that the official Clojure and Contrib builds need to come from a official site that we control, not Clojars.
October 26, 2010 - Brainstorm
- Properly formatted pom.xml required
- Hudson polls master branch on git repo hourly and runs tests.
- Nightly Snapshots are built at 2:00AM ET
- Releases are built as requested (see below)
- Must have groupId of org.clojure
- artifactId is the remaining package name hyphenated (e.g. stu.library is stu-library)
- Must have version which is the current SNAPSHOT version number. (X.Y.Z-SNAPSHOT where X.Y.Z is greater than last release's version)
- Must have `test' and `package' goals.
- This (and any additional requirements as of yet undetermined) will be validated before maven touches the file. Failing validation fails the build.
- If there exists a git tag for the release (format TBD) a release package is created, pending a passing build.
- The version number is the one in pom.xml without -SNAPSHOT.
- Potential mechanism: gitattributes. We can define a filter to remove -SNAPSHOT from the pom when it is pulled so maven will version correctly.
- This would be required to build packages in all cases. Otherwise unintended inconsistencies could arise.
- Otherwise, a snapshot is created based on the version element in pom.xml.
- We still need to come up with a way to have hudson place artifact files in /snapshots or /releases accordingly.
- The current hudson plugin requires a hard-coded URI.
- Some options:
- Shell script which does the copy
- Possibly a better hudson plugin (not yet investigated)
- (Insert other, more brilliant idea here)
- We could dodge the issue entirely and not separate snapshots from full releases. There should be no risk of colliding version numbers given a nightly, timestamped snapshot build.
- This will break some Maven-aware tools, possibly including IDEs
Response to Issues (above):
- For now, the build box is probably fine. We will probably want to investigate a Nexus solution as load/usage increases. (That may be considerably sooner than this bullet would lead you to believe.)
- A leiningen task which generates the contrib-specific pom.xml would probably be welcome, but is not required by the process. Unless manually editing XML is unacceptable, in which case it is.
- Agreed about Clojars, at least in the context of contrib.
- Consider a pre-release "staging" repository (supported by Nexus Professional)
- How exactly we would pre-validate pom.xml files
- Whether to create and maintain a parent pom.xml file for clojure-contrib and require lib pom.xml files to refer to it
- Whether leiningen can participate fully in the scheme we're developing. Possible answers include yes, no, and yes with modifications and plugins.
What We're Ruling Out:
- Inventing / reinventing any tools. There is a consensus among lib authors that using existing tools, while not perfect, is desirable.