<< Back to previous view

[CLJ-1161] sources jar has bad versions.properties resource Created: 11/Feb/13  Updated: 16/Dec/15  Resolved: 16/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4, Release 1.5
Fix Version/s: Release 1.8

Type: Defect Priority: Minor
Reporter: Steve Miner Assignee: Alex Miller
Resolution: Completed Votes: 18
Labels: build

Attachments: Text File 0001-CLJ-1161-Remove-version.properties-from-sources-JAR.patch     Text File clj-1161.patch    
Patch: Code
Approval: Ok

 Description   

The "sources" jar (at least since Clojure 1.4 and including 1.5 RC) has a bad version.properties file in it. The resource clojure/version.properties is literally:

version=${version}

The regular Clojure jar has the correct version string in that resource.

I came across a problem when I was experimenting with the sources jar (as used by IDEs). I naively added the sources jar to my classpath, and Clojure died on start up. The bad clojure/versions.properties file was found first, which led to a parse error as the clojure version was being set.

Cause: We configure the maven-source-plugin plugin in our pom.xml to exclude the clojure/version.properties file, and this works for SNAPSHOT builds but NOT for release builds. One difference during release builds is that we are running different goals and profiles, which seems to cause the configuration for the maven-source-plugin to be configured as defined in the oss-parent pom (https://repo1.maven.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom) instead of our configuration, and so the exclusion is not done.

Solution: If we use the same execution id as used in the oss-parent for the maven-source-plugin configuration, our configuration will override the parent pom configuration and the exclusion will take effect, both in snapshot builds and in release builds. The patch makes this change, just using "attach-sources" as the execution id.

Patch: clj-1161.patch

I was able to reproduce the bad source jar locally by running a command like:

mvn --batch-mode -Dtag=whatever release:prepare -DreleaseVersion=1.8.0-xyz -DdevelopmentVersion=1.8.1-SNAPSHOT

Note that this will generate some local files AND make a local tag and commit! So be careful testing locally if you have commit access (Stu!). You can undo that commit with `git reset HEAD~1` and the tag with `git tag -d whatever`. Before the patch, you should see:

$ jar xf target/clojure-1.8.0-xyz-sources.jar clojure/version.properties
$ cat clojure/version.properties
version=${version}   # this is the bad stuff

After, you should see that target/clojure-1.8.0-xyz-sources.jar does not contain clojure/version.properties at all:

$ jar tf target/clojure-1.8.0-xyz-sources.jar | grep version
# nothing found


 Comments   
Comment by Steve Miner [ 11/Feb/13 10:04 AM ]

Notes from the dev mailing list:

The "sources" JAR is generated by another Maven plugin, configured here:
https://github.com/clojure/clojure/blob/clojure-1.5.0-RC15/pom.xml#L169-L181

The simplest solution might be to just exclude the file from the sources jar. It looks like maven-source-plugin has an excludes option which would do the trick:

http://maven.apache.org/plugins/maven-source-plugin/jar-mojo.html#excludes

Comment by Jeff Valk [ 21/Apr/14 8:20 AM ]

This issue is marked closed, but I'm still seeing it: the clojure-1.6.0-sources.jar, clojure-1.5.1-sources.jar, etc on Maven Central still contain the bad version.properties files. More specifically, it looks like the fix has been applied to builds in the SNAPSHOTS repository, but not to RELEASES.

Fix applied: https://oss.sonatype.org/content/repositories/snapshots/org/clojure/clojure/
Not fixed: https://oss.sonatype.org/content/repositories/releases/org/clojure/clojure/

Comment by Alex Miller [ 24/Apr/14 4:15 PM ]

Not sure what's needed here, but marking incomplete.

Comment by Jeff Valk [ 25/Apr/14 11:13 AM ]

Would a fix for this update existing sources jars (1.5.1, 1.6.0, etc) on Central? Or would any fix have to wait on the next Clojure release?

Comment by Alex Miller [ 25/Apr/14 12:37 PM ]

For all the same reasons that mutable state is undesirable, changing an existing release jar in the central Maven repository is also undesirable. While it's not technically impossible, we will not update existing releases and this will need to wait for the next. I've looked at this problem a little and I do not yet know enough to know how to fix it or why it even varies between snapshot and release. Help welcome!

In which tool do you see a resulting problem from this?

Comment by Jeff Valk [ 25/Apr/14 11:56 PM ]

Despite the way I phrased the question, I'd hoped that would be the answer. It's the right policy.

Unfortunately, this issue leaves the released sources jars essentially unusable from a tools standpoint. CIDER now has source code navigation from stacktraces – you can jump into both Clojure and Java function definitions from the error/trace. For the latter, the sources jar (for Clojure or any other Java library) needs to be on the classpath as a dev dependency. There's more host interop support in the works for CIDER too ("embrace the host platform"), but not being able to add a dependency on a stable Clojure sources jar presents a wrinkle.

Are the official Clojure releases built by Hudson? The Hudson build right before the 1.6.0 release (#532) and the one right after (#534) both show the exclusion fix, as does the git clojure-1.6.0 tag, which when I check out and build from source, is fine. The Hudson builds with release tags (e.g. 1.6 = #533, 1.6-RC1 = #512, etc), though, don't show any artifacts other than a pom.xml. This would seem to make it rather hard to audit builds...am I missing something?

Comment by Michael Griffiths [ 11/Dec/15 3:43 PM ]

This still seems to be the case for the 1.7 and 1.8 sources jars; version.properties contains the above in both https://repo1.maven.org/maven2/org/clojure/clojure/1.7.0/clojure-1.7.0-sources.jar and https://repo1.maven.org/maven2/org/clojure/clojure/1.8.0-RC3/clojure-1.8.0-RC3-sources.jar

Comment by Alex Miller [ 11/Dec/15 9:47 PM ]

Thanks for bumping this - I keep meaning to look at it.

Comment by Bozhidar Batsov [ 15/Dec/15 1:38 AM ]

I really hope this is finally going to be resolved. It has caused us quite a lot of pain in CIDER.

Comment by Alex Miller [ 15/Dec/15 8:57 AM ]

While the final fix was simple, it took many painful hours of Maven detective work to track it down. We are on track to get it in the next RC.

Comment by Bozhidar Batsov [ 15/Dec/15 2:53 PM ]

Fantastic news!

Generated at Tue Jun 28 11:55:06 CDT 2016 using JIRA 4.4#649-r158309.