Enhance AOT compilation process to emit classfiles only for explicitly-specified namespaces
Description
Environment
Attachments
- 30 Nov 2011, 12:54 AM
- 10 Dec 2010, 09:34 PM
- 29 Nov 2010, 01:18 PM
- 20 Nov 2010, 03:28 AM
Activity
Stuart HallowayAugust 31, 2017 at 7:42 PM
I recommend that we close this ticket and concentrate our efforts on giving good advice to toolmakers and application developers. AOT compilation needs to transitively cover things as it does today in order to play nice with Java classloaders for the same reasons I just posted in the description of CLJ-1544.
Projects that AOT their code should use sift or equivalent and ship only the class files that correspond with their project.
Consumers of AOT projects will need to AOT any protocol deps no already AOTed, again see CLJ-1544.
Stuart HallowayOctober 12, 2016 at 9:24 PM
Agreed that sift is not a complete solution. If you AOT app A and use lib B, you must AOT lib B too, even if the original releaseer didn't.
Kevin DowneySeptember 7, 2016 at 9:44 PM
something like boot's sift is not a complete solution.
lein his included that functionality for sometime, and back in 2010 it defaulted to removing classes transitively generated. that turned out to break things so it was disabled. the functionality still exists but defaults to off.
the setting if someone wants to try it: https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L330
the issue that lead to it being turned off by default: https://github.com/technomancy/leiningen/issues/141
Stuart HallowaySeptember 7, 2016 at 9:15 PM
I believe that we should close this ticket. AOT ought to make all the classfiles as it does today to avoid classloader inversion problems (see e.g. http://dev.clojure.org/jira/browse/CLJ-1544), and tools can us e.g. boot's sift to remove classes not part of the current project from delivered jars.
We might want to make some tools that identify jars that erroneously includes compiled classes from other namespaces, i.e. resulting from not using sift.
Alex MillerJanuary 8, 2015 at 10:30 PM
As Stu said above, we do not have consensus on whether this is the right way to go with this. I am planning to look at all outstanding AOT tickets (including this one) early in 1.8.
Details
Assignee
UnassignedUnassignedReporter
Chas EmerickChas EmerickLabels
Patch
Code and TestPriority
Major
Details
Details
Assignee
Reporter
Labels
Patch
Priority
Major
Why This Was Declined
The current behavior of transitive compilation is a valuable semantic. It is arguably essential to e.g. AOT protocols ahead of their call sites.
On the other hand, you should not ship AOT compiled classes for which your project is not the source. As worded in the original ticket:
"Having the option of shipping either all AOT-compiled classfiles or mixed source/AOT depending upon one's distribution requirements would make that phase of work with a clojure codebase significantly easier and less error-prone."
This latter goal can be accomplished by keeping "what you compile" and "what you ship" cleanly separate. The compiler does not know or care about the latter, and tools can automate this with e.g. boot's
sift.Historical Context
This was originally/erroneously reported by Howard Lewis Ship in the clojure-contrib assembla:
My build file specifies the namespaces to AOT compile but if I include another namespace (even from a JAR dependency) that is not AOT compiled, the other namespace will be compiled as well. In my case, I was using clojure-contrib's clojure.contrib.str-utils2 namespace, and I got a bunch of clojure/contrib/str_utils2 classes in my output directory. I think that the AOT compiler should NOT precompile any namespaces that are transitively reached, only namespaces in the set specified by the command line are appropriate. As currently coded, you will frequently find unwanted third-party dependencies in your output JARs; further, if multiple parties depend on the same JARs, this could cause bloating and duplication in the eventual runtime classpath.The only question in my mind is what the default should be. We're all used to the current behaviour, but I'd guess that any nontrivial project where the form of the distributable matters (i.e. the source/AOT mix), providing as much control as possible by default makes the most sense. Given the tooling that most people are using, it's trivial (and common practice, IIUC) to provide a comprehensive list of namespaces one wishes to compile, so making that the default shouldn't be a hurdle to anyone. If an escape hatch is desired, a --transitive switch to clojure.lang.Compile could be added.