<< Back to previous view

[CLJ-1403] ns-resolve might throw ClassNotFoundException but should return nil Created: 14/Apr/14  Updated: 14/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

The doc of ns-resolve states that in case the symbol cannot be resolved, it should return nil.

user=> (doc ns-resolve)
-------------------------
clojure.core/ns-resolve
([ns sym] [ns env sym])
  Returns the var or Class to which a symbol will be resolved in the
  namespace (unless found in the environment), else nil.  Note that
  if the symbol is fully qualified, the var/Class to which it resolves
  need not be present in the namespace.
nil

However if the symbol contains dots and is not a resolvable Class, a ClassNotFoundException is thrown

user=> (ns-resolve *ns* 'foo.bar)
ClassNotFoundException foo.bar  java.net.URLClassLoader$1.run (URLClassLoader.java:372)
user=> (pst *e)
ClassNotFoundException foo.bar
	java.net.URLClassLoader$1.run (URLClassLoader.java:372)
	java.net.URLClassLoader$1.run (URLClassLoader.java:361)
	java.security.AccessController.doPrivileged (AccessController.java:-2)
	java.net.URLClassLoader.findClass (URLClassLoader.java:360)
	clojure.lang.DynamicClassLoader.findClass (DynamicClassLoader.java:61)
	java.lang.ClassLoader.loadClass (ClassLoader.java:424)
	java.lang.ClassLoader.loadClass (ClassLoader.java:357)
	java.lang.Class.forName0 (Class.java:-2)
	java.lang.Class.forName (Class.java:340)
	clojure.lang.RT.classForName (RT.java:2065)
	clojure.lang.Compiler.maybeResolveIn (Compiler.java:6963)
	clojure.core/ns-resolve (core.clj:4026)
nil


 Comments   
Comment by Alex Miller [ 14/Apr/14 2:07 PM ]

Can you include the (pst *e) ?

Comment by Nicola Mometto [ 14/Apr/14 2:10 PM ]

Added result of (pst *e) in the description





[CLJ-1402] sort-by calls keyfn more times than is necessary Created: 11/Apr/14  Updated: 11/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Steve Kim Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: performance


 Description   

clojure.core/sort-by evaluates keyfn for every pairwise comparison. This is wasteful when keyfn is expensive to compute.

user=> (def keyfn-calls (atom 0))
#'user/keyfn-calls
user=> (defn keyfn [x] (do (swap! keyfn-calls inc) x))
#'user/keyfn
user=> @keyfn-calls
0
user=> (sort-by keyfn (repeatedly 10 rand))
(0.1647483850582695 0.2836687590331822 0.3222305842748623 0.3850390922996001 0.41965440953966326 0.4777580378736771 0.6051704988802923 0.659376178201709 0.8459820304223701 0.938863131161208)
user=> @keyfn-calls
44


 Comments   
Comment by Steve Kim [ 11/Apr/14 11:46 AM ]

CLJ-99 is a similar issue





[CLJ-1401] CompilerException / IllegalStateException when reloading namespaces Created: 10/Apr/14  Updated: 12/Apr/14

Status: Reopened
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Mike Anderson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: compiler, errormsgs


 Description   
user> (ns op)
nil
op> (defn * [a b] (clojure.core/* a b))
WARNING: * already refers to: #'clojure.core/* in namespace: op, being replaced by: #'op/*
#'op/*
op> (ns use-op (:require [op :refer :all]))
WARNING: * already refers to: #'clojure.core/* in namespace: use-op, being replaced by: #'op/*
nil
use-op> (ns use-op (:require [op :refer :all]))
IllegalStateException * already refers to: #'op/* in namespace: use-op  clojure.lang.Namespace.warnOrFailOnReplace (Namespace.java:88)
use-op> (clojure.repl/pst *e)
IllegalStateException * already refers to: #'op/* in namespace: use-op
	clojure.lang.Namespace.warnOrFailOnReplace (Namespace.java:88)
	clojure.lang.Namespace.reference (Namespace.java:110)
	clojure.lang.Namespace.refer (Namespace.java:168)
	clojure.core/refer (core.clj:3920)
	use-op/eval2402/loading--4958--auto----2403 (NO_SOURCE_FILE:1)
	use-op/eval2402 (NO_SOURCE_FILE:1)
	clojure.lang.Compiler.eval (Compiler.java:6703)
	clojure.lang.Compiler.eval (Compiler.java:6692)
	clojure.lang.Compiler.eval (Compiler.java:6666)
	clojure.core/eval (core.clj:2927)
	clojure.main/repl/read-eval-print--6625/fn--6628 (main.clj:239)
	clojure.main/repl/read-eval-print--6625 (main.clj:239)

I would expect (at worst) a similar warning to the initial namespace loading, rather than an exception here.



 Comments   
Comment by Alex Miller [ 11/Apr/14 8:26 AM ]

Could you put together a better reproducible test case for this that does not depend on core.matrix? Also, please include the (pst *e) when it occurs.

Comment by Andy Fingerhut [ 11/Apr/14 10:19 AM ]

I have tried the smallest possible Leiningen project I could think of that would cause the warnings about redefinitions, to see if I could get the exception to occur. 'lein new try1' to create the skeleton project, then edit src/try1/core.clj to contain only the following function definitions:

(defn merge
  "This definition of merge replaces clojure.core/merge"
  [x y]
  (- x y))

(defn *
  [x y]
  (* x y))

Then start a REPL with 'lein repl', and I see this behavior:

user=> (require '[try1.core :as c])
WARNING: merge already refers to: #'clojure.core/merge in namespace: try1.core, being replaced by: #'try1.core/merge
WARNING: * already refers to: #'clojure.core/* in namespace: try1.core, being replaced by: #'try1.core/*
nil
user=> (require '[try1.core :as c] )
nil
user=> (require '[try1.core :as c] :reload)
WARNING: merge already refers to: #'clojure.core/merge in namespace: try1.core, being replaced by: #'try1.core/merge
WARNING: * already refers to: #'clojure.core/* in namespace: try1.core, being replaced by: #'try1.core/*
nil

Ths all looks like behavior as I would expect, and I did not see the exception that Mike reports.

It seems that either Ctrl+Alt+L in Counterclockwise does something different than (require ... :reload), or there is something different about Mike's namespace in addition to redefining names in clojure.core that is causing the problem.

Comment by Alex Miller [ 11/Apr/14 11:17 AM ]

Marking this as NR for now - would be happy to see it reopened with an easily reproducible test case.

Comment by Mike Anderson [ 12/Apr/14 12:41 AM ]

To reproduce:

(ns op)
(defn * [a b] (clojure.core/* a b)) ;; gives warning
(ns use-op (:require [op :refer :all])) ;; gives warning
(ns use-op (:require [op :refer :all])) ;; gives error!

I believe Counterclockwise is simply loading the namespace again with CTRL-Alt+L, which is causing the ns form to be re-executed.

The docstring implies that ns can be used multiple times ("Sets ns to the namespace named by name (unevaluated), creating it if needed") so I would certainly expect multiple invocations of ns to be a no-op





[CLJ-1400] Error "Can't refer to qualified var that doesn't exist" should name the bad symbol Created: 09/Apr/14  Updated: 10/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Compiler, errormsgs
Environment:

OS X


Approval: Triaged

 Description   

Def of var with a ns that doesn't exist will yield this error:

user> (def foo/bar 1)
CompilerException java.lang.RuntimeException: Can't refer to qualified var that doesn't exist, compiling:(NO_SOURCE_PATH:1:1)

Cause: Compiler.lookupVar() returns null if the ns in a qualified var does not exist yet.

Proposed: The error message would be improved by naming the symbol and throwing a CompilerException with file/line/col info. It's not obvious, but this may be the only case where this error occurs. If so, the error message could be more specific that the ns is the part that doesn't exist.

Patch:

Screened by:






[CLJ-1391] Allow logical operators on assert expressions Created: 26/Mar/14  Updated: 26/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Sanel Zukan Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: clojure.test


 Description   

With current code, it is not possible to express logical operators on some clojure.test assert expressions. For example, this will work:

(is (thrown? Exception <some-expression>))

however, here will fail:

(is (not (thrown? Exception <some-expression>)))

since '(thrown?)' is not an ordinary function, but looks like. This also adds confusion which is hard to explain to others unless '(is)' code was shown first.

Also, if the one would like to implement macro (e.g. 'is-not-thrown?') in form:

(defmacro is-not-thrown? [e expr]
  `(is (not ('thrown? ~e ~expr))))

which could be even more confusing for a person not knowing how 'thrown?' is implemented.






[CLJ-1389] Re-loading a namespace ignores metadata specified for the namespace Created: 20/Mar/14  Updated: 20/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: metadata, namespace, repl


 Description   

Using the REPL I added some metadata to a namespace and reloaded it.

(ns io.aviso.rook-test5)

to

(ns io.aviso.rook-test5
  "A testing namespace"
  {:inherted   :namespace
   :overridden :namespace})

But requesting the meta data yields nil:

(-> 'io.aviso.rook-test5 find-ns meta)
=> nil

I have tested a few variations, such as putting the metadata on the symbol instead of providing an attribute map. In all cases, the metadata from before the load persists.

Using remove-ns before re-loading the namespace does the right thing ... the metadata shows up as expected.






[CLJ-1385] Docstrings for `conj!` and `assoc!` should suggest using the return value; effect not always in-place Created: 16/Mar/14  Updated: 06/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Pyry Jahkola Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: collections, docstring

Attachments: Text File CLJ-1385-reword-docstrings-on-transient-update-funct.patch    

 Description   

The docstrings of both `assoc!` and `conj!` say "Returns coll.", suggesting the transient edit happens always in-place, `coll` being the first argument.

However, the fact that the following example omits the key `8` in its result proves that in-place edits aren't always the case:

(let [a (transient {})]
      (dotimes [x 9]
        (assoc! a x :ok))
      (persistent! a))
    ;;=> {0 :ok, 1 :ok, 2 :ok, 3 :ok, 4 :ok, 5 :ok, 6 :ok, 7 :ok}

Instead, programmers should be guided towards using constructs like `reduce` with transients:

(persistent! (reduce #(assoc! %1 %2 :ok)
                 (transient {})
                 (range 9)))
    ;;=> {0 :ok, 1 :ok, 2 :ok, 3 :ok, 4 :ok, 5 :ok, 6 :ok, 7 :ok, 8 :ok}

The easiest way to achieve this is by changing the docstrings of (at least) `conj!` and `assoc!` to not read "Returns coll." but instead tell that the change is destructive.



 Comments   
Comment by Alex Miller [ 16/Mar/14 8:49 AM ]

When modifying transient collections, it is required to use the collection returned from functions like assoc!. The ! here indicates its destructive nature. The transients page (http://clojure.org/transients) describes the calling pattern pretty explicitly: "You must capture and use the return value in the next call."

I do not agree that we should be guiding programmers away from using functions like assoc! – transients are used as a performance optimization and using assoc! or conj! in a loop is often the fastest version of that. However I do think it would be helpful to make the docstring more explicit.

Comment by Gary Fredericks [ 05/Apr/14 10:23 AM ]

Alex I think you must have misread the ticket – the OP is suggesting guiding toward using the return value of assoc!, not avoiding assoc! altogether.

And the docstring is not simply inexplicit, it's actually incorrect specifically in the case that the OP pointed out. conj! and assoc do not return coll at the point where array-maps transition to hash-maps, and the fact that they do otherwise is supposed to be an implementation detail as far as I understand it.

Comment by Alex Miller [ 05/Apr/14 11:55 AM ]

@Gary - you're right, I did misread that.

assoc and conj both explicitly say "return a new collection" whereas assoc! and conj! say "Returns coll." I read that as "returns the modified collection" without regard to whether it's the identical instance, but I can read it your way too.

Would saying "Returns updated collection." transmit the right idea? Using "collection" instead of "coll" removes the concrete tie to the variable and "updated" hints more strongly that you should use the return value.

Comment by Pyry Jahkola [ 05/Apr/14 12:47 PM ]

@Alex, that update makes it sound right to me, FWIW.

Comment by Gary Fredericks [ 05/Apr/14 2:37 PM ]

Yeah, I think that's better. Thanks Alex. I'd be happy to submit a patch for that but I'm assuming patches are too heavy for this kind of change?

Comment by Andy Fingerhut [ 06/Apr/14 3:35 PM ]

Patches are exactly what has been done in the past for this kind of change, if it is in a doc string and not on the clojure.org web page.

Comment by Alex Miller [ 06/Apr/14 4:13 PM ]

Yup, patch desired.

Comment by Gary Fredericks [ 06/Apr/14 5:32 PM ]

Glad I asked.

Patch is attached that also updates the docstring for pop! which had the same issue, though arguably it's less important since afaik pop! does always return the identical collection (but I don't think this is part of the contract).





[CLJ-1381] Improve support for extending protocols to primitive arrays Created: 13/Mar/14  Updated: 13/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5, Release 1.6
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: protocols


 Description   

It is possible to extend protocols to primitive arrays but specifying the class for the type is a little tricky:

(defprotocol P (p [_]))
(extend-protocol P (Class/forName "[B") (p [_] "bytes"))
(p (byte-array 0))   ;; => "bytes"

However, things go bad if you try to do more than one of these:

(extend-protocol P 
  (Class/forName "[B") (p [_] "bytes")
  (Class/forName "[I") (p [_] "ints"))
CompilerException java.lang.UnsupportedOperationException: nth not supported on this type: Character, compiling:(NO_SOURCE_PATH:1:1)
	clojure.lang.Compiler.analyze (Compiler.java:6380)
	clojure.lang.Compiler.analyze (Compiler.java:6322)
	clojure.lang.Compiler$MapExpr.parse (Compiler.java:2879)
	clojure.lang.Compiler.analyze (Compiler.java:6369)
	clojure.lang.Compiler.analyze (Compiler.java:6322)
	clojure.lang.Compiler$InvokeExpr.parse (Compiler.java:3624)
	clojure.lang.Compiler.analyzeSeq (Compiler.java:6562)
	clojure.lang.Compiler.analyze (Compiler.java:6361)
	clojure.lang.Compiler.analyze (Compiler.java:6322)
	clojure.lang.Compiler$BodyExpr$Parser.parse (Compiler.java:5708)
	clojure.lang.Compiler$FnMethod.parse (Compiler.java:5139)
	clojure.lang.Compiler$FnExpr.parse (Compiler.java:3751)
Caused by:
UnsupportedOperationException nth not supported on this type: Character
	clojure.lang.RT.nthFrom (RT.java:857)
	clojure.lang.RT.nth (RT.java:807)
	clojure.core/emit-hinted-impl/hint--5951/fn--5953 (core_deftype.clj:758)
	clojure.core/map/fn--4207 (core.clj:2487)
	clojure.lang.LazySeq.sval (LazySeq.java:42)
	clojure.lang.LazySeq.seq (LazySeq.java:60)
	clojure.lang.RT.seq (RT.java:484)
	clojure.lang.RT.countFrom (RT.java:537)
	clojure.lang.RT.count (RT.java:530)
	clojure.lang.Cons.count (Cons.java:49)
	clojure.lang.Compiler.analyze (Compiler.java:6352)
	clojure.lang.Compiler.analyze (Compiler.java:6322)

The code in {parse-impls} is seeing the second {(Class/forName "[I")} as a function, not as a new type. One workaround for this is to only extend the protocol to one type at a time.

It would be even better (moving into enhancement area) if there was a syntax here to specify primitive array types - we already have the syntax of {bytes, ints, longs}, etc for type hints and those seem perfectly good to me.






[CLJ-1376] Initialize internal maps to more efficient version Created: 11/Mar/14  Updated: 11/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: performance


 Description   

In reviewing some hashing stuff, I noticed that there are many places internal to Clojure that use maps initialized with PersistentHashMap.EMPTY. Many of these maps are likely to have a small number of entries such that a PersistentArrayMap might be more efficient.

These are the candidates:

src/jvm/clojure/lang/ARef.java
19:private volatile IPersistentMap watches = PersistentHashMap.EMPTY;

src/jvm/clojure/lang/Compiler.java
3009:				IPersistentMap m = PersistentHashMap.EMPTY;
3819:					       KEYWORDS, PersistentHashMap.EMPTY,
3820:					       VARS, PersistentHashMap.EMPTY,
3964:	IPersistentMap closes = PersistentHashMap.EMPTY;
3977:	IPersistentMap keywords = PersistentHashMap.EMPTY;
3978:	IPersistentMap vars = PersistentHashMap.EMPTY;
5121:                            ,CLEAR_SITES, PersistentHashMap.EMPTY
7259:			       KEYWORDS, PersistentHashMap.EMPTY,
7260:			       VARS, PersistentHashMap.EMPTY
7418:			IPersistentMap opts = PersistentHashMap.EMPTY;
7475:			IPersistentMap fmap = PersistentHashMap.EMPTY;
7522:					       KEYWORDS, PersistentHashMap.EMPTY,
7523:					       VARS, PersistentHashMap.EMPTY,
7912:                            ,CLEAR_SITES, PersistentHashMap.EMPTY

src/jvm/clojure/lang/LispReader.java
755:					RT.map(GENSYM_ENV, PersistentHashMap.EMPTY));

src/jvm/clojure/lang/MultiFn.java
39:	this.methodTable = PersistentHashMap.EMPTY;
41:	this.preferTable = PersistentHashMap.EMPTY;
49:		methodTable = methodCache = preferTable = PersistentHashMap.EMPTY;

src/jvm/clojure/lang/Var.java
48:	final static Frame TOP = new Frame(PersistentHashMap.EMPTY, null);
175:	setMeta(PersistentHashMap.EMPTY);
341:	IPersistentMap ret = PersistentHashMap.EMPTY;

Approach: Two possible approaches - initialize to PersistentArrayMap.EMPTY or call RT.map(). The latter requires function invocation so is slightly slower, but has the benefit of localizing map construction into a single place.






[CLJ-1371] divide(Object, Object) with (NaN, 0) does not return NaN Created: 07/Mar/14  Updated: 07/Mar/14

Status: Reopened
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Yongqian Li Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: math


 Description   

user=> (def x Double/NaN)
#'user/x
user=> (/ x 0)

ArithmeticException Divide by zero clojure.lang.Numbers.divide (Numbers.java:156)
user=> (/ Double/NaN 0)
Double/NaN



 Comments   
Comment by Alex Miller [ 07/Mar/14 7:50 AM ]

As per the Java Language Specification (http://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.2.4),

"All numeric operations with NaN as an operand produce NaN as a result."

Comment by Yongqian Li [ 07/Mar/14 7:54 AM ]

But in the first example it produces an ArithmeticException.

Comment by Alex Miller [ 07/Mar/14 9:27 AM ]

Ah, I see the question now.

Here we are dividing a double by a long. In the first case, this is parsed as divide(Object, long) which then calls divide(Object, Object), which throws ArithmeticException if the second arg is 0 (regardless of the first arg).

In the second case it's parsed as divide(double, long) which just relies on Java to properly upcast the primitive long to a double to do the divide.

Note that making this call with 2 doubles does return NaN:

user=> (def x Double/NaN)
#'user/x
user=> (/ x 0.0)
NaN

or type hinting x to a double works as well:

user=> (def x Double/NaN)
#'user/x
user=> (/ ^double x 0.0)
NaN

I think one option to "fix" this behavior would be to add checks in divide(Object, Object) to check whether x is NaN and instead return NaN.





[CLJ-1368] Document usage for case with non-readable constants Created: 02/Mar/14  Updated: 02/Mar/14

Status: Reopened
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Herwig Hochleitner Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docs, interop


 Description   

Problem

It is pretty obscure how to get constant-time dispatch for e.g. Enums, even if user knows about case.

Proposal

The possibility to dispatch to arbitrary constants with case, by wrapper macro, should be documented.

Wording

  • Should it warn against doing that with unstable values?
  • Should it mention anything else than java Enums?

Case Techniques

Case is documented for accepting all readable forms as test-constants. However, it can also be made to use any compile-time-known constants as test-constants, by wrapping it in another macro.

Sometimes this is appropriate, e.g. when dispatching on a java Enum.
Other times, less so, e.g. when dispatching on objects whose hash changes when the vm is restarted (breaks AOT).

Implications

This technique is an application of a more general technique: Passing non-literals to a macro from another macro.
Are there other macros that have use cases like this?

References

https://groups.google.com/d/topic/clojure/3yGjDO2YnjQ/discussion



 Comments   
Comment by Herwig Hochleitner [ 02/Mar/14 11:25 AM ]

This is a duplicate of http://dev.clojure.org/jira/browse/CLJ-1367

Actually, it's an alternate solution





[CLJ-1367] Allow case statement to compare java constants Created: 02/Mar/14  Updated: 02/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Adam Clements Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: interop


 Description   

As raised on the mailing list: https://groups.google.com/forum/#!topic/clojure/3yGjDO2YnjQ

It's not possible to use java constants in a case statement. condp = could be used in this case but these are things which could be used in a java switch statement and so it's annoying to give up constant time dispatch. For example:

(case (.getActionMasked event)
MotionEvent/ACTION_POINTER_DOWN :down
MotionEvent/ACTION_UP :up
MotionEvent/ACTION_POINTER_UP :up
MotionEvent/ACTION_MOVE :move
MotionEvent/ACTION_CANCEL :cancel
MotionEvent/ACTION_OUTSIDE :outside
:none))

Doesn't work, but there is no reason this couldn't be resolved at compile time and dispatched in constant time.



 Comments   
Comment by Herwig Hochleitner [ 02/Mar/14 11:32 AM ]

Another solution for this problem: http://dev.clojure.org/jira/browse/CLJ-1368





[CLJ-1364] Primitive VecSeq does not implement equals or hashing methods Created: 19/Feb/14  Updated: 19/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: collections


 Description   

VecSeq (as produced by (seq (vector-of :int 1 2 3))) does not implements equals, hashCode, or hasheq and does not play with any other Clojure collections or sequences appropriately in this regard.

user=> (def rs (range 3))
user=> (def vs (seq (vector-of :int 0 1 2)))
user=> rs
(0 1 2)
user=> vs
(0 1 2)
user=> (.equals rs vs)
true
user=> (.equals vs rs)    ;; expect: true
false
user=> (.equiv rs vs)
true
user=> (.equiv vs rs)
true
user=> (.hashCode rs)
29824
user=> (.hashCode vs)     ;; expect to match (.hashCode rs)
2081327893
user=> (System/identityHashCode vs)  ;; show that we're just getting Object hashCode
2081327893
user=> (.hasheq rs)
29824
user=> (.hasheq vs)       ;; expect same as (.hasheq rs) but not implemented at all
IllegalArgumentException No matching field found: hasheq for class clojure.core.VecSeq  clojure.lang.Reflector.getInstanceField (Reflector.java:271)





[CLJ-1360] clojure.string/split strips trailing delimiters Created: 18/Feb/14  Updated: 18/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tim McCormack Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

clojure.string/split and clojure.string/split-lines inherit the bizarre default behavior of java.lang.String#split(String,int) in stripping trailing consecutive delimiters:

(clojure.string/split "banana" #"an")
⇒ ["b" "" "a"]
(clojure.string/split "banana" #"na")
⇒ ["ba"]
(clojure.string/split "nanabanana" #"na")
⇒ ["" "" "ba"]

In the case of split-lines, processing a file line by line and rejoining results in truncation of trailing newlines in the file. In both cases, the behavior is surprising and cannot be inferred from the docstrings.

This behavior should either be fixed or documented.



 Comments   
Comment by Andy Fingerhut [ 18/Feb/14 10:51 AM ]

Probably documenting would be safer than changing the behavior at this point, given that some people may actually rely on the current behavior after testing, deploying, etc.

I don't currently have a suggestion for a modified doc string, but note that there are examples of this behavior and how one can use an extra "-1" limit argument at the end to get all split strings: http://clojuredocs.org/clojure_core/clojure.string/split





[CLJ-1347] finalize won't work in reified objects - document Created: 10/Feb/14  Updated: 01/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5, Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Jozef Wagner Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

java 7



 Description   

Finalize is called for reified objects even when they are still reachable. It gets called second time at proper time.

user=> (def x (reify Object (finalize [o] (println "OH MY!"))))
#'user/x
user=> (System/gc)
nil
OH MY!
user=> x
#<user$reify__1496 user$reify__1496@53fb35af>
user=> (System/gc)
nil
user=> (def x nil)
#'user/x
user=> (System/gc)
nilOH MY!

Deftype seems to work fine

user=> (deftype T [] Object (finalize [o] (println "great success")))
user.T
user=> (def y (->T))
#'user/y
user=> (System/gc)
nil
user=> (def y nil)
#'user/y
user=> (System/gc)
great success


 Comments   
Comment by Alex Miller [ 10/Feb/14 8:38 AM ]

Just a note: the calls to System/gc don't necessarily cause finalizers to run on the first try - sometimes it took more than one for that to succeed for me. You'd think System/runFinalizers would do it but I had no luck at all with that.

Comment by Gary Fredericks [ 13/Feb/14 10:01 PM ]

reify actually creates two objects – the first is created by reify*, and then reify immediately calls with-meta on it, creating a copy.

The docstring sort of describes this behavior: "reify always implements clojure.lang.IObj and transfers meta data of the form to the created object."

Comment by Jozef Wagner [ 14/Feb/14 5:01 AM ]

Oh, so finalizer is a no-no in reify. Should be mentioned in docs IMO.

Comment by Gary Fredericks [ 14/Feb/14 6:28 AM ]

Just for fun you could do something tricksy like:

^::second-object
(reify Object
  (finalize [self]
    (when (::second-object (meta self))
      ...)))

(have not actually run this)

Comment by Gary Fredericks [ 01/Mar/14 1:36 PM ]

It looks like the class generated by reify always has a constructor that takes a metadata argument, so it doesn't seem out of the question to eliminate the extra object altogether.

I'll try to keep digging on this.





[CLJ-1346] clojure.core.VecSeq does not implement method equals Created: 09/Feb/14  Updated: 09/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5, Release 1.6
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

.equals is asymmetric for seq's of primitive vectors and PersistentVectors, because clojure.core.VecSeq does not implement Java's equals method. It implements equiv, so clojure.core/= is fine:

user=> (def v1 [1 2 3])
#'user/v1
user=> (def v2 (vector-of :long 1 2 3))
#'user/v2
user=> (= v1 v2)
true
user=> (.equals v1 v2)
true
user=> (= (seq v1) (seq v2))
true
user=> (.equals (seq v1) (seq v2))
true
user=> (= v2 v1)
true
user=> (.equals v2 v1)
true
user=> (= (seq v2) (seq v1))
true
;; This is the one that is not like the others, and a symptom of the problem
user=> (.equals (seq v2) (seq v1))
false





[CLJ-1342] Byte comparison boxes both bytes and converts to longs to compare (which is slow) Created: 06/Feb/14  Updated: 06/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: compiler

Attachments: File bytebox.clj    

 Description   

This came up in a much more complicated example but consider a case like this:

(defn simple []
  (let [b (byte-array [(byte 0)])
        m (byte 0)]
    (= m (aget b 0))))

In the compiled bytecode, both m and (aget b 0) are known to be bytes, but both are boxed using Byte.valueOf(), then cast using RT.uncheckedLongCast() and finally compared as longs:

26: iload_2
  27: invokestatic  #69  // Method java/lang/Byte.valueOf:(B)Ljava/lang/Byte;
  30: checkcast     #81  // class java/lang/Number
  33: invokestatic  #85  // Method clojure/lang/RT.uncheckedLongCast:(Ljava/lang/Object;)J

In a tight loop manipulating and matching against byte arrays, this boxing is significant for performance.

Attached is a test that demonstrates the performance difference between the byte[] and long[] performance to get an idea of the difference.



 Comments   
Comment by Nicola Mometto [ 06/Feb/14 9:10 PM ]

The description states that Util.equiv() has a byte/byte comparison variant but it doesn't look like it actually exists.

Comment by Nicola Mometto [ 06/Feb/14 9:17 PM ]

By the way, tools.emitter.jvm uses i2l to cast the byte to a long instead of boxing && unboxing to a long

Comment by Alex Miller [ 06/Feb/14 9:39 PM ]

Thanks Nicola - I must have confused it with the boolean/boolean version.





[CLJ-1333] Documentation for "=" is misleading Created: 30/Jan/14  Updated: 14/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: George Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docs
Environment:

linux 2.6.32-431.el6.x86_64



 Description   

Document for clojure.core/= says it compares numbers in a type-independent manner. In reality the comparission is made in a type dependent manner. If the above statement was true then (= 1 1.0) would eval to true not false;

clojure.core/=
([x] [x y] [x y & more])
Equality. Returns true if x equals y, false if not. Same as
Java x.equals except it also works for nil, and compares
numbers and collections in a type-independent manner. Clojure's immutable data
structures define equals() (and thus =) as a value, not an identity,
comparison.



 Comments   
Comment by Kevin Downey [ 02/Feb/14 4:58 PM ]

I think this is a little more complex than described.

= does compare things in a jvm type independent manner, but it does use what people have taken to calling "equality classes"

(= [1 2] '(1 2))

(= {:a 1} (doto (java.util.HashMap.) (.put :a 1)))

etc.

now for numbers, it seems logical to me, to have floating point and precise numbers in distinct equality classes

in which case, 1.0 and 1 are in distinct equality classes, so not equal.

Comment by Gary Fredericks [ 13/Feb/14 10:16 PM ]

The docstring is definitely misleading for people unfamiliar with this sort of thing though. Numbers are probably the first thing that the words "type independent manner" bring to mind. A brief pointer to the == function might be useful.

Comment by George [ 14/Feb/14 5:47 AM ]

I find == function to be confusing
For example
(== 1 1.0) => true
(== 1 1.0M) => false ; what is wrong with this comparison?

and doc says:
Returns non-nil if nums all have the equivalent value (type-independent)

Comment by Alex Miller [ 14/Feb/14 7:56 AM ]

@George - that last example (== 1 1.0M) is actually a bug that is fixed in 1.6 where it will return true.





[CLJ-1332] Exceptions are not cached in lazy seqs Created: 29/Jan/14  Updated: 13/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Yongqian Li Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

It is confusing that exceptions will only be thrown once when it is possible to iterate over a seq many times.

user=> (def a (for [i (reverse (range 2))] (/ 1 i)))
#'user/a
user=> (println a)

ArithmeticException Divide by zero clojure.lang.Numbers.divide
(Numbers.java:156)
(user=> (println a)
(1)
nil
user=> (println a)
(1)
nil



 Comments   
Comment by Gary Fredericks [ 13/Feb/14 10:29 PM ]

The cause of this is the lazy-seq macro which uses :once metadata to signal to the compiler that the thunk it creates will only be called once.

When the evaluation of a lazy seq throws an exception, trying to walk the seq again causes the function to be called a second time. Since its closed over values have likely been cleared by that point, you get different behavior.

Glancing at LazySeq.java made me pretty convinced you can't cache exceptions without adding an extra check somewhere in the standard codepath for lazy seq traversal.

Comment by Yongqian Li [ 13/Feb/14 11:38 PM ]

Btw, I ran into this issue while trying to evaluate a lazy-seq in a future in order to do some processing concurrently in the background. Any suggestions for workarounds?





[CLJ-1327] Clojure Primitives extend Serializable without serialVersionUID Created: 20/Jan/14  Updated: 20/Jan/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Kuldeep Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux



 Description   

Clojure keywords for instance are serializable but do not define a serialVersionUID.






[CLJ-1326] Inconsistent reflection warnings when target is a literal Created: 19/Jan/14  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: errormsgs


 Description   
user=> (set! *warn-on-reflection* true)
true
user=> (.get {} 0)
Reflection warning, NO_SOURCE_PATH:2:1 - call to get can't be resolved.
nil
user=> (.get {1 1} 0)
nil
user=> (.get ^:foo {1 1} 0)
Reflection warning, NO_SOURCE_PATH:4:1 - call to get can't be resolved.
nil
user=> (.get {1 (inc 0)} 0)
Reflection warning, NO_SOURCE_PATH:5:1 - call to get can't be resolved.
nil

Similar issues apply to other literals (vector literals, list literals)






[CLJ-1323] AsmReflector throws exceptions on JDK8 Created: 13/Jan/14  Updated: 23/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File clj-1323-disable.diff    

 Description   

After the commit of the updated ASM library for CLJ-713, Clojure builds and passes all tests except for one, compare-reflect-and-asm in reflect.clj.

This can be narrowed down somewhat to a difference in behavior of the following 2 forms evaluated with the latest Clojure and JDK8:

;; The following two lines work with the latest (Jan 11 2014) Clojure 1.6.0-master-SNAPSHOT
;; if run on JDK 6 or JDK 7, but throw an exception with JDK 8.

(import '[clojure.asm ClassReader ClassVisitor Type Opcodes])
(def r (ClassReader. "java.lang.Object"))

I am not certain, but from a bit of Google searching it appears that this may be a limitation of the ASM library version 4 – it throws exceptions when attempting to read class files produced by JDK 8, because of a newer classfile version number. Links that seem to support this conclusion:

http://mail-archive.ow2.org/asm/2013-02/msg00000.html

http://forge.ow2.org/tracker/index.php?func=detail&aid=316375&group_id=23&atid=350023

A couple of alternatives are:

(1) update ASM again to one that supports JDK 8 class files

(2) disable the compare-reflect-and-asm test. Clojure itself does not use the AsmReflector for anything except this unit test. The Java reflector is the default one.



 Comments   
Comment by Alex Miller [ 13/Jan/14 8:16 AM ]

1) There is no released ASM that supports JDK 8 yet. ASM 5 will but it will not be final till JDK 8 is in the final stages of release.

2) Probably more likely.

Comment by Nicola Mometto [ 19/Mar/14 9:01 AM ]

As of now, both JDK8 and ASM5 are out.
I just tried compiling clojure on JDK8 with ASM5 and all compiles fine

Comment by Alex Miller [ 19/Mar/14 9:07 AM ]

How are you running this test?

Comment by Nicola Mometto [ 19/Mar/14 9:11 AM ]

I downloades ASM5, replaced the bundled ASM that comes with clojure with that one after changing che package name to "clojure.asm" and run `mvn install`, all the tests pass.

Comment by Alex Miller [ 19/Mar/14 9:24 AM ]

I was actually talking about the JDK 8 change - was curious about exactly what was being changed?

Comment by Alex Miller [ 19/Mar/14 10:04 AM ]

In particular, I'm assuming that you're not altering the build.xml to change the compilation -source or -target and running with JAVA_HOME / path set to JDK 8.

We don't have any plans to actually build Clojure with JDK 8 any time soon, so I'm not overly concerned about that. But it does appear that the embedded ASM 4 cannot read newer class files from JDK 8. Afaik, the only place that happens is in clojure.reflect.java in the AsmReflector, which is not the default reflector. The JavaReflector will properly reflect the Java 8 classes.

ASM 5 has only been out a couple days and already has at least one serious bug reported - I'd like that to see more use before we switch to it, so maybe this is a good target for the release after Clojure 1.6.

Comment by Alex Miller [ 23/Mar/14 12:17 PM ]

Patch to temporarily disable the failing test until we have an ASM that supports JDK 8.





[CLJ-1322] doseq with several bindings causes "ClassFormatError: Invalid Method Code length" Created: 10/Jan/14  Updated: 10/Jan/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Miikka Koskinen Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Clojure 1.5.1, java 1.7.0_25, OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.12.04.2)


Approval: Triaged

 Description   
user=> (def a1 (range 10))
#'user/a1
user=> (doseq [x1 a1 x2 a1 x3 a1 x4 a1 x5 a1 x6 a1 x7 a1 x8 a1] (do))
CompilerException java.lang.ClassFormatError: Invalid method Code length 69883 in class file user$eval1032, compiling:(NO_SOURCE_PATH:2:1)

While this example is silly, it's a problem we've hit a couple of times. It's pretty surprising when you have just a couple of lines of code and suddenly you get the code length error.






[CLJ-1321] Documentation improvement for clojure.walk, to note use of recursion that can easily blow the JVM stack Created: 09/Jan/14  Updated: 09/Jan/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Lee Spector Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: documentation
Environment:

JVM



 Description   

prewalk and postwalk are use recursion in ways that will blow the stack for sufficiently deep nested structures.

I suggest that this be noted in various clojure.walk documentation strings since this kind of recursion/limit seems to be rare in Clojure, and hence will be unexpected.

(It'd be even better to remove the recursion/limit via something like zippers and iteration, but this issue is just a suggestion for improvement of the documentation.)






[CLJ-1316] for doesn't support :let binding as its first seq-expr Created: 30/Dec/13  Updated: 16/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Jay Fields Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

jvm clojure


Approval: Triaged

 Description   
user> (for [y [2 3 4] 
            :let [x 1]]
        [x y])
([1 2] [1 3] [1 4])
user> (for [:let [x 1]
            y [2 3 4]]
        [x y])
IllegalStateException Can't pop empty vector  clojure.lang.PersistentVector.pop (PersistentVector.java:380)

Cause:

Solution:

Patch:
Screened by:



 Comments   
Comment by Andy Fingerhut [ 30/Dec/13 9:53 AM ]

Related (perhaps identical?) ticket CLJ-207 was declined.

Comment by Jay Fields [ 30/Dec/13 10:03 AM ]

It does look like a duplicate. I find it surprising that this doesn't work, but it does work for doseq:

main=> (doseq [:let [x 1] y [2 3 4]] (println x y))
1 2
1 3
1 4
nil

I think you'll keep getting this bug report as long as that inconsistency exists.

Comment by Jay Fields [ 30/Dec/13 10:05 AM ]

for completeness, I think it's worth mentioning that I can't simply change the ordering (like Alex's example above), due to the cost of the value I'm calculating. I only want it to occur once, and I have to use a separate 'let (as Rich recommended)

Comment by Gary Fredericks [ 05/Jan/14 3:37 PM ]

Brandon Bloom pointed out that one difference between for and doseq is that for is lazy, and so for an initial :let it's not clear whether it should be evaluated immediately or after the first item is requested. doseq doesn't have that ambiguity.

Comment by Jay Fields [ 08/Jan/14 10:42 AM ]

@Gary, I think that's a good question, but either choice would be better than the current inconsistency. If you made it lazy, I can't really think of a downside. Even if it wasn't lazy, that would match the current performance characteristics of code that's already wrapping the for in a let.





[CLJ-1311] gen-interface uses DynamicClassLoader when not compiling, gen-class doesn't Created: 20/Dec/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Joel Kaasinen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring, gen-class, interop


 Description   

The documentation for both gen-class and gen-interface says: "When not compiling, does nothing."

However, gen-interface does the right thing and uses DynamicClassLoader.defineClass when not compiling. This means e.g. that gen-interface works from the repl.

I don't see a reason why gen-class couldn't do the same. Obviously, the docstrings would need to be updated too.






[CLJ-1309] Bindings after :as in list destructuring should throw error Created: 19/Dec/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4, Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: ben wolfson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Compiler, errormsgs


 Description   

If you try to define a vector binding with anything at all after an :as parameter, you do not get a compiler error, and the binding is silently swallowed:

user> ((fn [[:as y z]] y) [1 2])
[1 2]

If you try to actually use the binding, there will be a compiler error (the compiler will complain that there's no binding for the symbol), but the actual error has already happened, and should be reported earlier.






[CLJ-1308] extend-type doesn't type-hint correctly as promised by the doc when the class is determined at run-time Created: 15/Dec/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: typehints


 Description   

extend-type works with non-constant expressions as its type:

(extend-type (class 1) proto (protof [this]))

However, in this case this will get tagged with `(class 1)`, which is clearly wrong; the doc explicitely states that the args will be proberly type-hinted: "[..] Propagates the class as a type hint on the first argument of all fns."

I don't know if extend-type is not supposed to work with non-constant Classes, in which case it should be stated in the doc or if the current behaviour is wrong.






[CLJ-1305] Add optional not-found argument when invoking vectors or sets as functions Created: 12/Dec/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Dave Tenny Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Maps, keywords, and symbols when used as operators allow optional second arguments for 'default-not-found' values is if to 'get'.

({:a 1} :b 'b) => b

However sets don't support this behavior (though they do with 'get') and vectors don't allow the optional default-not-found in their pseudo 'nth' semantics.

user=> (#{:a  :b} :b 'notfound)
ArityException Wrong number of args (2) passed to: PersistentHashSet  clojure.lang.AFn.throwArity (AFn.java:437)





[CLJ-1300] take-while with n<1 behaves like (repeat 1) Created: 22/Nov/13  Updated: 22/Nov/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.1, Release 1.2, Release 1.3, Release 1.4, Release 1.5, Release 1.6
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Yaron Peleg Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
user=> (take-nth -1 [1 3 4])
; hangs
user=> (take 5 (take-nth -1 [5 9 14]))
(1 1 1 1 1)

I understand this behavior may be intentionally undefined,
but raising the issue on IRC didn't yield an answer on whether
this is a bug or grey area.






[CLJ-1298] Add more type predicate fns to core Created: 21/Nov/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5, Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alex Fowler Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Add more built-in type predicates:

1) Definitely missing: (atom? x), (ref? x), (deref? x), (named? x), (map-entry? x), (lazy-seq? x).
2) Very good to have: (throwable? x), (exception? x), (pattern? x).

The first group is especially important for writing cleaner code with core Clojure.



 Comments   
Comment by Alex Miller [ 21/Nov/13 8:42 AM ]

In general many of the existing predicates map to interfaces. I'm guessing these would map to checks on the following types:

atom? = Atom (final class)
ref? = IRef (interface)
deref? = IDeref (interface)
named? = Named (interface, despite no I prefix)
map-entry? = IMapEntry (interface)
lazy-seq? = LazySeq (final class)

throwable? = Throwable
exception? = Exception, but this seems less useful as it feels like the right answer when you likely actually want throwable?
pattern? = java.util.regex.Pattern

Comment by Alex Fowler [ 21/Nov/13 9:02 AM ]

Yes, they do, and sometimes the code has many checks like (instance? clojure.lang.Atom x). Ok, you can write a little function (atom? x) but it has either to be written in all relevant namespaces or required/referred there from some extra namespace. All this is just a burden. For example, we have predicates like (var? x) or (future? x) which too map to Java classes, but having them abbreviated often makes possible to write a cleaner code.

I feel the first group to be especially significant for it being about core Clojure concepts like atom and ref. Having to fall to manual Java classes check to work with them feels inorganic. Of course we can, but why then do we have (var? x), (fn? x) and other? Imagine, for example:

(cond
(var? x) (...)
(fn? x) (...)
(instance? clojure.lang.Atom x) (...)
(or (instance? clojure.lang.Named x) (instance? clojure.lang.LazySeq x)) (...))

vs

(cond
(var? x) (...)
(fn? x) (...)
(atom? x) (...)
(or (named? x) (lazy-seq? x)) (...))

The second group is too, essential since these concepts are fundamental for the platform (but you're right with the (exception? x) one).

Comment by Alex Fowler [ 22/Nov/13 6:35 AM ]

Also, obviously I missed the (boolean? x) predicate in the original post. Did not even guess it is absent too until I occasionally got into it today. Currently the most clean way we have is to do (or (true? x) (false? x)). Needles to say, it looks weird next to the present (integer? x) or (float? x).





[CLJ-1296] locking expressions cause vars to be dereferenced, even if not executed, unless wrapped in let Created: 17/Nov/13  Updated: 17/Nov/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: performance


 Description   

Description of one example with poor performance discovered by Michał Marczyk in the discussion thread linked below.

The difference between the compiled versions of:

(defn foo [x]
  (if (> x 0)
    (inc x)
    (locking o
      (dec x))))

and

(defn bar [x]
  (if (> x 0)
    (inc x)
    (let [res (locking o
                (dec x))]
      res)))

is quite significant. foo gets compiled to a single class, with invocations handled by a single invoke method; bar gets compiled to a class for bar + an extra class for an inner function which handles the (locking o (dec x)) part – probably very similar to the output for the version with the hand-coded locking-part (although I haven't really looked at that yet). The inner function is a closure, so calling it involves an allocation of a closure object; its ctor receives the closed-over locals as arguments and stores them in two fields (lockee and x). Then they get loaded from the fields in the body of the closure's invoke method etc.

Note: The summary line may be too narrow a description of the root cause, and simply the first example of a case where this issue was noticed and examined. Please make the summary and this description more accurate if you diagnose this issue.

See discussion thread on Clojure group here: https://groups.google.com/forum/#!topic/clojure/x86VygZYf4Y






[CLJ-1292] print docstring should specify nil return value Created: 01/Nov/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Phill Wolf Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring, print


 Description   

The docstring for print does not mention its return value. The docstring should clarify whether print dependably returns nil or shouldn't be depended on to (lest, for example, something leak out as the inadvertent return value of print's caller).






[CLJ-1291] struct-map docstring incomplete, inconsistent Created: 01/Nov/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Phill Wolf Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring


 Description   

The docstring for struct-map refers to "structure-basis" and "keyvals" while the parameters are "s" and "inits". The docstring says "keyvals can also contain keys not in the basis" but does not say what happens in that case.






[CLJ-1290] clojure.xml parse docstring omits InputSource Created: 01/Nov/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Phill Wolf Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring, xml


 Description   

The clojure.xml parse docstring mentions that parameter s "can be a File, InputStream or String naming a URI." But those choices do not cover a common case, parsing the value of a String. Actually, parse also allows InputSource, which solves the problem. The docstring should mention InputSource (or clarify its omission, if not inadvertent).

user> (use '[clojure.xml :as xml])
nil
user> (import '[java.io StringReader])
java.io.StringReader
user> (import '[org.xml.sax InputSource])
org.xml.sax.InputSource
user> (xml/parse (InputSource. (StringReader. "<egg>green</egg>")))
{:tag :egg, :attrs nil, :content ["green"]}





[CLJ-1286] Fix reader spec and regex to match code for keywords starting with digits Created: 31/Oct/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reader


 Description   

The reader page at http://clojure.org/reader states that symbols (and keywords) cannot start with a number and the regex used in LispReader (and EdnReader) also has this intention. CLJ-1252 addressed this by fixing the broken reader regex to match the spec. However, that broke some existing code so we rolled back the change. There is still a disconnect here and this ticket serves to decide what to do instead.

I presume that we are effectively deciding that keywords like :5 are ok to read. If so, we should alter the regex to more accurately capture that intent - right now it allows these purely by accident due to backtracking. A secondary question is whether the Clojure and EDN reader spec should also explicitly allow these as valid. My preference would be to have the reader and the spec match, so I would lobby to loosen the reader spec.



 Comments   
Comment by Nicola Mometto [ 12/Nov/13 4:50 PM ]

what about keywords like :1/1 or :1/a? Clojure currently accepts the latter but not the former.





[CLJ-1284] Clojure functions and reified objects should expose a public static field to identify their proper Clojure name Created: 24/Oct/13  Updated: 25/Oct/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs


 Description   

There are several examples of frameworks that attempt to de-mangle a Java class name into a Clojure symbol (including namespace); this is useful for writing out an improved, Clojure-specific stack trace when reporting exceptions.

Existing libraries are based on regular expression matching and guesswork, and can occasionally give incorrect results, such as when a namespace or function name actually contains an underscore.

It would be helpful for authors of such frameworks if Clojure would expose a static final field on such classes with the proper name that should appear in the stack trace; libraries would then be able to use reflection to access the proper name of the field, without the current guesswork.

I would suggest CLOJURE_SOURCE_NAME as a reasonable name for such a field.

Other Clojure class constructs beyond functions, such as reified types and protocol implementations, would also benefit, though it is less obvious what exact string value would properly and unambiguously identify what purpose the class plays.



 Comments   
Comment by Alex Miller [ 24/Oct/13 8:31 PM ]

FYI, there is a patch on the way in for 1.6 that contains a new demunge function in Compiler. However, the munged name is not always reversible so having the original around is a good idea.

Comment by Andy Fingerhut [ 25/Oct/13 11:10 AM ]

The patch Alex is referring to is attached to CLJ-1083.

Comment by Andy Fingerhut [ 25/Oct/13 11:13 AM ]

Howard, there seems to be some overlap in the intent between this ticket and CLJ-1278. I guess either of them could be done without the other, but wanted to check.





[CLJ-1280] Create reusable exception that can carry file/line/col info Created: 18/Oct/13  Updated: 18/Oct/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

This concept already exists in multiple places in Clojure - Compiler$CompilerException and the Exception classes buried in EdnReader and LispReader. It would also be useful in other places where IllegalArgument or other other exceptions are thrown.

For example, this protocol exception throws an IllegalArgumentException and could transmit the file, line, and column info at the location of the error but it seems weird to use any of the existing exceptions for this purpose.

(defprotocol Bar (m [this]) (m [this arg]))





[CLJ-1276] Can't make a dispatch map containing forward-declared fns Created: 09/Oct/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Coventry Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File unbound-eg.tgz    

 Description   

If from (ns tst2) you try to call tst1/c, which calls tst1/f via dispatch map which was defined when tst1/f was forward declared, you get an "unbound fn" error. E.g.

user=> (dorun (map eval 
                   '[(ns tst1) 
                     (declare f) 
                     (def d {:k f}) 
                     (defn c [] ((d :k)))
                     (defn f [] :success)
                     (ns tst2 (:require [tst1]))
                     (tst1/c)]))

IllegalStateException Attempting to call unbound fn: #'tst1/f  clojure.lang.Var$Unbound.throwArity (Var.java:43)
tst2=> (clojure.repl/pst *e)
IllegalStateException Attempting to call unbound fn: #'tst1/f
	clojure.lang.Var$Unbound.throwArity (Var.java:43)
	tst1/c (NO_SOURCE_FILE:5)
	tst2/eval25 (NO_SOURCE_FILE:8)
	clojure.lang.Compiler.eval (Compiler.java:6642)
	clojure.lang.Compiler.eval (Compiler.java:6605)
	clojure.core/eval (core.clj:2883)
	clojure.core/map/fn--4222 (core.clj:2513)
	clojure.lang.LazySeq.sval (LazySeq.java:40)
	clojure.lang.LazySeq.seq (LazySeq.java:49)
	clojure.lang.RT.seq (RT.java:484)
	clojure.core/seq (core.clj:133)
	clojure.core/dorun (core.clj:2811)


 Comments   
Comment by Alex Coventry [ 09/Oct/13 10:43 PM ]

TEttinger pointed out on IRC that the forms in the example run without error if you wrap them in a (do) block. Here is an example using files. Relevant code is in src/unbound_eg/tst[12].clj. Example output shown below.

http://clojure-log.n01se.net/date/2013-10-09.html#23:52

lap% lein repl
nREPL server started on port 50125 on host 127.0.0.1
REPL-y 0.2.1
Clojure 1.5.1
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)

user=> (require '[unbound-eg.tst2 :as t2])

IllegalStateException Attempting to call unbound fn: #'unbound-eg.tst1/f clojure.lang.Var$Unbound.throwArity (Var.java:43)
user=> (pst)
IllegalStateException Attempting to call unbound fn: #'unbound-eg.tst1/f
clojure.lang.Var$Unbound.throwArity (Var.java:43)
unbound-eg.tst1/c (tst1.clj:4)
unbound-eg.tst2/eval2233 (tst2.clj:3)
clojure.lang.Compiler.eval (Compiler.java:6619)
clojure.lang.Compiler.load (Compiler.java:7064)
clojure.lang.RT.loadResourceScript (RT.java:370)
clojure.lang.RT.loadResourceScript (RT.java:361)
clojure.lang.RT.load (RT.java:440)
clojure.lang.RT.load (RT.java:411)
clojure.core/load/fn--5018 (core.clj:5530)
clojure.core/load (core.clj:5529)
clojure.core/load-one (core.clj:5336)
nil





[CLJ-1272] Agent thread executors do not use the global uncaught exception handler Created: 01/Oct/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: David Greenberg Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: agents


 Description   

If you use Thread.setDefaultUncaughtExceptionHandler to catch all application exceptions, and then throw an exception in a future, that exception will get swallowed up in deployment environments that don't watch stdout. It seems that the agent's executors ought to delegate to the global handler.

This issue bit us, in that we deploy and monitor our system only through its logs and metrics, and never actually saw that exceptions were being thrown in futures.






[CLJ-1263] Allow static compilation of function invocations Created: 14/Sep/13  Updated: 07/Nov/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Anderson Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: compiler


 Description   

This proposal is to allow metadata on functions to prevent a fully dynamic var deref to be used whenever the function is called.

When the function is invoked, JVM "invokevirtual" instruction will be used, which is faster than the current implementation (var deref + IFn cast + invokinterface) and has less restrictions (no need to predefine interfaces to match the function parameters). The JVM is generally able to compile such invokevirtual instructions into extremely efficient code - effectively as fast as pure Java.

This is intended to pave the way to better support for statically compiled, high performance code. In particular, it allow:

  • Supporting arbitrary JVM primitives (float, int, byte, char etc.) as well as just double/long.
  • Supporting typed return values e.g. "String". This could eliminate many casts and type checks.
  • Supporting typed reference arguments (e.g. String).

Suggested usage:

(defn ^:static foo ^int [^String a ^String b]
(+ (count a) (Integer/parseInt b)))

Existing code / semantics should not be affected



 Comments   
Comment by Alex Fowler [ 18/Sep/13 5:08 AM ]

Very nice! That is what would really improve experience with certain tasks. I think it will also make possible to work with primitive arrays without the conversions?

Comment by Mike Anderson [ 19/Sep/13 5:44 PM ]

Hi Alex - which aspect of "work with primitive arrays" are you referring to? This feature would certainly help with passing primitive arguments to/from functions that use primitive arrays. It would also potentially help avoid some casts of primitive array arguments themselves. I don't think it helps in any other way - perhaps a separate issue would be appropriate if there is another thing you are trying to do?

Comment by Kevin Downey [ 29/Oct/13 11:50 AM ]

this issue is confusing, because there was/is a :static feature in clojure(which seems to be disabled in the compiler at the moment) and this proposal doesn't mention the existing work at all.

I also think this proposal is begging the question, there is no discussion of other possible solutions to the performance problem (whatever that is) that this is trying to solve.

the (var.deref()(IFn)).invoke(...) is pretty fundamental to the feel of clojure, in fact the existing :static keyword seems to be disabled in the compiler exactly because it complicates those semantics. so we should have a very clear proposal (not a wishlist) if we want to change that with some very clear wins.

maybe an optimizing clojure compiler would be a better approach.

Comment by Mike Anderson [ 30/Oct/13 11:01 PM ]

Hi Kevin,

This is partly in response to this discussion on Clojure-Dev, where we discussed there are quite a lot of performance issues around the way that Clojure passes arguments currently:
https://groups.google.com/d/topic/clojure/H5P25eYKBj4/discussion

Also I believe it reinstates the original intention of "^:static": I can't find where this is/was officially documented, but Arthur's answer in this SO question suggests that this was the case:
http://stackoverflow.com/questions/7552632/what-does-static-do-in-clojure

I think the proposal is relatively clear: it's probably the minimal change required to get static/direct (i.e. not via an indirect var reference / IFn) function invocations without affecting any of the semantics of current code.

This is sufficiently important for me that it's preventing me from shifting some performance-critical code to Clojure (even with primitive type hints). e.g. here's a simple case with a small primitive function:

(defn ^long foo [^long x]
(inc x))

(c/quick-bench (dotimes [i 100] (foo i))) ;; c = criterium
=> Execution time mean : 194.334293 ns

(c/quick-bench (dotimes [i 100] (inc i)))
=> Execution time mean : 71.539048 ns

i.e. the indirect function invocation is costing us nearly 170% overhead. In Java the equivalent functions perform identically: the overhead is zero because with static function invocation the JVM JIT is able to eliminate all the function call overhead.

In the long term, I agree that a proper optimising compiler would be the best way forward (perhaps Clojure 2.0/CinC can give us this?) but in the meantime I think this is a pragmatic way to improve performance with minimal impact on existing code. Even with an optimising compiler, I think we' would need some way to specifiy the "optimised" semantics rather than the indirect var deref behaviour, and "^:static" seems like a reasonable way to do so (unless anyone has a better idea?)

Comment by Kevin Downey [ 04/Nov/13 3:58 PM ]

have you looked at the definition of int and how it uses :inline/definline to avoid the call overhead?

Comment by Mike Anderson [ 05/Nov/13 4:27 AM ]

Good point Kevin - :inline and definline seem like a good approach in many cases (although it's marked as "experimental" - does that mean we can't rely on it to work in future releases?).

This proposal is still somewhat different: the inline solutions and its variants are effectively doing macro expansion to generate code without a function call on the Clojure side. The approach in this proposal would still emit a function call in bytecode, but do so in a way that the JVM can subsequently inline and optimise much more efficiently. Both have their uses, I think?

Commented edited Nov 7 2013 by Andy Fingerhut: Regarding definline marked as experimental, it has been so marked since Clojure 1.0's release, and the plan is to keep it marked that way in the pending Clojure 1.6 release. See discussion thread on CLJ-1281. No plans to remove it that I am aware of.

Comment by Kevin Downey [ 06/Nov/13 2:06 PM ]

my point is your benchmark above is not a comparison of clojure's current deref + cast + invoke vs. invokevirtual, inc is being inlined in to a static method call there

Comment by Kevin Downey [ 06/Nov/13 2:32 PM ]

I've been noodling around this, and it is entirely possible to generate and invoke code in clojure right now without paying the extra deref() cost:

 (defn ^long fib [^long n]
   (case n
     0 0
     1 1
     (+ (fib (dec n))
        (fib  (dec (dec n))))))

can be written as

(declare TheR1798)

(definterface I1797
  (^long fib_Invoke_1 [^long n]))

(deftype R1798 []
  I1797
  (^long fib_Invoke_1
    [this1799 ^long n]
    (case n
      0 0
      1 1
      (+ (.fib_Invoke_1 this1799 (dec n))
         (.fib_Invoke_1 this1799 (dec (dec n)))))))

(def TheR1798 (new R1798))

(defn ^long fib [^long n]
  (.fib_Invoke_1 TheR1798  n)))

now the recursive calls are invokeinterfaces, and the resulting function seems to have mean execution time about 5 times smaller using criterium to bench mark

it is entirely possible to write a macro that translates one in to other, and the weird names in the above are because I have a little proof of concept that does that.

the body of the bytecode for the regular fib function first shown looks something like:

  public final java.lang.Object invokePrim(long);
    Code:
       0: lload_1       
       1: lstore_3      
       2: lload_3       
       3: l2i           
       4: tableswitch   { // 0 to 1
                     0: 28
                     1: 40
               default: 52
          }
      28: lconst_0      
      29: lload_3       
      30: lcmp          
      31: ifne          52
      34: getstatic     #33                 // Field const__1:Ljava/lang/Object;
      37: goto          94
      40: lconst_1      
      41: lload_3       
      42: lcmp          
      43: ifne          52
      46: getstatic     #37                 // Field const__3:Ljava/lang/Object;
      49: goto          94
      52: getstatic     #57                 // Field const__5:Lclojure/lang/Var;
      55: invokevirtual #70                 // Method clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
      58: checkcast     #6                  // class clojure/lang/IFn$LO
      61: lload_1       
      62: invokestatic  #75                 // Method clojure/lang/Numbers.dec:(J)J
      65: invokeinterface #77,  3           // InterfaceMethod clojure/lang/IFn$LO.invokePrim:(J)Ljava/lang/Object;
      70: getstatic     #57                 // Field const__5:Lclojure/lang/Var;
      73: invokevirtual #70                 // Method clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
      76: checkcast     #6                  // class clojure/lang/IFn$LO
      79: lload_1       
      80: invokestatic  #75                 // Method clojure/lang/Numbers.dec:(J)J
      83: invokestatic  #75                 // Method clojure/lang/Numbers.dec:(J)J
      86: invokeinterface #77,  3           // InterfaceMethod clojure/lang/IFn$LO.invokePrim:(J)Ljava/lang/Object;
      91: invokestatic  #81                 // Method clojure/lang/Numbers.add:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Number;
      94: areturn       
    LineNumberTable:
      line 243: 0
      line 244: 2
      line 247: 52
      line 247: 52
      line 247: 61
      line 248: 70
      line 248: 79
      line 248: 79
    LocalVariableTable:
      Start  Length  Slot  Name   Signature
             2      92     3 G__301   J
             0      94     0  this   Ljava/lang/Object;
             0      94     1     n   J

  public java.lang.Object invoke(java.lang.Object);
    Code:
       0: aload_0       
       1: aload_1       
       2: checkcast     #89                 // class java/lang/Number
       5: invokestatic  #93                 // Method clojure/lang/RT.longCast:(Ljava/lang/Object;)J
       8: invokeinterface #77,  3           // InterfaceMethod clojure/lang/IFn$LO.invokePrim:(J)Ljava/lang/Object;
      13: areturn       

the body of the "optimized" version looks like:

  public long fib_Invoke_1(long);
    Code:
       0: lload_1       
       1: lstore_3      
       2: lload_3       
       3: l2i           
       4: tableswitch   { // 0 to 1
                     0: 28
                     1: 38
               default: 48
          }
      28: lconst_0      
      29: lload_3       
      30: lcmp          
      31: ifne          48
      34: lconst_0      
      35: goto          80
      38: lconst_1      
      39: lload_3       
      40: lcmp          
      41: ifne          48
      44: lconst_1      
      45: goto          80
      48: aload_0       
      49: checkcast     #6                  // class com/thelastcitadel/kernel/I2364
      52: lload_1       
      53: invokestatic  #53                 // Method clojure/lang/Numbers.dec:(J)J
      56: invokeinterface #55,  3           // InterfaceMethod com/thelastcitadel/kernel/I2364.fib_Invoke_1:(J)J
      61: aload_0       
      62: checkcast     #6                  // class com/thelastcitadel/kernel/I2364
      65: lload_1       
      66: invokestatic  #53                 // Method clojure/lang/Numbers.dec:(J)J
      69: invokestatic  #53                 // Method clojure/lang/Numbers.dec:(J)J
      72: invokeinterface #55,  3           // InterfaceMethod com/thelastcitadel/kernel/I2364.fib_Invoke_1:(J)J
      77: invokestatic  #59                 // Method clojure/lang/Numbers.add:(JJ)J
      80: lreturn       
    LineNumberTable:
      line 251: 0
      line 251: 2
      line 251: 48
      line 251: 48
      line 251: 52
      line 251: 61
      line 251: 65
      line 251: 65
    LocalVariableTable:
      Start  Length  Slot  Name   Signature
             2      78     3 G__2363   J
             0      80     0  this   Lcom/thelastcitadel/kernel/R2365;
             0      80     1     n   J

so the calls are not invokevirtual (due to the way clojure compiles stuff, you cannot type anything inside a record as being that record's type), but the interface is unique and only has one instance, so I think the jvm's class hierarchy analysis makes short work of that.

if I have time I may try and complete my macro and release it as a library, but given tools.analyzer.jvm someone should be able to do better than my little proof of concept very quickly.

Comment by Andy Fingerhut [ 07/Nov/13 12:48 PM ]

I don't know if my editing of Mike Anderson's Nov 5 2013 comment is notified to people watching this ticket, so adding a new comment so those interested in definline's experimental status can know to go back and re-read it.





[CLJ-1256] Support type-hinted overrides of function parameters Created: 06/Sep/13  Updated: 09/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Anderson Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: compiler, interop, typehints


 Description   

Problem: in many cases, the Clojure compiler has enough information about the type of a function argument to statically emit maximally efficient code on the JVM (i.e. without instance? checks, type casts or other forms of dynamic polymorphic dispatch). We are currently unable to do so in Clojure, which pushes developers with strong performance requirements to use some unidiomatic or convoluted workarounds.

Proposal is simply to allow functions to take type-hinted overloads of function arguments, e.g.

(defn foo
([^double x] (Math/floor x))
([^float x] (Math/floor (double x)))
([^String s] (count s)))

An "Object" version of the code with the correct arity will always be emitted, which will maintain compatibility with the IFn interface and ensure that the function can still be used in dynamic / interactive contexts. If the "Object" version is not explicitly provided, then it will be generated to use instance? checks that subsequently delegate to the appropriate typed version of the function (or throw an InvalidArgumentException if no match is found).

Matching rules would be the same as Java.

This will be backwards compatible with all existing uses of defn. In particular, it should extend / enhance / supercede the existing handling of primitive functions.

In the future, this technique might be used alongside core.typed to ensure that the most efficient function version is chosen based on type inference.






[CLJ-1255] Support Abstract Base Classes with "reify" Created: 06/Sep/13  Updated: 06/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Anderson Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: interop


 Description   

Problem:

  • Various Java APIs depend on extension of abstract base classes rather than interfaces
  • "proxy" has limitations (no access to protected fields or super)
  • "proxy" has performance overhead because of an extra layer of functions / parameter boxing etc.
  • "gen-class" is complex and is complected with compilation / bytecode generation

In summary, Clojure does not currently have a good / convenient way to extend a Java abstract base class dynamically.

Proposal is to extend "reify" to allow a single abstract base class as well as interfaces/protocols. Code generation would occur as if the abstract base class had been directly extended in Java (i.e. with full access to protected members and with fully type-hinted fields). This change would be backwards-compatible with all existing uses of "reify".






[CLJ-1243] Cannot resolve public generic method from package-private base class Created: 01/Aug/13  Updated: 17/Nov/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3, Release 1.4, Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Stuart Sierra Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop

Attachments: GZip Archive clj-1243-demo1.tar.gz    

 Description   

The Clojure compiler cannot resolve a public generic method inherited from a package-private base class.

Instructions to reproduce:

  • In package P1
    • Define a package-private class A with generic type parameters
    • Define a public method M in A using generic types in either its arguments or return value
    • Define a public class B which extends A
  • In package P2
    • Construct an instance of B
    • Invoke B.M()

This is valid in Java. In Clojure, invoking B.M produces a reflection warning, followed by the error "java.lang.IllegalArgumentException: Can't call public method of non-public class." No amount of type-hinting prevents the warning or the error.

Attachment clj-1243-demo1.tar.gz contains sample code and script to demonstrate the problem.

Examples of Java projects which use public methods in package-private classes:



 Comments   
Comment by Stuart Sierra [ 01/Aug/13 5:11 PM ]

It is also not possible to call the method reflectively from Java.

This may be a bug in Java reflection: JDK-4283544

But why does it only happen on generic methods?

Comment by Stuart Sierra [ 08/Aug/13 11:59 AM ]

According to Rich Hickey, the presence of bridge methods is unspecified and inconsistent across JDK versions.

A possible solution is to use ASM to examine the bytecode of third-party Java classes, instead of the reflection API. That way the Clojure compiler would have access to the same information as the Java compiler.

Comment by Andy Fingerhut [ 17/Nov/13 11:01 PM ]

CLJ-1183 was closed as a duplicate of this one. Mentioning it here in case anyone working on this ticket wants to follow the link to it and read discussion or test cases described there.





[CLJ-1232] Functions with non-qualified return type hints force import of hinted classes when called from other namespace Created: 18/Jul/13  Updated: 17/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tassilo Horn Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: typehints


 Description   

You can add a type hint to function arglists to indicate the return type of a function like so.

user> (import '(java.util List))
java.util.List
user> (defn linkedlist ^List [] (java.util.LinkedList.))
#'user/linkedlist
user> (.size (linkedlist))
0

The problem is that now when I call `linkedlist` exactly as above from another namespace, I'll get an exception because java.util.List is not imported in there.

user> (in-ns 'user2)
#<Namespace user2>
user2> (refer 'user)
nil
user2> (.size (linkedlist))
CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: List, compiling:(NO_SOURCE_PATH:1:1)
user2> (import '(java.util List)) ;; Too bad, need to import List here, too.
java.util.List
user2> (.size (linkedlist))
0

There are two workarounds: You can import the hinted type also in the calling namespace, or you always use fully qualified class names for return type hints. Clearly, the latter is preferable.

But clearly, that's a bug that should be fixed. It's not in analogy to type hints on function parameters which may be simple-named without having any consequences for callers from other namespaces.



 Comments   
Comment by Andy Fingerhut [ 16/Apr/14 3:47 PM ]

To make sure I understand, Nicola, in this ticket you are asking that the Clojure compiler change behavior so that the sample code works correctly with no exceptions, the same way as it would work correctly without exceptions if one of the workarounds were used?

Comment by Tassilo Horn [ 17/Apr/14 12:18 AM ]

Hi Andy. Tassilo here, not Nicola. But yes, the example should work as-is. When I'm allowed to use type hints with simple imported class names for arguments, then doing so for return values should work, too.





[CLJ-1231] fn and letfn don't support hinting the function's return type Created: 17/Jul/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Tassilo Horn Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: typehints


 Description   

You can add type hints to the argument vector(s) of `defn` to declare the return type of a function like so (with warn-on-reflection being true):

user> (defn foo ^String [s] s)
#'user/foo
user> (.substring (foo "hallo") 1 2)
"a"

But sadly, the same doesn't work with `fn` and `letfn`:

user> (.substring ((fn ^String [s] s) "hallo") 1 2)
Reflection warning, NO_SOURCE_PATH:1:1 - call to substring can't be resolved.
"a"
user> (letfn [(foo ^String [s] s)]
	(.substring (foo "hallo") 1 2))
Reflection warning, NO_SOURCE_PATH:2:7 - call to substring can't be resolved.
"a"

I don't see why this feature is available to `defn` but not to `fn` and `letfn`. I even consider it a kind of defect, because anything else including :pre/:post are also supported by the latter two, so the support for hinting the return type should be there simply because of analogy.






[CLJ-1230] print-table displays tables incorrectly when one of the cells is a string that has newlines Created: 09/Jul/13  Updated: 31/Jan/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Ben Booth Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: print
Environment:

Mac OS X, Clojure 1.5.1



 Description   

When using print-table to print an ASCII table to stdout, the table display breaks if any of the values is a string with any new lines in it. For example:

user=> (print-table [{:a "test" :b "test\ntest2"}])

|   :a |         :b |
|------+------------|
| test | test
test2 |
nil

I would expect the output to look something like this:

user=> (print-table [{:a "test" :b "test\ntest2"}])

|   :a |         :b |
|------+------------|
| test | test       +
|      | test2      |
nil

The + symbol on the right border means that the row continues over multiple lines. This is similar to how the PostgreSQL psql tool displays table with multi-line rows:

user=# select 'test' col1, E'test\ntest2\ntest3' col2;
 col1 | col2  
------+-------
 test | test +
      | test2+
      | test3
(1 row)

Time: 0.776 ms


 Comments   
Comment by Ben Booth [ 09/Jul/13 2:10 PM ]

JIRA destroyed my formatting, and looks like I can't edit it to fix it. Here is what I meant to say:

When using print-table to print an ASCII table to stdout, the table display breaks if any of the values is a string with any new lines in it. For example:

user=> (print-table [{:a "test" :b "test\ntest2"}])

|   :a |         :b |
|------+------------|
| test | test
test2 |

I would expect the output to look something like this:

user=> (print-table [{:a "test" :b "test\ntest2"}])
|   :a |         :b |
|------+------------|
| test | test       +
|      | test2      |

The + symbol on the right border means that the row continues over multiple lines. This is similar to how the PostgreSQL psql tool displays table with multi-line rows:

labtrack=# select 'test' col1, E'test\ntest2' col2;
 col1 | col2  
------+-------
 test | test +
      | test2
(1 row)
Comment by Andy Fingerhut [ 22/Jul/13 2:39 AM ]

I have no direct knowledge of this, but my guess would be that the Clojure team would consider this an enhancement request rather than a defect.

You are likely to get what you want faster either by writing your own version of print-table that works as you wish, or see whether these projects already behave as desired, or the authors are willing to enhance them: https://github.com/cldwalker/table or https://github.com/joegallo/doric





[CLJ-1223] Improve App Engine Support by Providing an Option to Use the App Engine ThreadManger Created: 22/Jun/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: James Thornton Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Google App Engine



 Description   

Clojure support for App Engine can be improved now that App Engine supports sockets and threads.

Sockets
https://developers.google.com/appengine/docs/java/sockets/overview

Threads
https://developers.google.com/appengine/docs/java/javadoc/com/google/appengine/api/ThreadManager

The new sockets API uses java.net.Socket so it should just work without much/any modification to core. The Sockets API is an important addition because it enables you to connect to Compute Engine servers from App Engine without having to use the HTTP client.

Presently Clojure's agents and futures do not work on App Engine, but the new ThreadManager API now allows you to execute short-lived threads; however, you must use the ThreadManger API to create threads – "you cannot invoke new Thread() yourself or use the default thread factory" and "each request is limited to 50 concurrent request threads" – so adding support for threads will require a build option and some modifications.

Supporting agents on App Engine could be done by adding a Clojure build option for App Engine that includes the App Engine JDK dependency and injects the App Engine TreadManager factory into jvm/clojure/lang/Agent.java .

Google App Engine SDK for Java
https://developers.google.com/appengine/downloads#Google_App_Engine_SDK_for_Java

See also https://developers.google.com/appengine/docs/java/runtime#The_Sandbox ...

A Java application can create a new thread, but there are some restrictions on how to do it. These threads can't "outlive" the request that creates them. (On a backend server, an application can spawn a background thread, a thread that can "outlive" the request that creates it.)

An application can...

Implement java.lang.Runnable; and
Create a thread factory by calling com.google.appengine.api.ThreadManager.currentRequestThreadFactory()
call the factory's newRequestThread method, passing in the Runnable, newRequestThread(runnable)
or use the factory object returned by com.google.appengine.api.ThreadManager.currentRequestThreadFactory() with an ExecutorService (e.g., callExecutors.newCachedThreadPool(factory)).

However, you must use one of the methods on ThreadManager to create your threads. You cannot invoke new Thread() yourself or use the default thread factory.

An application can perform operations against the current thread, such as thread.interrupt().

Each request is limited to 50 concurrent request threads.

App Engine JRE Whitelist:
https://developers.google.com/appengine/docs/java/jrewhitelist

From the current App Engine Magic README (https://github.com/gcv/appengine-magic)...

"Google App Engine maintains a whitelist of permitted classes in Java's standard library. Other classes will cause your application to fail to deploy. Examples include threads and sockets. If you use those in your application, it will not work. This means that you cannot use Clojure's agents or futures. In addition, if one of your dependencies uses those, your application will also not work. For example, clojure.java.io (and its fore-runner, duck-streams from clojure-contrib), uses java.net.Socket, a forbidden class."






[CLJ-1221] Should repackage jsr166 and include known version with Clojure Created: 20/Jun/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Unresolved Votes: 0
Labels: reducers


 Description   

Clojure 1.5 reducers work with either the JDK version of forkjoin (JDK 1.7+) or with an external jsr166 jar. This causes complexity for users and complexity in the build to deal with the two options.

jsr166 code is public domain and it is common for other projects to repackage the handful of files and ship it with the project (similar to what we do with asm). This would allow us just use a known existing version of jsr166 across all jdks and we could get rid of the custom build wrangling we introduced in Clojure 1.5.

jsr166y is compatible with JDK 1.6+ and is the version that (for example) Scala currently repackages. That's the best choice for JDK 1.6 and 1.7. In JDK 1.8, the best choice will (temporarily) be the built-in version in java.util.concurrent which tracks jsr166e but then as soon as there are updates will become jsr166e. Many fork/join fixes are ported to both y and e right now.

Some choices here for JDK 1.8:

  • go for maximal compatibility just use repackaged jsr166y regardless of JDK (simplest)
  • check for jdk version # and use java.util.concurrent instead
  • check for jdk version # and repackage jsr166e and use it instead

Not sure yet which of these is best choice right now.






[CLJ-1218] mapcat is too eager Created: 16/Jun/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Gary Fredericks Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: lazy


 Description   

The following expression prints 1234 and returns 1:

(first (mapcat #(do (print %) [%]) '(1 2 3 4 5 6 7)))

The reason is that (apply concat args) is not maximally lazy in its arguments, and indeed will realize the first four before returning the first item. This in turn is essentially unavoidable for a variadic concat.

This could either be fixed just in mapcat, or by adding a new function (to clojure.core?) that is a non-variadic equivalent to concat, and reimplementing mapcat with it:

(defn join
  "Lazily concatenates a sequence-of-sequences into a flat sequence."
  [s]
  (lazy-seq (when-let [[x & xs] (seq s)] (concat x (join xs)))))


 Comments   
Comment by Gary Fredericks [ 17/Jun/13 7:54 AM ]

I realized that concat could actually be made lazier without changing its semantics, if it had a single [& args] clause that was then implemented similarly to join above.

Comment by John Jacobsen [ 27/Jul/13 8:08 AM ]

I lost several hours understanding this issue last month [1, 2] before seeing this ticket in Jira today... +1.

[1] http://eigenhombre.com/2013/07/13/updating-the-genome-decoder-resulting-consequences/

[2] http://clojurian.blogspot.com/2012/11/beware-of-mapcat.html

Comment by Gary Fredericks [ 05/Feb/14 1:35 PM ]

Updated join code to be actually valid.





[CLJ-1214] Compiler runs out of memory on a small snippet of code Created: 31/May/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Praki Prakash Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: compiler
Environment:

Linux 3.2.0-39-generic


Attachments: File fubar.clj    

 Description   

Clojure compiler runs out of memory when loading the attached file. Transcript below.

$ java -cp ~/.m2/repository/org/clojure/clojure/1.5.1/clojure-1.5.1.jar:. clojure.main
Clojure 1.5.1
user=> (load "fubar")
OutOfMemoryError GC overhead limit exceeded  [trace missing]
user=> 

The file contents are:

  (ns fu.bar)

  (defn foo[l] (concat (drop-last l) (repeat (last l))))

  (def ^:const bar (foo [#(print "") #(println ";")]))

  bar

If I remove the metadata on bar, it works. Removal of namespace seems to fix it as well. Pretty strange.

Although I realize this code is not quite kosher, it would be nice to have the compiler deal with it.



 Comments   
Comment by Andy Fingerhut [ 01/Jun/13 7:40 PM ]

If you really do have 'bar' on a line by itself at the end of file fubar.clj, it seems you are asking it to evaluate the value of bar, which by the definitions is an infinite list, and will of course exhaust all available memory if you attempt to evaluate it.

It seems to me the more odd thing is not that it runs out of memory as shown, but that it does not run out of memory when you remove the metadata on bar.

What is the purpose of having 'bar' on a line by itself at the end of the file?

If I try this but remove 'bar' as the last line of the file, loading the file causes no errors regardless of whether there is metadata on bar's definition or not. It is strange that doing (load "fubar") followed by (first fu.bar/bar) seems to go into an infinite loop if the :const is there on bar, but quickly returns the correct answer if the metadata is removed.

Comment by Praki Prakash [ 01/Jun/13 8:40 PM ]

This code snippet is a minimal test case to show that the compiler runs out of memory. What I meant by "it works" was that the compiler doesn't run out of memory and successfully loads the file (or in my real code base, the namespace is compiled).

In my code, I use bar (or whatever the real thing is) as source of sequence of functions. The sole reference to bar is needed to trigger this issue. I believe that bar is not being fully evaluated here and thus no infinite loop. If I try to print it, yes, it will ultimately fail.

Comment by Praki Prakash [ 01/Jun/13 9:04 PM ]

Having thought about this a bit more, it seems to me that when bar is annotated with const, the compiler is trying to evaluate the associated expression which exhausts the heap? I have never looked at the compiler source and thus not sure if this is what is happening. If it is, then it seems like one should be really careful when adding metadata.

That still leaves the other question about the namespace requirement to cause memory exhaustion. I quite distinctly recall having to add the namespace when trying to come up with minimal test case to reproduce the bug.

If you think this is really user error, I would accept it

Comment by Andy Fingerhut [ 02/Jun/13 4:56 AM ]

It is not just any old metadata that causes this issue, only :const metadata. I tried testing with :const replaced with :dynamic and :private, and there was no problem.

This might shed some light on the issue: https://github.com/clojure/clojure/blob/master/changes.md#215-const-defs

It appears that ^:const is causing the compiler to evaluate the value at compile time. The value in your example is unbounded, so that can never complete.





[CLJ-1212] Silent truncation/downcasting of primitive type on reflection call to overloaded method (Math/abs) Created: 28/May/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Matthew Willson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: primitives, typehints
Environment:

Clojure 1.5.1
OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1)



 Description   

I realise relying on reflection when calling these kinds of methods isn't a great idea performance-wise, but it shouldn't lead to incorrect or dangerous behaviour.

Here it seems to trigger a silent downcast of the input longs, giving a truncated integer output:

user> (defn f [a b] (Math/abs (- a b)))
Reflection warning, NO_SOURCE_PATH:1:15 - call to abs can't be resolved.
#'user/f
user> (f 1000000000000 2000000000000)
727379968
user> (class (f 1000000000000 2000000000000))
java.lang.Integer
user> (defn f [^long a ^long b] (Math/abs (- a b)))
#'user/f
user> (f 1000000000000 2000000000000)
1000000000000
user> (class (f 1000000000000 2000000000000))
java.lang.Long



 Comments   
Comment by Matthew Willson [ 28/May/13 12:50 PM ]

For an even simpler way to replicate the issue:

user> (#(Math/abs %) 1000000000000)
Reflection warning, NO_SOURCE_PATH:1:3 - call to abs can't be resolved.
727379968

Comment by Andy Fingerhut [ 28/May/13 1:36 PM ]

I was able to reproduce the behavior you see with these Java 6 JVMs on Ubuntu 12.04.2:

java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

However, I tried two Java 7 JVMs, and it gave the following behavior which looks closer to what you would hope for. I do not know what is the precise difference between Java 6 and Java 7 that leads to this behavior difference, but this is some evidence that this has something to do with Java 6 vs. Java 7.

user=> (set! warn-on-reflection true)
true
user=> (defn f [a b] (Math/abs (- a b)))
Reflection warning, NO_SOURCE_PATH:1:15 - call to abs can't be resolved.
#'user/f
user=> (f 1000000000000 2000000000000)
1000000000000
user=> (class (f 1000000000000 2000000000000))
java.lang.Long

Above behavior observed with Clojure 1.5.1 on these JVMs:

Ubuntu 12.04.2 plus this JVM:
java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

Mac OS X 10.8.3 plus this JVM:
java version "1.7.0_15"
Java(TM) SE Runtime Environment (build 1.7.0_15-b03)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Comment by Matthew Willson [ 29/May/13 5:17 AM ]

Ah, interesting.
Maybe it's a difference in the way the reflection API works in java 7?

Here's the bytecode generated incase anyone's curious:

public java.lang.Object invoke(java.lang.Object);
Code:
0: ldc #14; //String java.lang.Math
2: invokestatic #20; //Method java/lang/Class.forName:(Ljava/lang/String;)Ljava/lang/Class;
5: ldc #22; //String abs
7: iconst_1
8: anewarray #24; //class java/lang/Object
11: dup
12: iconst_0
13: aload_1
14: aconst_null
15: astore_1
16: aastore
17: invokestatic #30; //Method clojure/lang/Reflector.invokeStaticMethod:(Ljava/lang/Class;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/Object;
20: areturn

Comment by Matthew Willson [ 29/May/13 5:20 AM ]

Just an idea (and maybe this is what's happening under java 7?) but given it's a static method and all available overloaded variants are presumably known at compile time, perhaps it could generate code along the lines of:

(cond
(instance? Long x) (Math/abs (long x))
(instance? Integer x) (Math/abs (int x))
;; ...
)

Comment by Andy Fingerhut [ 29/May/13 3:19 PM ]

In Reflector.java method invokeStaticMethod(Class c, String methodName, Object[] args) there is a call to getMethods() followed by a call to invokeMatchingMethod(). getMethods() returns the 4 java.lang.Math/abs methods in different orders on Java 6 and 7, causing invokeMatchingMethod() to pick a different one on the two JVMs:

java version "1.6.0_39"
Java(TM) SE Runtime Environment (build 1.6.0_39-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode)

user=> (pprint (seq (clojure.lang.Reflector/getMethods java.lang.Math 1 "abs" true)))
(#<Method public static int java.lang.Math.abs(int)>
#<Method public static long java.lang.Math.abs(long)>
#<Method public static float java.lang.Math.abs(float)>
#<Method public static double java.lang.Math.abs(double)>)
nil

java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

user=> (pprint (seq (clojure.lang.Reflector/getMethods java.lang.Math 1 "abs" true)))
(#<Method public static double java.lang.Math.abs(double)>
#<Method public static float java.lang.Math.abs(float)>
#<Method public static long java.lang.Math.abs(long)>
#<Method public static int java.lang.Math.abs(int)>)
nil

That might be a sign of undesirable behavior in invokeMatchingMethod() that is too dependent upon the order of methods given to it.

As you mention, type hinting is good for avoiding the significant performance hit of reflection.





[CLJ-1208] Namespace is not loaded on defrecord class init Created: 03/May/13  Updated: 03/May/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tim McCormack Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

As a user of Clojure interop from Java, I want defrecords (and deftypes?) to load their namespaces upon class initialization so that I can simply construct and use AOT'd record classes without manually requiring their namespaces first.

Calling the defrecord's constructor may or may not result in "Attempting to call unbound fn" exceptions, depending on what code has already been run.

This issue has been raised several times over the years, but I could not find an existing ticket for it:






[CLJ-1207] Importing a class that does not exist fails to report the name of the class that did not exist Created: 29/Apr/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs
Environment:

1.5.1, OS X


Waiting On: Howard Lewis Ship

 Description   

Pop quiz: What Java class is missing from the classpath?

java.lang.NoClassDefFoundError: Could not initialize class com.annadaletech.nexus.util.logging__init
 at java.lang.Class.forName0 (Class.java:-2)
    java.lang.Class.forName (Class.java:264)
    clojure.lang.RT.loadClassForName (RT.java:2098)
    clojure.lang.RT.load (RT.java:430)
    clojure.lang.RT.load (RT.java:411)
    clojure.core$load$fn__5018.invoke (core.clj:5530)
    clojure.core$load.doInvoke (core.clj:5529)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$load_one.invoke (core.clj:5336)
    clojure.core$load_lib$fn__4967.invoke (core.clj:5375)
    clojure.core$load_lib.doInvoke (core.clj:5374)
    clojure.lang.RestFn.applyTo (RestFn.java:142)
    clojure.core$apply.invoke (core.clj:619)
    clojure.core$load_libs.doInvoke (core.clj:5413)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invoke (core.clj:619)
    clojure.core$require.doInvoke (core.clj:5496)
    clojure.lang.RestFn.invoke (RestFn.java:512)
    novate.console.app$eval1736$loading__4910__auto____1737.invoke (app.clj:1)
    novate.console.app$eval1736.invoke (app.clj:1)
    clojure.lang.Compiler.eval (Compiler.java:6619)
    clojure.lang.Compiler.eval (Compiler.java:6608)
    clojure.lang.Compiler.load (Compiler.java:7064)
    user$eval1732.invoke (NO_SOURCE_FILE:1)
    clojure.lang.Compiler.eval (Compiler.java:6619)
    clojure.lang.Compiler.eval (Compiler.java:6582)
    clojure.core$eval.invoke (core.clj:2852)
    clojure.main$repl$read_eval_print__6588$fn__6591.invoke (main.clj:259)
    clojure.main$repl$read_eval_print__6588.invoke (main.clj:259)
    clojure.main$repl$fn__6597.invoke (main.clj:277)
    clojure.main$repl.doInvoke (main.clj:277)
    clojure.lang.RestFn.invoke (RestFn.java:1096)
    clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__584.invoke (interruptible_eval.clj:56)
    clojure.lang.AFn.applyToHelper (AFn.java:159)
    clojure.lang.AFn.applyTo (AFn.java:151)
    clojure.core$apply.invoke (core.clj:617)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1788)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke (interruptible_eval.clj:41)
    clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__625$fn__628.invoke (interruptible_eval.clj:171)
    clojure.core$comp$fn__4154.invoke (core.clj:2330)
    clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__618.invoke (interruptible_eval.clj:138)
    clojure.lang.AFn.run (AFn.java:24)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1110)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:603)
    java.lang.Thread.run (Thread.java:722)

If you guess "com.annadaletech.nexus.util.logging__init" you are wrong!

Wait, I'll give you a hint:

(ns com.annadaletech.nexus.util.logging
  (:use [clojure.string :only [trim-newline]]
        [clojure.pprint :only [code-dispatch pprint with-pprint-dispatch *print-right-margin*]])
  (:import [java.io StringWriter]
           [org.slf4j MDC MarkerFactory Marker LoggerFactory]
           [java.util.concurrent.locks ReentrantLock]))

Oh, sorry, did that not help?

The correct answer is "org.slf4j.MDC".

Having that information in the stack trace would have saved me nearly an hour. I think it is worth the effort to get that reported correctly.



 Comments   
Comment by Gabriel Horner [ 10/May/13 1:56 PM ]

When I try this on a fresh project, I get this error:
"ClassNotFoundException org.slf4j.MDC
java.net.URLClassLoader$1.run (URLClassLoader.java:202)
java.security.AccessController.doPrivileged (AccessController.java:-2)"

Howard, could you give us a project.clj or better yet a github repository that recreates this issue?

Comment by Howard Lewis Ship [ 10/May/13 4:51 PM ]

I'll see what I can do. Probably be next week. Thanks for looking at this.

Comment by Gary Fredericks [ 26/May/13 8:20 AM ]

This reminds me of an issue with `lein run` that resulted from it trying to figure out whether you wanted to run a namespace or a java class:

https://github.com/technomancy/leiningen/issues/1182





[CLJ-1206] 'eval' of closures or fns with runtime metadata within a call expr yields "No matching ctor found" exceptions Created: 28/Apr/13  Updated: 28/Apr/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jason Wolfe Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I ran into some issues with 'eval' when writing compilation strategies for Graph. It seems these may have been known for some time [1], but I couldn't find a ticket for them, so here we are.

Clojure docs [2] say "If the operator is not a special form or macro, the call is considered a function call. Both the operator and the operands (if any) are evaluated, from left to right," and "Any object other than those discussed above will evaluate to itself." While bare fns do seem to evaluate to themselves in all cases, when in a call expression, the evaluation of the operator fails on fn objects that are closures or have run-time metadata applied:

;; raw non-closures are fine
user> (eval (fn [x] (inc x)))
#<user$eval30559$fn_30560 user$eval30559$fn_30560@354ee11c>

;; raw closures are fine
user> (eval (let [y 1] (fn [x] (+ x y))))
#<user$eval30511$fn_30512 user$eval30511$fn_30512@3bac3a34>

;; non-closures in exprs are fine
user> (eval `(~(fn [x] (inc x)) 1))
2

;; but closures in exprs cause an error
user> (eval `(~(let [y 1] (fn [x] (+ x y))) 1))
IllegalArgumentException No matching ctor found for class user$eval30535$fn__30536 clojure.lang.Reflector.invokeConstructor (Reflector.java:163)

;; as do fns with metadata in exprs
user> (eval `(~(with-meta (fn [x] (inc x)) {:x 1}) 1))
IllegalArgumentException No matching ctor found for class clojure.lang.AFunction$1 clojure.lang.Reflector.invokeConstructor (Reflector.java:163)

[1] http://stackoverflow.com/a/11287181
[2] http://clojure.org/evaluation






[CLJ-1201] There should also be writing in clojure.edn Created: 15/Apr/13  Updated: 23/May/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Vitaly Shukela Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: edn


 Description   

In clojure.edn I see only "read" and "read-string".

For symmetry I expect "write" and "write-string" to be nearby. At first it could be just alias for "pr" and "pr-str", but in furure they may limited version of "pr" which only produces valid input for clojure.edn/read.



 Comments   
Comment by Andy Fingerhut [ 23/May/13 5:56 PM ]

Related clojure-dev message: https://groups.google.com/forum/?fromgroups#!topic/clojure-dev/fLJWh9A3OuA

and enhancement proposal wiki page: http://dev.clojure.org/display/design/Representing+EDN





[CLJ-1199] Record values are not 'eval'uated, unlike values of PersistentMap: Created: 13/Apr/13  Updated: 13/Apr/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jason Wolfe Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I'm not sure if this is by design, but it caught me off guard.

user> (defrecord A [x])
user.A

user> (eval (hash-map :x `long))
{:x #<core$long clojure.core$long@5de54eb7>}
user> (eval (->A `long))
#user.A{:x clojure.core/long}
user> (eval (map->A (hash-map :x `long)))
#user.A{:x clojure.core/long}

and in case it matters, here's a simplified version of the real use case where this came up, with no eval – just a macro:

user> (defmacro munge-meta1 [x] (assoc x :schema (->A (:schema (meta x)))))
#'user/munge-meta1
user> (munge-meta1 ^{:schema long} {})
{:schema #user.A{:x long}}

user> (defmacro munge-meta2 [x] (assoc x :schema (hash-map :x (:schema (meta x)))))
#'user/munge-meta2
user> (munge-meta2 ^{:schema long} {})
{:schema {:x #<core$long clojure.core$long@5de54eb7>}}

This seems to be fixed by moving the record creation post-evaluation, so it's not a big deal, just surprising (plus I haven't yet convinced myself that this will always work if the user's schema itself contains record-creating forms, although it seems to work OK):

user> (defmacro munge-meta1 [x] (assoc x :schema `(->A ~(:schema (meta x)))))
#'user/munge-meta1
user> (munge-meta1 ^{:schema long} {})
{:schema #user.A{:x #<core$long clojure.core$long@5de54eb7>}}

I brought this up on the mailing list here:

https://groups.google.com/forum/?fromgroups=#!topic/clojure-dev/UgD35E1RQTo






[CLJ-1198] Apply metadata to primitive fns causes them to lose their primitive-ness Created: 13/Apr/13  Updated: 13/Apr/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jason Wolfe Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

user> (def f (fn [^long x] x))
#'user/f
user> (.invokePrim (with-meta f {}) 1)
IllegalArgumentException No matching method found: invokePrim for class clojure.lang.AFunction$1 clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:53)
user> (contains? (ancestors (class f)) clojure.lang.IFn$LO)
true
user> (contains? (ancestors (class (with-meta f {}))) clojure.lang.IFn$LO)
false

We're working on libraries that use metadata on functions to track information about their arguments (schemata, etc), and this currently blocks us from fully supporting primitive fns.






[CLJ-1195] emit-hinted-impl expands to non-ns-qualified invocation of 'fn' Created: 09/Apr/13  Updated: 09/Apr/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jason Wolfe Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mac os X Clojure 1.5.1



 Description   

(ns plumbing.tmp
(:refer-clojure :exclude [fn]))

(defprotocol Foo
(foo [this]))

(extend-protocol Foo
Object
(foo [this]))

yields

CompilerException java.lang.RuntimeException: Unable to resolve symbol: fn in this context, compiling/Users/w01fe/prismatic/prismatic/plumbing/src/plumbing/tmp.clj:7:1)

This makes it difficult to construct a namespace that provides a replacement def for fn.






[CLJ-1192] vec function is substantially slower than into function Created: 06/Apr/13  Updated: 11/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Luke VanderHart Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: performance

Approval: Triaged

 Description   

(vec coll) and (into [] coll) do exactly the same thing. However, due to into using transients, it is substantially faster. On my machine:

(time (dotimes [_ 100] (vec (range 100000))))
"Elapsed time: 732.56 msecs"

(time (dotimes [_ 100] (into [] (range 100000))))
"Elapsed time: 491.411 msecs"

This is consistently repeatable.

Since vec's sole purpose is to transform collections into vectors, it should do so at the maximum speed available.



 Comments   
Comment by Andy Fingerhut [ 07/Apr/13 5:50 PM ]

I am pretty sure that Clojure 1.5.1 also uses transient vectors for (vec (range n)) (probably also some earlier versions of Clojure, too).

Look at vec in core.clj. It checks whether its arg is a java.util.Collection, which lazy seqs are, so calls (clojure.lang.LazilyPersistentVector/create coll).

LazilyPersistentVector's create method checks whether its argument is an ISeq, which lazy seqs are, so it calls PersistentVector.create(RT.seq(coll)).

All 3 of PersistentVector's create() methods use transient vectors to build up the result.

I suspect the difference in run times are not because of transients or not, but because of the way into uses reduce, and perhaps may also have something to do with the perhaps-unnecessary call to RT.seq in LazilyPersistentVector's create method (in this case, at least – it is likely needed for other types of arguments).

Comment by Alan Malloy [ 14/Jun/13 2:17 PM ]

I'm pretty sure the difference is that into uses reduce: since reducers were added in 1.5, chunked sequences know how to reduce themselves without creating unnecessary cons cells. PersistentVector/create doesn't use reduce, so it has to allocate a cons cell for each item in the sequence.

Comment by Gary Fredericks [ 08/Sep/13 1:55 PM ]

Is there any downside to (defn vec [coll] (into [] coll)) (or the inlined equivalent)?

Comment by Ghadi Shayban [ 11/Apr/14 5:13 PM ]

While I agree that there are improvements and possibly low-hanging fruit, FWIW https://github.com/clojure/tools.analyzer/commit/cf7dda81a22f4c9c1fe64c699ca17e7deed61db4#commitcomment-5989545

showed a 5% slowdown from a few callsites in tools.analyzer.

This ticket's benchmark is incomplete in that it covers a single type of argument (chunked range), and flawed as it timing the expense of realizing the range. (That could be a legit benchmark case, but it shouldn't be the only one).

Sorry to rain on a parade. I promise like speed too!





[CLJ-1181] clojure.pprint/code-dispatch breaks on certain types of anonymous functions Created: 10/Mar/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Devin Walters Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: print


 Description   
(with-out-str 
  (with-pprint-dispatch code-dispatch 
                        (pp/pprint (read-string "(fn* [x] x)"))))

breaks because the format string here: https://github.com/clojure/clojure/blob/master/src/clj/clojure/pprint/dispatch.clj#L378 expects a sequence. In the case of (fn* [x] x) it is passed a symbol.



 Comments   
Comment by Jean Niklas L'orange [ 18/Mar/13 5:40 PM ]

I think the main "issue" here resides within the undocumented functionality of fn*. (fn* [x] x) is a semantically working function, but (fn [x] x) expands into (fn* ([x] x)). Anonymous function literals expand into (fn* [gensyms] (...)), and as such, it also accepts expressions like (fn* [x] x). Should pprint pretty print expressions which has used fn* directly, or should it "just" ignore it?





[CLJ-1172] Cross-linking between clojure.lang.Compiler and clojure.lang.RT Created: 28/Feb/13  Updated: 21/Jun/13

Status: Reopened
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Yegor Bugayenko Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

version 1.5.0-RC17



 Description   

This is my code (an example):

import clojure.lang.Compiler;
import clojure.lang.RT;
import clojure.lang.Var;

Compiler.load("(+ 5 %)");
Var foo = RT.var("bar", "foo");
Object result = foo.invoke(10);
assert result.toString().equals("15");

This is what I'm getting:

ava.lang.ExceptionInInitializerError
	at clojure.lang.Compiler.<clinit>(Compiler.java:47)
	at foo.main(Main.java:75)
Caused by: java.lang.NullPointerException
	at clojure.lang.RT.baseLoader(RT.java:2043)
	at clojure.lang.RT.load(RT.java:417)
	at clojure.lang.RT.load(RT.java:411)
	at clojure.lang.RT.doInit(RT.java:447)
	at clojure.lang.RT.<clinit>(RT.java:329)
	... 36 more

The same code worked just fine with version 1.4. Looks like Compiler is using RT and RT is using Compiler, both statically.



 Comments   
Comment by Yegor Bugayenko [ 04/Mar/13 11:40 AM ]

I cross-posted this question to SO: http://stackoverflow.com/questions/15207596

Comment by Yegor Bugayenko [ 05/Mar/13 12:04 AM ]

calling RT.init() before Compiler.load() solves the problem

Comment by Andy Fingerhut [ 05/Mar/13 4:17 AM ]

Yegor, do you consider it OK to close this ticket as not being a problem, or at least one with a reasonable workaround?

Comment by Yegor Bugayenko [ 05/Mar/13 1:11 PM ]

Yes, of course. Let's close it.

Comment by Andy Fingerhut [ 05/Mar/13 6:14 PM ]

Ticket submitter agrees that this is not an issue, or that there is a reasonable workaround.

Comment by Andy Fingerhut [ 13/Mar/13 12:58 AM ]

This issue came up again on the Clojure group. https://groups.google.com/forum/?hl=en_US&fromgroups=#!topic/clojure/2xdLNMb9yyQ

I did some testing, and the issue did not exist in Clojure 1.5.0-RC3 and before, and it has existed since 1.5.0-RC4. There was only one commit between those two points:

https://github.com/clojure/clojure/commit/9b80a552fdabeabdd93951a625b55ae49c2f8d83

Maybe this new behavior is an intended consequence of that change. I don't know. In any case, it seems like perhaps the "No need to call RT.init() anymore" message might be outdated?

Comment by Andy Fingerhut [ 13/Mar/13 12:59 AM ]

Reopening since it came up again, and there is some more info known about the issue. I'll let someone who knows more about the issue decide whether to close it.

Comment by Edward [ 23/Mar/13 10:31 AM ]

Doing this RT.load("clojure/core"); at the top works avoids the message from RT.init()

Comment by Jean Niklas L'orange [ 21/Jun/13 10:36 AM ]

It seems like RT.load("clojure/core") does not hide the message anymore - at least not from 1.5.1.





[CLJ-1167] repl value of *file* is "NO_SOURCE_PATH", of *source-path* is "NO_SOURCE_FILE" Created: 19/Feb/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Brian Marick Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: repl


 Comments   
Comment by Brian Marick [ 19/Feb/13 4:22 PM ]

Forgot to mention: I think it's intended to be the other way around, given the names.





[CLJ-1159] clojure.java.io/delete-file doesn't return the status of the deletion(true/false) Created: 10/Feb/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: AtKaaZ Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs, io
Environment:

any



 Description   

initially reported it here(somehow):
https://groups.google.com/d/msg/clojure/T9Kvr0IL0kg/wcKBfR9w_1sJ

Basically clojure.java.io/delete-file doesn't ever return false (even when silently is true, it returns the value of silently), it's due to how it's implemented - but it's obvious from the code, so I'll stop here.

Thanks.

PS: this is what I'm using as my current workaround:
(defn delete-file
"
an implementation that returns the true/false status
which clojure.java.io/delete-file doesn't do(tested in 1.5.0-RC14)
"
[f & [silently]]
(let [ret (.delete (clojure.java.io/file f))]
(cond (or ret silently)
ret
:else
(throw (java.io.IOException. (str "Couldn't delete " f)))
)
)
)

I'm sure you guys can find a better way, but as a clojure newbie(really!) that's what I have.



 Comments   
Comment by AtKaaZ [ 10/Feb/13 2:01 PM ]

I kinda just realized it affects all versions since and including 1.2, because it appears that its implementation was the same since then.

If it's not meant to return the result of the delete, maybe it should specifically return nil and/or the doc say something?

Comment by Sean Corfield [ 10/Feb/13 2:21 PM ]

As noted in a thread on the Clojure ML, you can pass a known value in the second argument position to detect a delete that failed:

(clojure.java.io/delete-file some-file :not-deleted)

This returns true on success and :not-deleted on failure.

However the docstring could be better worded to make that intention clear. Perhaps:

Delete file f. Return true if it succeeds. If silently is nil or false, raise an exception if it fails, else return the value of silently.
This allows you to detect whether the delete succeeded without catching an exception by passing a non-true truthy value as the second argument.





[CLJ-1152] PermGen leak in multimethods and protocol fns when evaled Created: 30/Jan/13  Updated: 10/Dec/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Chouser Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: memory, protocols

Approval: Triaged

 Description   

There is a PermGen memory leak that we have tracked down to protocol methods and multimethods called inside an eval, because of the caches these methods use. The problem only arises when the value being cached is an instance of a class (such as a function or reify) that was defined inside the eval. Thus extending IFn or dispatching a multimethod on an IFn are likely triggers.

My fellow LonoClouder, Jeff Dik describes how to reproduce and work around the problem:

The easiest way that I have found to test this is to set "-XX:MaxPermSize" to a reasonable value so you don't have to wait too long for the PermGen space to fill up, and to use "-XX:+TraceClassLoading" and "-XX:+TraceClassUnloading" to see the classes being loaded and unloaded.

leiningen project.clj
(defproject permgen-scratch "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.5.0-RC1"]]
  :jvm-opts ["-XX:MaxPermSize=32M"
             "-XX:+TraceClassLoading"
             "-XX:+TraceClassUnloading"])

You can use lein swank 45678 and connect with slime in emacs via M-x slime-connect.

To monitor the PermGen usage, you can find the Java process to watch with "jps -lmvV" and then run "jstat -gcold <PROCESS_ID> 1s". According to the jstat docs, the first column (PC) is the "Current permanent space capacity (KB)" and the second column (PU) is the "Permanent space utilization (KB)". VisualVM is also a nice tool for monitoring this.

Multimethod leak

Evaluating the following code will run a loop that eval's (take* (fn foo [])).

multimethod leak
(defmulti take* (fn [a] (type a)))

(defmethod take* clojure.lang.Fn
  [a]
  '())

(def stop (atom false))
(def sleep-duration (atom 1000))

(defn run-loop []
  (when-not @stop
    (eval '(take* (fn foo [])))
    (Thread/sleep @sleep-duration)
    (recur)))

(future (run-loop))

(reset! sleep-duration 0)

In the lein swank session, you will see many lines like below listing the classes being created and loaded.

[Loaded user$eval15802$foo__15803 from __JVM_DefineClass__]
[Loaded user$eval15802 from __JVM_DefineClass__]

These lines will stop once the PermGen space fills up.

In the jstat monitoring, you'll see the amount of used PermGen space (PU) increase to the max and stay there.

-    PC       PU        OC          OU       YGC    FGC    FGCT     GCT
 31616.0  31552.7    365952.0         0.0      4     0    0.000    0.129
 32000.0  31914.0    365952.0         0.0      4     0    0.000    0.129
 32768.0  32635.5    365952.0         0.0      4     0    0.000    0.129
 32768.0  32767.6    365952.0      1872.0      5     1    0.000    0.177
 32768.0  32108.2    291008.0     23681.8      6     2    0.827    1.006
 32768.0  32470.4    291008.0     23681.8      6     2    0.827    1.006
 32768.0  32767.2    698880.0     24013.8      8     4    1.073    1.258
 32768.0  32767.2    698880.0     24013.8      8     4    1.073    1.258
 32768.0  32767.2    698880.0     24013.8      8     4    1.073    1.258

A workaround is to run prefer-method before the PermGen space is all used up, e.g.

(prefer-method take* clojure.lang.Fn java.lang.Object)

Then, when the used PermGen space is close to the max, in the lein swank session, you will see the classes created by the eval'ing being unloaded.

[Unloading class user$eval5950$foo__5951]
[Unloading class user$eval3814]
[Unloading class user$eval2902$foo__2903]
[Unloading class user$eval13414]

In the jstat monitoring, there will be a long pause when used PermGen space stays close to the max, and then it will drop down, and start increasing again when more eval'ing occurs.

-    PC       PU        OC          OU       YGC    FGC    FGCT     GCT
 32768.0  32767.9    159680.0     24573.4      6     2    0.167    0.391
 32768.0  32767.9    159680.0     24573.4      6     2    0.167    0.391
 32768.0  17891.3    283776.0     17243.9      6     2   50.589   50.813
 32768.0  18254.2    283776.0     17243.9      6     2   50.589   50.813

The defmulti defines a cache that uses the dispatch values as keys. Each eval call in the loop defines a new foo class which is then added to the cache when take* is called, preventing the class from ever being GCed.

The prefer-method workaround works because it calls clojure.lang.MultiFn.preferMethod, which calls the private MultiFn.resetCache method, which completely empties the cache.

Protocol leak

The leak with protocol methods similarly involves a cache. You see essentially the same behavior as the multimethod leak if you run the following code using protocols.

protocol leak
(defprotocol ITake (take* [a]))

(extend-type clojure.lang.Fn
  ITake
  (take* [this] '()))

(def stop (atom false))
(def sleep-duration (atom 1000))

(defn run-loop []
  (when-not @stop
    (eval '(take* (fn foo [])))
    (Thread/sleep @sleep-duration)
    (recur)))

(future (run-loop))

(reset! sleep-duration 0)

Again, the cache is in the take* method itself, using each new foo class as a key.

A workaround is to run -reset-methods on the protocol before the PermGen space is all used up, e.g.

(-reset-methods ITake)

This works because -reset-methods replaces the cache with an empty MethodImplCache.



 Comments   
Comment by Chouser [ 30/Jan/13 9:10 AM ]

I think the most obvious solution would be to constrain the size of the cache. Adding an item to the cache is already not the fastest path, so a bit more work could be done to prevent the cache from growing indefinitely large.

That does raise the question of what criteria to use. Keep the first n entries? Keep the n most recently used (which would require bookkeeping in the fast cache-hit path)? Keep the n most recently added?

Comment by Jamie Stephens [ 18/Oct/13 9:35 AM ]

At a minimum, perhaps a switch to disable the caches – with obvious performance impact caveats.

Seems like expensive LRU logic is probably the way to go, but maybe don't have it kick in fully until some threshold is crossed.

Comment by Alex Miller [ 18/Oct/13 4:28 PM ]

A report seeing this in production from mailing list:
https://groups.google.com/forum/#!topic/clojure/_n3HipchjCc

Comment by Adrian Medina [ 10/Dec/13 11:43 AM ]

So this is why we've been running into PermGen space exceptions! This is a fairly critical bug for us - I'm making extensive use of multimethods in our codebase and this exception will creep in at runtime randomly.





[CLJ-1149] Unhelpful error message from :use (and use function) when arguments are malformed Created: 17/Jan/13  Updated: 28/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4, Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Sean Corfield Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: errormsgs

Approval: Triaged

 Description   

the following exception happens when you have something like this(bad):

(ns runtime.util-test
(:use [midje.sweet :reload-all]))

as opposed to any of these(correct):

(ns runtime.util-test
(:use midje.sweet :reload-all))

(ns runtime.util-test
(:use [midje.sweet] :reload-all))

and the exception is:
Exception in thread "main" java.lang.IllegalArgumentException: No value supplied for key: true
at clojure.lang.PersistentHashMap.create(PersistentHashMap.java:77)
at clojure.core$hash_map.doInvoke(core.clj:365)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:617)
at clojure.core$load_lib.doInvoke(core.clj:5352)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$load_libs.doInvoke(core.clj:5403)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:621)
at clojure.core$use.doInvoke(core.clj:5497)

Note that this is similar to the equally unhelpful message shown in http://dev.clojure.org/jira/browse/CLJ-1140 although that is a different root cause.

Probably best to enhance the `use` function to validate its arguments before trying to apply hash-map?



 Comments   
Comment by Gary Fredericks [ 26/May/13 3:17 PM ]

I believe this applies to require as well.





[CLJ-1147] Threading macro (->) does not permit inline function declarations Created: 14/Jan/13  Updated: 26/May/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Stephen Nelson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

(-> [1 2 3] (fn [args] apply + args))

CompilerException java.lang.Exception: Unsupported binding form: 1, compiling:(NO_SOURCE_PATH:1:13)

The expression is expanded to:

(fn [1 2 3] [args] apply + args)

If this is intended behaviour then at the least the compiler error message is confusing. It would be preferable if the -> macro checked for (fn..) before treating a form as a sequence and injecting the argument.



 Comments   
Comment by Andy Fingerhut [ 15/Jan/13 12:56 AM ]

Note that this works as you might have hoped:

(-> [1 2 3] ((fn [args] (apply + args))))

because it expands into:

((fn [args] (apply + args)) [1 2 3])

Your suggestion that -> check for (fn ...) before treating it as a sequence and injecting the argument leaves open the question: Why only (fn ...) should be treated specially? Why not (let ...), (for ...), (doseq ...), etc? And if you go that far, how do you decide what should be allowed and what not?

Comment by Gabriel Horner [ 17/May/13 2:56 PM ]

I agree with Andy, that it's not realistic suggestion to check for fn,let,etc. Perhaps a doc fix would help here but I'm not sure if we just want to call out (fn ...). I'd recommend closing this unless Stephen speaks up.

Comment by Stephen Nelson [ 19/May/13 10:29 PM ]

I'm happy with Andy's synopsis of the problem, and it's reasonable not to change the behaviour of the threading macro specifically for (fn..).

However, this is a mistake that I'm sure many others make/have made and it's hard to diagnose what is going wrong without dumping interpreted form – hardly a reasonable expectation for a novice user.

Before closing this issue, I'd like to see improved failure reporting, such as causing the threading macro to throw a compile error or warning if passed a raw (unwrapped) function declaration (are there legitimate use cases this would affect?).

Comment by Gary Fredericks [ 26/May/13 2:21 PM ]

Throwing an error on (-> [1 2 3] (fn ...)) would certainly affect any perverse individual using a local redefinition of 'fn.

I think the best that can be done here is a mention in the docstring.





[CLJ-1146] Symbol name starting with digits to defn throws "Unmatched delimiter )" Created: 13/Jan/13  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Linus Ericsson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs, reader
Environment:

$java -jar clojure-1.5.0-RC2.jar

$java -version
java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-10M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)
Mac OS X:
System Version: Mac OS X 10.6.8 (10K549)
Kernel Version: Darwin 10.8.0



 Description   

When trying to use an invalid symbol name when defining a function, the error message thrown is a confusing and wrong one. The error message is "RuntimeException Unmatched delimiter: ) clojure.lang.Util.runtimeException (Util.java:219)", which unfortunately is the only message seen in nrepled emacs.

$ java -jar clojure-1.5.0-RC2.jar
Clojure 1.5.0-RC2
user=> (defn 45fn [] nil)
NumberFormatException Invalid number: 45fn clojure.lang.LispReader.readNumber (LispReader.java:255)
[]
nil
RuntimeException Unmatched delimiter: ) clojure.lang.Util.runtimeException (Util.java:219)

Expected:
When trying to (defn or (def a thing with a non valid symbol name, the last thrown error message should be one stating that the given symbol name is not a valid one.






[CLJ-1142] Incorrect divide-by-zero error with floating point numbers Created: 08/Jan/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Tim McCormack Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: math


 Description   

The unary call for clojure.core// treats a dividend of 0.0 differently than the binary call, likely due to inlining.

(/ 0.0) ;; java.lang.ArithmeticException: Divide by zero
(/ 1 0.0) ;;= Infinity
(/ 1 (identity 0.0)) ;; java.lang.ArithmeticException: Divide by zero


 Comments   
Comment by Tim McCormack [ 08/Jan/13 11:22 PM ]

The relevant code seems to be this in clojure.lang.Numbers/divide:

if(yops.isZero((Number)y))
  throw new ArithmeticException("Divide by zero");

Making Numbers/divide be more restrictive than double arithmetic seems like a bug; explicitly throwing an ArithmeticException instead of letting the JVM figure it just seems like more work than necessary.





[CLJ-1141] Allow pre and post-conditions in defprotocol and deftype macros Created: 02/Jan/13  Updated: 04/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alexander Kiel Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: deftype, protocols
Environment:

Dos not matter.



 Description   

The fn special form and the defn macro allow pre- and post-conditions. It would be nice if one could use that conditions also in method declarations of the defprotocol and deftype macro.

Currently I use the extend function as workaround where one can specify the methods using a map of keyword-name and fn special form.



 Comments   
Comment by Michael Drogalis [ 06/Jan/13 6:22 PM ]

Using :pre and :post, IMO, isn't a good idea. Handling assertions is a two part game. The mechanism needs to account for both detection and reaction, and the latter is missing.

This isn't a perfect work-around, as it's a little verbose, but using Dire might work better than using extend. In addition, you get the "reaction" functionality that's missing from :pre and :post

Example for protocol preconditions: https://gist.github.com/4471276

Comment by Alexander Kiel [ 07/Jan/13 11:52 AM ]

@Michael I read your gist and the README of Dire. I think the supervision concept of Erlang has it's places but I don't like it for pre- and post-conditions. For me, such conditions have two proposes:

  1. they should document the code and
  2. they should fail fast to detect failures early.

To support my first point, your pre- and post-conditions are just lexical too far away from the actual function definition. For the second point: I think in the case of violations the program should just crash. One could maybe wrap some part of the program with one of your exception supervisors handling an AssertionError. But I don't think that handling pre- and post-condition violations for individual functions is a good thing.

Comment by Michael Drogalis [ 07/Jan/13 5:28 PM ]

@Alexander Indeed, your points are correct. Dire is meant to be exactly what you described. Lexically removed from application logic, and opportunity to recover from crashing. That was my best shot at aiding your needs quickly, anyway.





[CLJ-1138] data-reader returning nil causes exception Created: 22/Dec/12  Updated: 14/Feb/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4, Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Steve Miner Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reader
Environment:

clojure 1.5 beta2, Mac OS X 10.8.2, java version "1.6.0_37"



 Description   

If a data-reader returns nil, the reader throws java.lang.RuntimeException: No dispatch macro... The error message implies that there is no dispatch macro for whatever the first character of the tag happens to be.

Here's a simple example:

(binding [*data-readers* {'f/ignore (constantly nil)}] (read-string "#f/ignore 42 10"))

RuntimeException No dispatch macro for: f clojure.lang.Util.runtimeException (Util.java:219)



 Comments   
Comment by Steve Miner [ 22/Dec/12 9:43 AM ]

clj-1138-allow-data-reader-to-return-nil-instead-of-throwing.patch allows a data-reader to return nil instead of throwing. Does sanity check that possible tag or record isJavaIdentifierStart(). Gives better error message for special characters that might actually be dispatch macros (rather than assuming it's a tagged literal).

Comment by Steve Miner [ 22/Dec/12 10:06 AM ]

clj-1138-data-reader-return-nil-for-no-op.patch allows a data-reader returning nil to be treated as a no-op by the reader (like #_). nil is not normally a useful value (actually it causes an exception in Clojure 1.4 through 1.5 beta2) for a data-reader to return. With this patch, one could get something like a conditional feature reader using data-readers.

Comment by Steve Miner [ 22/Dec/12 10:26 AM ]

clj-1138-allow-data-reader-to-return-nil-instead-of-throwing.patch is the first patch to consider. It merely allows nil as a value from a data-reader and returns nil as the final value. I think it does what was originally intended for dispatch macros, and gives a better error message in many cases (mostly typos).

The second patch, clj-1138-data-reader-return-nil-for-no-op.patch, depends on the other being applied first. It takes an extra step to treat a nil value returned from a data-reader as a no-op for the reader (like #_).

Comment by Steve Miner [ 23/Dec/12 11:52 AM ]

It turns out that you can work around the original problem by having your data-reader return '(quote nil) instead of plain nil. That expression conveniently evaluates to nil so you can get a nil if necessary. This also works after applying the patches so there's still a way to return nil if you really want it.

(binding [*data-readers* {'x/nil (constantly '(quote nil))}] (read-string "#x/nil 42"))
;=> (quote nil)

Comment by Andy Fingerhut [ 07/Feb/13 9:20 AM ]

Patch clj-1138-allow-data-reader-to-return-nil-instead-of-throwing.patch dated Dec 22 2012 still applies cleanly to latest master if you use the following command:

% git am --keep-cr -s --ignore-whitespace < clj-1138-allow-data-reader-to-return-nil-instead-of-throwing.patch

Without the --ignore-whitespace option, the patch fails only because some whitespace was changed in Clojure master recently.

Comment by Andy Fingerhut [ 13/Feb/13 11:24 AM ]

OK, now with latest master (1.5.0-RC15 at this time), patch clj-1138-allow-data-reader-to-return-nil-instead-of-throwing.patch no longer applies cleanly, not even using --ignore-whitespace in the 'git am' command given above. Steve, if you could see what needs to be updated, that would be great. Using the patch command as suggested in the "Updating stale patches" section of http://dev.clojure.org/display/design/JIRA+workflow wasn't enough, so it should probably be carefully examined by hand to see what needs updating.

Comment by Steve Miner [ 14/Feb/13 12:21 PM ]

I removed my patches. Things have changes recently with the LispReader and new EdnReader.





[CLJ-1136] Type hinting for array classes does not work in binding forms Created: 20/Dec/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4, Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Luke VanderHart Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop, typehints
Environment:

replicated on OpenJDK 7u9 on Ubuntu 12.04, and Hotspot 1.6.0_37 on OSX Lion



 Description   

Type hints don't work as expected in binding forms.

The following form results in a reflection warning:

(let [^{:tag (Class/forName "[Ljava.lang.Object;")} a (make-array Object 2)]
(aget a 0))

However, hinting does appear to work correctly on vars:

(def ^{:tag (Class/forName "[Ljava.lang.Object;")} a (make-array Object 2))
(aget a 0) ;; no reflection warning



 Comments   
Comment by Ghadi Shayban [ 20/Dec/12 10:51 PM ]

It's a little more insidious than type hinting: the compiler doesn't evaluate metadata in the binding vec.

This doesn't throw the necessary exception...

(let [^{:foo (Class/forName "not real")} bar 42]
bar)

neither this...

(let [^{gyorgy ligeti} a 42]
a)

Gyorgy Ligeti never resolves.

These two equivalent examples don't reflect:
(let [^objects a (make-array Object 2)]
(aget a 0))

(let [a ^objects (make-array Object 2)]
(aget a 0))

Comment by Ghadi Shayban [ 21/Dec/12 11:09 AM ]

On only the left-hand side of a local binding, metadata on a symbol is not analyzed or evaluated.





[CLJ-1133] Certain actions on mutable fields in deftype lead to very strange error messages Created: 18/Dec/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Vladimir Matveev Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: deftype
Environment:

Archlinux x86_64, Windows 7 x86_64



 Description   

Consider the following code:

(definterface Test
(^void fail []))

(deftype TestImpl
[^{:unsynchronized-mutable true :tag int} x]
Test
(fail [this]
(set! x (dec x))))

Its compilation fails with the following message:
CompilerException java.lang.VerifyError: (class: test/TestImpl, method: fail signature: ()V) Expecting to find integer on stack, compiling.../test.clj:27)

The following code works:

(definterface Test
(^void fail []))

(deftype TestImpl
[^{:unsynchronized-mutable true :tag int} x]
Test
(fail [this]
(set! x (int (dec x)))))

The only change here is that I have wrapped (dec x) form into (int) call.

I understand that in fact the former code should not work anyway (or at least should not work as I have expected) because (dec) is defined as a call to clojure.lang.Numbers.dec(), which is overloaded for double, long and Object only (in fact, changing :tag int to :tag long in the first example allows the program to compile). However, the error message is completely uninformative and misleading; it also looks like that it is a consequence of compiler error. It is also not a problem of this concrete example; I found this error in more complex interface method implementation where (set!) call was right in the middle of its body.

I'm using Clojure 1.4.0 and have experienced this problem on Archlinux x86_64 and Windows 7 x86_64.

Full stack trace of the error, in case it would be helpful:

java.lang.VerifyError: (class: test/TestImpl, method: fail signature: ()V) Expecting to find integer on stack, compiling.../test.clj:27)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6462)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5054)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3674)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6453)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.access$100(Compiler.java:37)
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:518)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5919)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5054)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3674)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6453)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.eval(Compiler.java:6508)
at clojure.lang.Compiler.load(Compiler.java:6952)
at clojure.lang.Compiler.loadFile(Compiler.java:6912)
at clojure.lang.RT$3.invoke(RT.java:307)
at test$eval3224.invoke(NO_SOURCE_FILE:43)
at clojure.lang.Compiler.eval(Compiler.java:6511)
at clojure.lang.Compiler.eval(Compiler.java:6477)
at clojure.core$eval.invoke(core.clj:2797)
at clojure.main$repl$read_eval_print__6405.invoke(main.clj:245)
at clojure.main$repl$fn__6410.invoke(main.clj:266)
at clojure.main$repl.doInvoke(main.clj:266)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.main$repl_opt.invoke(main.clj:332)
at clojure.main$main.doInvoke(main.clj:428)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.Var.invoke(Var.java:411)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
Caused by: java.lang.VerifyError: (class: test/TestImpl, method: fail signature: ()V) Expecting to find integer on stack
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at clojure.lang.RT.classForName(RT.java:2039)
at clojure.lang.Compiler$HostExpr.maybeClass(Compiler.java:957)
at clojure.lang.Compiler$HostExpr.access$400(Compiler.java:736)
at clojure.lang.Compiler$NewExpr$Parser.parse(Compiler.java:2473)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
... 45 more



 Comments   
Comment by Vladimir Matveev [ 18/Dec/12 1:51 PM ]

Shouldn't have set major priority; but I cannot edit issue again

Comment by Andy Fingerhut [ 19/Dec/12 1:20 AM ]

Reduced priority to minor, since ticket creator could not do so themselves.





[CLJ-1132] For record type Rec, (instance? Rec (map->Rec {...})) need not return true, though (instance? Rec (Rec. ...)) does. Created: 18/Dec/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Christopher Genovese Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: defrecord
Environment:

Apache Tomcat/6.0.24 JVM/1.6.0_26-b03 Linux 2.6.32-279.el6.x86_64

Clojure 1.4.0, Ring 1.1.6, Compojure 1.1.3, Lein-Ring Plugin 0.7.5 (for lein ring uberwar)


Attachments: File recordbug.tgz    

 Description   

(defrecord Rec ...)

(instance? Rec (Rec. ...)) ;=> true
(instance? Rec (map->Rec {...})) ;=> can be false

This occurs when the code is wrapped in a tomcat servlet by `lein ring
uberwar`, but not when run at the REPL or under Jetty, say.

Although produced under ring, this seems to me to be a general problem
with the map-> constructor, as (true? (instance? Rec (map->Rec {...})))
should be an invariant, regardless of the environment or context.
The problem seems to arise in some aspect of the class loading process:

(.getClassLoader Rec) ;=> WebappClassLoader (delegate: false, repositories: /WEB-INF/classes/, parent: org.apache.catalina.loader.StandardClassLoader@790bc49d)
(.getClassLoader (class (Rec. ...))) ;=> WebappClassLoader (same as the previous line)
(.getClassLoader (class (map->Rec ...))) ;=> clojure.lang.DynamicClassLoader@2e7227a8

The map->Rec delegates to the create method, which seems to be where the problem lies.

The record namespace is AOT compiled, properly I think/hope, and the requisite classes
imported as they should be.

I have attached a minimal web app that reproduces the problem and shows
the configuration. As a sanity check, I have also created a trivial
workaround defrecord* that creates a clojure-native map constructor,
for which the problem does not occur. See the README.md in the tar
file for usage details, and the project.clj for my configuration.

Again, I've only been able to reproduce the problem under Tomcat,
not under Jetty or at the REPL.






[CLJ-1131] Importing a non-existent class causes an exception that does not fully identify the source file Created: 17/Dec/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs


 Description   

I'm in the process of stripping out some OSGi support, and I missed one import.

The exception identifies "init.clj", but I'd prefer to see the full path there, as I have a few different "init.clj" files in my overall project.

:core-services:compileClojure
Reflection warning, com/annadaletech/nexus/services/registry.clj:37 - call to unregisterAll can't be resolved.
Reflection warning, com/annadaletech/nexus/services/registry.clj:131 - call to getConfiguration can't be resolved.
Reflection warning, com/annadaletech/nexus/services/registry.clj:150 - call to getConfiguration can't be resolved.
Exception in thread "main" java.lang.ClassNotFoundException: org.osgi.framework.ServiceRegistration, compiling:(init.clj:1)
	at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3387)
	at clojure.lang.Compiler.compile1(Compiler.java:7035)
	at clojure.lang.Compiler.compile1(Compiler.java:7025)
	at clojure.lang.Compiler.compile(Compiler.java:7097)
	at clojure.lang.RT.compile(RT.java:387)
	at clojure.lang.RT.load(RT.java:427)
	at clojure.lang.RT.load(RT.java:400)
	at clojure.core$load$fn__4890.invoke(core.clj:5415)
	at clojure.core$load.doInvoke(core.clj:5414)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invoke(core.clj:5227)
	at clojure.core$compile$fn__4895.invoke(core.clj:5426)
	at clojure.core$compile.invoke(core.clj:5425)
	at clojuresque.tasks.compile$main$fn__64.invoke(compile.clj:23)
	at clojuresque.cli$with_command_line_STAR_.invoke(cli.clj:92)
	at clojuresque.tasks.compile$main.doInvoke(compile.clj:6)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invoke(core.clj:601)
	at clojure.lang.Var.invoke(Var.java:419)
	at clojuresque.Driver.main(Driver.java:39)
Caused by: java.lang.ClassNotFoundException: org.osgi.framework.ServiceRegistration


 Comments   
Comment by Gabriel Horner [ 17/May/13 3:56 PM ]

While it's reasonable to want this for your case, having long path names in a stacktrace could be inconvenient for others. I'd recommend posting your desired change on the dev list - https://groups.google.com/forum/?fromgroups#!forum/clojure-dev . If they're ok with it, then I'd recommend submitting a patch.





[CLJ-1124] for-as Created: 10/Dec/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Yongqian Li Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

A common pattern in programming is building up some data structure step by step:

In Python:

x = {0: 1}
for item in stuff:
    x[item] = item * x.get(item - 1, 0)

etc.

In an imperative for loop this is easy since we have easy access to the "current" data structure being built up.

I propose the addition of a function for-as similar to as-> except the value of the last loop iteration is bound to the name.

So we can write the above as:

(last (for-as [x {0 1}]
        [item stuff]
  (assoc x item (* item (get x (- item 1) 0)))))

An (un-optimized) implementation might be something like:

(defmacro reduce-for [[res init] for-seq-exprs body-expr]
  `(reduce #(%2 %1) ~init
    (for ~for-seq-exprs
      (fn [~res]
        ~body-expr))))

Note: reduce-for does not return a seq, instead it returns the result of the last loop body iteration.






[CLJ-1119] inconsistent behavior of lazy-seq w/ macro & closure on excptions Created: 03/Dec/12  Updated: 08/Aug/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Hank Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

lazy-seq seems to evaluate inconsistently when body includes a macro and throws and exception. 1st evalutation throws the exceptions, subsequent ones return empty sequence.

demo code:

(defn gen-lazy []
(let [coll [1 2 3]]
(lazy-seq
(when-let [s (seq coll)]
(throw (Exception.))))))

(def lazy (gen-lazy))

(try
(println "lazy:" lazy)
(catch Exception ex
(println ex)))

(try
(println "lazy, again:" lazy)
(catch Exception ex
(println ex)))

It should throw an exception both times but only does on 1st. Generally speaking an expression shouldn't evaluate to different things depending on whether it's been evaluated before.

When removing the closure ...

(defn gen-lazy []
(lazy-seq
(when-let [s (seq [1 2 3])]
(throw (Exception.)))))

... or removing the when-let macro ...

(defn gen-lazy []
(let [coll [1 2 3]]
(lazy-seq
(seq coll)
(throw (Exception.)))))

It works i.e. consistently throws the exception, so seems to be some interaction between the closure and the macro at work here. This particular combination is used in the 'map' function.

See also: https://groups.google.com/forum/?fromgroups=#!topic/clojure/Z3EiBUQ7Inc



 Comments   
Comment by Hank [ 03/Dec/12 4:26 AM ]

N.B. The primary use case I have for this, in case it matters, is interrupting the evaluation of a 'map' expression in which the mapped fn is slow to evaluate (throwing an InterruptedException), because I am not interested in its result any more. Then later I re-evaluate it because I am interested in the result after all, however with above bug the lazy sequence terminates instead of recommencing where it left off.

(UPDATE: This use case is similar to the kind of ersatz continuations that Jetty does (RetryRequest runtime exception) or even Clojure itself when barging STM transactions with the RetryEx exception.)

Comment by Hank [ 03/Dec/12 8:45 AM ]

Related to CLJ-457 according to Christophe. His patch fixes this,too.

Comment by Hank [ 04/Dec/12 5:02 AM ]

Sorry Christophe's patch doesn't work for me here. It avoids evaluating the LazySeq a second time by prematurely throwing an exception. However a LazySeq might evaluate properly the second time around b/c the situation causing the exception was transient. As per comment above an evaluation might get interrupted, throwing InterruptedException the first time around but not the second time.

Also the observation with the closure and macro need explanation IMHO.

Comment by Hank [ 08/Dec/12 3:51 AM ]

further insight: 'delay' exhibits the same behavior and is a more simple case to examine. the macro suspicion is a red herring: as demoed below it is actually the closed over variable magically turns to nil, the when-let macro simply turned that into a nil for the whole expression.

(def delayed
  (let [a true]
    (delay
      (print "a=" a)
      (throw (Exception.)))))

(try
  (print "delayed 1:")
  (force delayed)
  (catch Exception ex (println ex)))

(try
  (print "delayed 2:")
  (force delayed)
  (catch Exception ex (println ex)))

prints:

delayed 1:a= true#<Exception java.lang.Exception>
delayed 2:a= nil#<Exception java.lang.Exception>

Comment by Hank [ 09/Dec/12 1:31 AM ]

The above leads to dodgy outcomes such as: The following expression leads to an Exception on 1st evaluation and to "w00t!" on subsequent ones.

(def delayed
  (let [a true]
    (delay
      (if a
        (throw (Exception.))
        "w00t!"))))

Try it like this:

(try
  (print "delayed 1:" )
  (println (force delayed))
  (catch Exception ex (println ex)))

(try
  (print "delayed 2:")
  (println (force delayed))
  (catch Exception ex (println ex)))

Results in:
delayed 1:#<Exception java.lang.Exception>
delayed 2:w00t!

This code shows that the problem is tied to the :once flag as suspected.

(def test-once
  (let [a true]
    (^{:once true} fn* foo []
	    (println "a=" a)
	    (throw (Exception.)))))

Invoking the fn twice will show 'a' turning from 'true' to 'nil', try it like this:

(try
  (print "test-once 1:")
  (test-once)
  (catch Exception ex (println ex)))

(try
  (print "test-once 2:")
  (test-once)
  (catch Exception ex (println ex)))

Results in:
test-once 1:a= true
#<Exception java.lang.Exception>
test-once 2:a= nil
#<Exception java.lang.Exception>

That doesn't happen when the ^{:once true} is removed. Now one could argue that above fn is invoked twice, which is exactly what one is not supposed to do when decorated with the :once flag, but I argue that an unsuccessful call doesn't count as invocation towards the :once flag. The delay and lazy-seq macros agree with me there as the resulting objects are not considered realized (as per realized? function) if the evaluation of the body throws an exception, and realization/evaluation of the body is therefore repeated on re-evaluation of the delay/lazy-seq.

Try this using

(realized? delayed)
after the first evaluation in the code above. In the implementation this can be seen e.g. here for clojure.lang.Delay (similarly for LazySeq), the body-fn is set to null (meaning realized) after the invocation returns successfully only.

The :once flag affects this part of the compiler only. Some field is set to nil there in the course of a function invocation, for the good reason of letting the garbage compiler collect objects, however this should to be done after the function successfully completes only. Can this be changed?

Comment by Hank [ 16/Dec/12 4:02 AM ]

A workaround for the case of the 'map' function as described in the 1st comment, works as this: The original map function, if you take out the cases for several colls, the performance enhancements for chunked seqs and forcing the coll argument to a seq, looks like this:

(defn map [f s]
  (lazy-seq
    (cons (f (first s)) (map f (rest s)))))

In my workaround I evaluate f twice:

(defn map [f s]
  (lazy-seq
    (f (first s))
    (cons (f (first s)) (map f (rest s)))))

Because the downstream functions that are slow to evaluate are all of the deref kind that cache their result (more lazy-seqs, delays, futures, promises), the InterruptedException can only happen during the 1st evaluation, while the tail call optimization that sets closed-over variables to nil (it pays to read this here: http://clojure.org/lazy) only happens on the second call. The first still creates an fn that captures the head of the sequence 's', however this is not being held onto as it is not returned.

I use this special version of map (and other, similarly rewritten functions based on lazy-seq such as iterate) when I want interruptible, restartable seq evaluations.

Comment by Hank [ 06/Aug/13 9:06 AM ]

Instead of above hack, implementing map and all other combinators using a lazy Y-combinator, and removing the :once meta-data tag in the lazy-seq definition fixes things properly. Since the compiler sort-of-hack that clears closed-over variables that's triggered by the :once meta-data tag is basically only for cases of recursion that can be implemented using the Y-combinator, that won't be needed anymore either.

Comment by Alex Miller [ 08/Aug/13 3:20 AM ]

Re the Delay reference above, this is covered (and addressed) in CLJ-1175 which is waiting for a final screen.





[CLJ-1100] Reader literals cannot contain periods Created: 02/Nov/12  Updated: 10/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Kevin Lynagh Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: reader

Approval: Vetted

 Description   

The reader tries to read a record instead of a literal if the tag contains periods.

user> (binding [*data-readers* {'foo/bar #'identity}] (read-string "#foo/bar 1"))
1
user> (binding [*data-readers* {'foo/bar.x #'identity}] (read-string "#foo/bar.x 1"))
ClassNotFoundException foo/bar.x  java.lang.Class.forName0 (Class.java:-2)

Summary of reader forms:

Kind Example Constraint Status
Record #user.Foo[1] record class name OK
Class #java.lang.String["abc"] class name OK
Clojure reader tag #uuid "c48d7d6e-f3bb-425a-abc5-44bd014a511d" not a class name, no "/" OK
Library reader tag #my/card "5H" not a class name, has "/" OK
  #my.ns/card "5H" not a class name, has "/" OK
  #my/playing.card "5H" not a class name, has "/" BROKEN - read as record

Note: reader tags should not be allowed to override the record reader.

Cause: In LispReader, CtorReader.invoke() decides between record and tagged literal based on whether the tag has a ".".

Proposed: Change the discriminator in CtorReader.

Alternative 1 (purely string inspection):

  • If name has a "/" -> readTagged (not a legal class name)
  • If name has no "/" or "." -> readTagged (records must have qualified names)
  • Else -> readRecord (also covers Java classes)

Tradeoffs: Clojure-defined data reader tags must not contain periods. Not possible to read a Java class with no package. Avoids unnecessary class loading/construction for all tags.

Alternative 2 (prioritize Class check):

  • Attempt readRecord (also covers Java classes)
  • If failed, attempt readTagged

Tradeoffs: Clojure tags could not override Java/record constructors - I'm not sure that's something we'd ever want to do, but this would cut that off. This alternative may attempt classloading when it would not have before.

Hybrids of these are also possible.

Patch:

Screened by:



 Comments   
Comment by Steve Miner [ 06/Nov/12 9:41 AM ]

The suggested patch (clj-1100-reader-literal-periods.patch) will break reading records when *default-data-reader-fn* is set. Try adding a test like this:

(deftest tags-containing-periods-with-default
      ;; we need a predefined record for this test so we (mis)use clojure.reflect.Field for convenience
      (let [v "#clojure.reflect.Field{:name \"fake\" :type :fake :declaring-class \"Fake\" :flags nil}"]
        (binding [*default-data-reader-fn* nil]
          (is (= (read-string v) #clojure.reflect.Field{:name "fake" :type :fake :declaring-class "Fake" :flags nil})))
        (binding [*default-data-reader-fn* (fn [tag val] (assoc val :meaning 42))]
          (is (= (read-string v) #clojure.reflect.Field{:name "fake" :type :fake :declaring-class "Fake" :flags nil})))))
Comment by Rich Hickey [ 29/Nov/12 9:36 AM ]

The problem assessment is ok, but the resolution approach may not be. What happens should be based not upon what is in data-readers but whether or not the name names a class.

Is the intent here to allow readers to circumvent records? I'm not in favor of that.

Comment by Steve Miner [ 29/Nov/12 4:01 PM ]

New patch following Rich's comments. The decision to read a record is now based on the symbol containing periods and not having a namespace. Otherwise, it is considered a data reader tag. User
defined tags are required to be qualified but they may now have periods in the name. Tests added to show that
data readers cannot override record classes. Note: Clojure-defined data reader tags may be unqualified, but they should not contain periods in order to avoid confusion with record classes.

Comment by Steve Miner [ 29/Nov/12 4:17 PM ]

I deleted my old patch and some comments referring to it to avoid confusion.

In Clojure 1.5 beta 1, # followed by a qualified symbol with a period in the name is considered a record and causes an exception for the missing record class. With the patch, only non-qualified symbols containing periods are considered records. That allows user-defined qualified symbols with periods in their names to be used as data reader tags.

Comment by Andy Fingerhut [ 07/Feb/13 9:05 AM ]

clj-1100-periods-in-data-reader-tags-patch-v2.txt dated Feb 7 2013 is identical to CLJ-1100-periods-in-data-reader-tags.patch dated Nov 29 2012, except it applies cleanly to latest master. The only change appears to be in some white space in the context lines.

Comment by Andy Fingerhut [ 07/Feb/13 12:53 PM ]

I've removed clj-1100-periods-in-data-reader-tags-patch-v2.txt mentioned in the previous comment, because I learned that CLJ-1100-periods-in-data-reader-tags.patch dated Nov 29 2012 applies cleanly to latest master and passes all tests if you use this command to apply it.

% git am --keep-cr -s --ignore-whitespace < CLJ-1100-periods-in-data-reader-tags.patch

I've already updated the JIRA workflow and screening patches wiki pages to mention this --ignore-whitespace option.

Comment by Andy Fingerhut [ 13/Feb/13 11:31 AM ]

Both of the current patches, CLJ-1100-periods-in-data-reader-tags.patch dated Nov 29 2012, and clj-1100-reader-literal-periods.patch dated Nov 6 2012, fail to apply cleanly to latest master (1.5.0-RC15) as of today, although they did last week. Given all of the changes around read / read-string and edn recently, they should probably be evaluated by their authors to see how they should be updated.

Comment by Steve Miner [ 14/Feb/13 12:23 PM ]

I deleted my patch: CLJ-1100-periods-in-data-reader-tags.patch. clj-1100-reader-literal-periods.patch is clearly wrong, but the original author or an administrator has to delete that.

Comment by Kevin Lynagh [ 14/Feb/13 1:28 PM ]

I cannot figure out how to remove my attachment (clj-1100-reader-literal-periods.patch) in JIRA.

Comment by Steve Miner [ 14/Feb/13 1:43 PM ]

Downarrow (popup) menu to the right of the "Attachments" section. Choose "manager attachments".

Comment by Kevin Lynagh [ 14/Feb/13 2:02 PM ]

Great, thanks Steve. Are you going to take another pass at this issue, or should I give it a go?

Comment by Steve Miner [ 14/Feb/13 3:04 PM ]

Kevin, I'm not planning to work on this right now as 1.5 is pretty much done. It might be worthwhile discussing the issue a bit on the dev mailing list before working on a patch, but that's up to you. I think my approach was correct, although now changes would have to be applied to both LispReader and EdnReader.

Comment by Alex Miller [ 09/Apr/14 10:29 AM ]

Updated description based on my understanding.





[CLJ-1081] REPL binding not working that works with with-bindings Created: 30/Sep/12  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Steven Devijver Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: repl


 Description   

This works as expected:

java -jar clojure-1.4.0.jar -e "(do (require 'clojure.repl) (.setDynamic #'clojure.repl/print-doc) (with-bindings {#'clojure.repl/print-doc str} (eval '(clojure.repl/doc println))))"

Output:

"{:ns #<Namespace clojure.core>, :name println, :arglists ([& more]), :added \"1.0\", :static true, :doc \"Same as print followed by (newline)\", :line 3325, :file \"clojure/core.clj\"}"

But the same thing does not work in the REPL:

java -jar clojure-1.4.0.jar -e "(do (require 'clojure.repl) (.setDynamic #'clojure.repl/print-doc) (clojure.main/repl :init (fn [] {#'clojure.repl/print-doc str}))))"

Output for Output of {{(doc println)}}:

user=> (doc println)
-------------------------
clojure.core/println
([& more])
Same as print followed by (newline)
nil
user=>




 Comments   
Comment by Steven Devijver [ 01/Oct/12 5:51 AM ]

Found a work-around:

java -jar clojure-1.4.0.jar -e "(do (require 'clojure.repl) (.setDynamic #'clojure.repl/print-doc) (with-bindings {#'clojure.repl/print-doc str} (clojure.main/repl)))))"

I'm still not sure whether the method above using :init should or should not work.





[CLJ-1057] Var's .setDynamic does not set :dynamic in metadata Created: 02/Sep/12  Updated: 03/Dec/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Brandon Bloom Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
((juxt (comp :dynamic meta) #(.isDynamic %)) #'*agent*)


 Comments   
Comment by Timothy Baldridge [ 03/Dec/12 9:10 AM ]

This is actually an enhancement as no where in the clojure code is provision made for syncing var's metadata and dynamic state. .isDynamic is the authoritative source, and the calling of .setDynamic is configured by the compiler. If you'd like to see this change, please, feel free to bring it up on clojure-dev for a discussion.





[CLJ-1043] Unordered literals does not preserve left-to-right evaluation of arguments Created: 16/Aug/12  Updated: 23/Sep/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Brandon Bloom Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Given: (defn f [x] (println x) x)

#{(f 2) (f 1)}

Prints:

1
2

But expected would be:

2
1

This issue is related to CLJS-288



 Comments   
Comment by Andy Fingerhut [ 23/Sep/12 6:01 PM ]

I have the same question as David Nolen for CLJS-288: Is this a bug, or just behavior you didn't expect?

It seems that vectors preserve the order of evaluation, so if you really want to control evaluation order you could use something like (set [(f 2) (f 1)]) or (set (map f [2 1])).

Comment by Brandon Bloom [ 23/Sep/12 7:38 PM ]

I'd consider the expected default behavior of any syntax or macro to evaluate each sub-form once each, from left to right. Conditional, repeated, or out-of-order evaluation should be documented as deviations from that norm. If you buy that, then this is either a code or a documentation bug. My vote is for code bug.





[CLJ-1037] Allow doc strings for both interfaces and concrete implementations Created: 04/Aug/12  Updated: 04/Aug/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Warren Lynn Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

In this post
http://groups.google.com/group/clojure/browse_thread/thread/84de74740928da76#

I mentioned the rationale (I think) why this is important and needed. Thank you for consideration.






[CLJ-1033] pr-str and read-string don't handle @ symbols inside keywords properly Created: 26/Jul/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Steven Ruppert Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: print, reader
Environment:

Ubuntu 12.04 LTS; Java 1.7.0_05 Java HotSpot(TM) Client VM



 Description   
user=> (read-string (pr-str {(keyword "key@other") :stuff}))
RuntimeException Map literal must contain an even number of forms  clojure.lang.Util.runtimeException (Util.java:170)

pr-str emits "{:key@other :stuff}", which read-string fails to interpret correctly. Either pr-str needs to escape the @ symbol, or read-string needs to handle the symbol inside a keyword.

Background: I'm passing a map with email addresses as keys through Storm bolts, which require a thrift-serializable form. Using the pr-str/read-string combo fails on these keys, so I've fallen back to JSON serialization.



 Comments   
Comment by Stuart Halloway [ 10/Aug/12 12:51 PM ]

The '@' character is not a legal character for keywords or symbols (see http://clojure.org/reader). Recategorized as enhancement request.

Comment by Steven Ruppert [ 10/Aug/12 1:04 PM ]

Then why doesn't (keyword "keywith@") throw an exception? It seems inconsistent with your statement.

Comment by Andy Fingerhut [ 13/Sep/12 2:23 PM ]

It is a long standing property of Clojure that it does not throw exceptions for everything that is illegal.

Comment by Steven Ruppert [ 17/Sep/12 2:16 PM ]

Yeah, but read-string clearly does. Is there a good reason that the "keyword" function can't throw an exception? With the other special rules on namespaces within symbol names, the "keyword" function really should be doing validation.

Another solution would be to allow a ruby-like :"symbol with disallowed characters" literal, but that would also be confusing with how the namespace is handled.

https://groups.google.com/forum/?fromgroups=#!topic/clojure/Ct5v9w0yNAE has some older discussion on this topic.

Comment by Andy Fingerhut [ 17/Sep/12 7:43 PM ]

Disclaimer: I'm not a Clojure/core member, just an interested contributor who doesn't know all the design decisions that were made here.

Steven, I think perhaps a couple of concerns are: (1) doing such checks would be slower than not doing them, and (2) implementing such checks means having to update them if/when the rules for legal symbols, keywords, namespace names, etc. change.

Would you be interested in writing strict versions of functions like symbol and keyword for addition to a contrib library? And test suites that try to hit a significant number of corner cases in the rules for what is legal and what is not? I mean those as serious questions, not rhetorical ones. This would permit people that want to use the strict versions of these functions to do so, and at the same time make it easy to measure performance differences between the strict and loose versions.

Comment by Steven Ruppert [ 13/Jan/13 10:58 PM ]

Looking back at this, the root cause of the problem is that the {pr} function, although it by default "print[s] in a way that objects can be read by the reader" [0], doesn't always do so. Thus, the easiest "fix" is to change its docstring to warn that not all keywords can be read back.

The deeper problem is that symbol don't have a reader form that can represent all actually possible keywords (in this case, those with "@" in them). Restricting the actually-possible keywords to match the reader form, i.e. writing a strict "keyword" function actually seems like a worse solution overall to me. The better solution would be to somehow extend the keyword reader form to allow it to express all possible keywords, possibly ruby's :"keyword" syntax. Plus, that solution would avoid having to keep hypothetical strict keyword/symbol functions in sync with reader operation, and write test cases for that, and so on.

Thus, the resolution of this bug comes down to how far we're willing to go. Changing the docstring would be the easiest, but extending the keyword form would be the "best" resolution, IMO.

[0]: http://clojuredocs.org/clojure_core/clojure.core/pr

Comment by Andy Fingerhut [ 19/Aug/13 10:54 AM ]

I happened across ticket CLJ-17 yesterday. Its discussion thread shows that the topic of validating the contents of constructed keywords and symbols has arisen before. At the time, a patch was written that modified the functions "symbol" and "keyword" so that the symbol/keyword was constructed as it does now, but then it was double-checked for readability using the clojure.lang.RT/readString method on the string arg given. It threw an exception if the intern and readString methods returned non-equal symbols (or if readString threw an exception).

Rich was concerned that this run-time overhead would be too high, and asked if anyone knew a faster way of doing it. Chas Emerick proposed making all symbols readable using a syntax like Common Lisp's #|symbol with whitespace|, with some checks for the common case where the quoting would be unnecessary. Rich was open to the idea of quoting arbitrary symbols, but that it would be a different ticket than that one.

I am not aware of anyone creating a ticket to introduce quoting of arbitrary symbols since then, but I could have missed it. This ticket could become that ticket, but its description would need significant editing, and code changes would be needed in a variety of places in Clojure.





[CLJ-1027] Outdated documentation for gen-class's :exposes-methods option Created: 18/Jul/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Dan Lidral-Porter Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring


 Description   

The docstring for gen-class says the following regarding the :exposes-methods option:

"It is sometimes necessary to call the superclass' implementation of an
overridden method. Those methods may be exposed and referred in
the new method implementation by a local name."

To me, this suggests that supplying something like `{foo fooSuper}` allows me to use the symbol `fooSuper` in my new method implementation. Doing this actually results in an error while compiling because `fooSuper` cannot be resolved. It seems that what actually happens is that a `fooSuper` instance method is defined, which calls the superclass's implementation. The docstring should be updated to reflect this.






[CLJ-1022] gen-class destroys method annotations Created: 03/Jul/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Maris Orbidans Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop


 Description   

When extending a class gen-class doesn't preserve method annotations.

If class com.bar.Foo has annotated methods then in MyClass all annotations are gone.

(gen-class
:name com.my.MyClass
:extends com.bar.Foo
:implements [com.google.common.base.Supplier]
:prefix demo-
:post-init post-init)

(defn demo-post-init [this]
(info "initialized")
(swank.swank/start-server :port 68478))

(defn demo-get [_]
(get-msg))

Class<?> aClass = Class.forName("com.my.MyClass");
Method[] methods = aClass.getMethods();

for (Method m : methods) {
Annotation[] annotations = m.getAnnotations();
System.out.println(m.getName()+" "+annotations.length);
for (Annotation a : annotations) { System.out.println(a.annotationType().getClass().getName()); }
}






[CLJ-1017] Metadata expressions are evaluated after the expression they affect Created: 23/Jun/12  Updated: 23/Jun/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Brandon Bloom Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Repro:

user=> (def x (atom 1))
#'user/x
user=> ^{:foo (swap! x inc)} {:bar (swap! x inc)}
{:bar 2}
user=> (meta *1)
{:foo 3}

Presumably this is because ^:foo x expands to (with-meta x {:foo true}) but probably should have reversed argument order or use a let expression.






[CLJ-1016] Global scope overrides lexical scope for classes (Clojure assumes no classes in default package / Clojure cannot handle yFiles JARs in classpath) Created: 21/Jun/12  Updated: 24/May/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Edward Z. Yang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File collision-workaround.patch    

 Description   

The most visible symptom of this bug is having a class named 'w' (default package) in your classpath (such classes are produced by Java obfuscation tools such as yFiles) and then attempting to load Clojure's core class. For example:

java -cp hotspotapi.jar:clojure-1.4.0-slim.jar clojure.main

(where hotspotapi.jar is a stereotypical example of an obfuscated JAR) results in:

Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
Caused by: java.lang.NoSuchFieldException: close, compiling:(clojure/core.clj:6139)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6462)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
at clojure.lang.Compiler$TryExpr$Parser.parse(Compiler.java:2178)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5919)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5054)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3674)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6453)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.access$100(Compiler.java:37)
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:518)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
at clojure.lang.Compiler.analyze(Compiler.java:6262)
at clojure.lang.Compiler.analyze(Compiler.java:6223)
at clojure.lang.Compiler.eval(Compiler.java:6515)
at clojure.lang.Compiler.load(Compiler.java:6952)
at clojure.lang.RT.loadResourceScript(RT.java:359)
at clojure.lang.RT.loadResourceScript(RT.java:350)
at clojure.lang.RT.load(RT.java:429)
at clojure.lang.RT.load(RT.java:400)
at clojure.lang.RT.doInit(RT.java:436)
at clojure.lang.RT.<clinit>(RT.java:318)
... 1 more
Caused by: java.lang.NoSuchFieldException: close
at java.lang.Class.getField(Class.java:1537)
at clojure.lang.Compiler$StaticFieldExpr.<init>(Compiler.java:1180)
at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:923)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
... 37 more
Could not find the main class: clojure.main. Program will exit.

To understand what is going on, consider this simple test:

import java.io.StringReader;

import clojure.lang.Compiler;
import clojure.lang.RT;

public class Test {
public static void main(String[] args) { RT.var("clojure.core", "require"); String s = "(let [mumble (new java.io.StringReader \"\")] (. mumble close))"; Compiler.load(new StringReader(s)); }
}

It should be clear that 'mumble' in the dot operator is referencing the locally defined mumble. However, if we define a class named 'mumble' in the default package, Clojure picks that one up instead.

To forestall any objections: yes, we know that placing classes in the default package is extremely poor form. Point of the matter is, the Java ecosystem is extremely diverse and there are a lot of JARs people may not have control over. While one might argue, "Don't put classes in the default namespace", point of the matter is, Clojure is wrong here, and these situations arise in practice, through no fault of the implementer.



 Comments   
Comment by Edward Z. Yang [ 21/Jun/12 11:01 AM ]

Here is a workaround patch which makes this error less likely to occur.

Comment by Andy Fingerhut [ 27/Aug/12 7:37 PM ]

Edward, it is Rich Hickey's policy only to consider for inclusion in Clojure patches written by people who have signed a Contributor Agreement: http://clojure.org/contributing

Were you interested in becoming a contributor?

Comment by Edward Z. Yang [ 27/Aug/12 9:24 PM ]

Sure, although the patch attached is emphatically not the one you want to actually applying, since it only band-aids the problem.

Comment by Andy Fingerhut [ 24/May/13 1:21 PM ]

I am not sure, but this ticket may be related to CLJ-1171. At least, there the issue was a global name not being shadowed by a local name bound with let. That seems similar to this issue.





[CLJ-1015] Standalone Clojure library for Java users Created: 14/Jun/12  Updated: 14/Jun/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Edward Z. Yang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Many of Clojure's data structures (e.g. PersistentHashMap) may be of interest to the wider Java community, and would benefit from being packaged in a way that makes them easy to include in projects. While they are public (and thus can be accessed by way of clojure.lang), Clojure's classloader is implemented in a way such that Clojure files such as core.clj end up being loaded, even when an end-user is not interested in the Clojure environment itself. The Java classes could also use some documentation!

I'd be happy to work on a patch but this change may require some restructuring of the build process, so it'd be good to get community sign-off first.



 Comments   
Comment by Andy Fingerhut [ 14/Jun/12 5:39 PM ]

I am assuming this ticket is a request that the original Clojure distribution be modified to easily build such a library of data structures, and be maintained as a separate build target, going forward.

Reference to related info: Below is a copy of most of a message from someone with userid Krukow posted to the Clojure Google group on June 3, 2012: https://groups.google.com/forum/?fromgroups#!topic/clojure/UyjmafY_ZE8

I created a project containing only the persistent data structures for use with Java et al.

https://github.com/krukow/clj-ds

It is the data structures only so no bootstrap penalty. There are also Java'ish "improvements" like basic Generics and improved performance on the iterator pattern.

It's still on Clojure 1.3 (As far as I recall), but I am planning on taking another iteration.

TODO:

  • better Generics support
  • more data structures (tries, RRB-trees)
  • include the reducers library support for parallelism




[CLJ-1013] Clojure's classloader cannot handle out-of-order loading Created: 13/Jun/12  Updated: 13/Jun/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Edward Z. Yang Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Here is a minimal test-case:

import java.io.IOException;

import clojure.lang.PersistentTreeMap;
import clojure.lang.RT;

public class TestClass {

static Class y = RT.class;
//static PersistentTreeMap x = PersistentTreeMap.EMPTY;

/**

  • @param args
  • @throws ClassNotFoundException
  • @throws IOException
    */
    public static void main(String[] args) throws IOException, ClassNotFoundException { PersistentTreeMap x = PersistentTreeMap.EMPTY; }

}

This results in the exception:

Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at clojure.lang.RT.loadClassForName(RT.java:2056)
at clojure.lang.RT.load(RT.java:419)
at clojure.lang.RT.load(RT.java:400)
at clojure.lang.RT.doInit(RT.java:436)
at clojure.lang.RT.<clinit>(RT.java:318)
at clojure.lang.PersistentTreeMap.<init>(PersistentTreeMap.java:45)
at clojure.lang.PersistentTreeMap.<clinit>(PersistentTreeMap.java:32)
at TestClass.main(TestClass.java:19)
Caused by: java.lang.NullPointerException
at clojure.lang.APersistentSet.contains(APersistentSet.java:33)
at clojure.lang.RT.contains(RT.java:700)
at clojure.core$contains_QMARK_.invoke(core.clj:1386)
at clojure.core$load_lib.doInvoke(core.clj:5255)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:603)
at clojure.core$load_libs.doInvoke(core.clj:5298)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:603)
at clojure.core$require.doInvoke(core.clj:5381)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core__init.load(Unknown Source)
at clojure.core__init.<clinit>(Unknown Source)
... 10 more

The crux of the issue appears Clojure's classloader doesn't understand how to handle out-of-order classloading.






[CLJ-1008] Make sorted maps and sets implement j.u.NavigableMap and NavigableSet interfaces Created: 02/Jun/12  Updated: 02/Jun/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Jim Blomo Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Since Clojure 1.5 will probably no longer target JVM 5, add support for the (Concurrent)?Navigable(Map|Set) interfaces. This should involve adding functions to PersistentTreeMap that descend the red-black tree to select the closest items.






[CLJ-1002] chunk-* functions not documented Created: 27/May/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Jim Blomo Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: documentation


 Description   

None of the chunk related functions defined in core.clj have documentation. While their implementations are straightforward, it means the functions do not show up in http://clojure.org/api. Are these not considered part of the API? If so, should they be private? Otherwise, I think they should have public documentation.

For searchability, the function are:

chunk-append, chunk, chunk-first, chunk-rest, chunk-next, chunk-cons, chunked-seq?






[CLJ-1001] Proxy cannot call proper super-class method Created: 23/May/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2, Release 1.3
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Guanpeng Xu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop
Environment:

Linux herberteuler 3.2.0-2-amd64 #1 SMP Sat May 12 23:08:28 UTC 2012 x86_64 GNU/Linux


Attachments: File proxy-bug.clj    

 Description   

Attached is a program that reproduces this issue. We have a proxy, `p', which sub-classes java.io.InputStream. There are three methods named `read' in java.io.InputStream: abstract int read(); int read(byte[] b); and int read(byte[] b, int off, int len); see http://docs.oracle.com/javase/6/docs/api/java/io/InputStream.html. In the definition of proxy `p', we implement the abstract variant of method `read', making `p' a concrete instance of java.io.InputStream.

The first invocation, (. p read), returns -1, which is expected.

The second invocation, (. p (read b 0 n)), should call int read(byte[] b, int off, int len); in java.io.InputStream. But these are actual behavior:

$ clojure1.2 ~/tmp/proxy-bug.clj
Exception in thread "main" java.lang.IllegalArgumentException: Wrong number of args (4) passed to: user$eval1$fn (proxy-bug.clj:0)
at clojure.lang.Compiler.eval(Compiler.java:5441)
at clojure.lang.Compiler.load(Compiler.java:5858)
at clojure.lang.Compiler.loadFile(Compiler.java:5821)
at clojure.main$load_script.invoke(main.clj:221)
at clojure.main$script_opt.invoke(main.clj:273)
at clojure.main$main.doInvoke(main.clj:354)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:365)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.Var.applyTo(Var.java:482)
at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalArgumentException: Wrong number of args (4) passed to: user$eval1$fn
at clojure.lang.AFn.throwArity(AFn.java:437)
at clojure.lang.AFn.invoke(AFn.java:51)
at user.proxy$java.io.InputStream$0.read(Unknown Source)
at user$eval1.invoke(proxy-bug.clj:9)
at clojure.lang.Compiler.eval(Compiler.java:5425)
... 10 more

$ clojure1.2 ~/tmp/proxy-bug.clj
Exception in thread "main" java.lang.IllegalArgumentException: Wrong number of args (4) passed to: user$eval1$fn (proxy-bug.clj:0)
at clojure.lang.Compiler.eval(Compiler.java:5441)
at clojure.lang.Compiler.load(Compiler.java:5858)
at clojure.lang.Compiler.loadFile(Compiler.java:5821)
at clojure.main$load_script.invoke(main.clj:221)
at clojure.main$script_opt.invoke(main.clj:273)
at clojure.main$main.doInvoke(main.clj:354)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:365)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.Var.applyTo(Var.java:482)
at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalArgumentException: Wrong number of args (4) passed to: user$eval1$fn
at clojure.lang.AFn.throwArity(AFn.java:437)
at clojure.lang.AFn.invoke(AFn.java:51)
at user.proxy$java.io.InputStream$0.read(Unknown Source)
at user$eval1.invoke(proxy-bug.clj:9)
at clojure.lang.Compiler.eval(Compiler.java:5425)
... 10 more



 Comments   
Comment by Guanpeng Xu [ 23/May/12 10:24 PM ]

The second behavior should be in Clojure 1.3:

$ clojure1.3 ~/tmp/proxy-bug.clj
Exception in thread "main" clojure.lang.ArityException: Wrong number of args (4) passed to: user$eval1$fn
at clojure.lang.AFn.throwArity(AFn.java:437)
at clojure.lang.AFn.invoke(AFn.java:51)
at user.proxy$java.io.InputStream$0.read(Unknown Source)
at user$eval1.invoke(proxy-bug.clj:9)
at clojure.lang.Compiler.eval(Compiler.java:6468)
at clojure.lang.Compiler.load(Compiler.java:6905)
at clojure.lang.Compiler.loadFile(Compiler.java:6866)
at clojure.main$load_script.invoke(main.clj:282)
at clojure.main$script_opt.invoke(main.clj:342)
at clojure.main$main.doInvoke(main.clj:426)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:401)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.Var.applyTo(Var.java:518)
at clojure.main.main(main.java:37)

Sorry for the inconvenience.

Comment by Russell Mull [ 01/Sep/13 3:12 AM ]

Verified with Clojure 1.5.1:

Stack Trace
clojure.lang.ArityException: Wrong number of args (4) passed to: user$eval147$fn
                                      AFn.java:437 clojure.lang.AFn.throwArity
                                       AFn.java:51 clojure.lang.AFn.invoke
                                  (Unknown Source) user.proxy/java.io.InputStream[fn]
                                  NO_SOURCE_FILE:9 user/eval147
                                Compiler.java:6619 clojure.lang.Compiler.eval
                                Compiler.java:6582 clojure.lang.Compiler.eval
                                     core.clj:2852 clojure.core/eval
                                      main.clj:259 clojure.main/repl[fn]
                                      main.clj:259 clojure.main/repl[fn]
                                      main.clj:277 clojure.main/repl[fn]
                                      main.clj:277 clojure.main/repl
                                  RestFn.java:1096 clojure.lang.RestFn.invoke
                         interruptible_eval.clj:56 clojure.tools.nrepl.middleware.interruptible-eval/evaluate[fn]
                                      AFn.java:159 clojure.lang.AFn.applyToHelper
                                      AFn.java:151 clojure.lang.AFn.applyTo
                                      core.clj:617 clojure.core/apply
                                     core.clj:1788 clojure.core/with-bindings*
                                   RestFn.java:425 clojure.lang.RestFn.invoke
                         interruptible_eval.clj:41 clojure.tools.nrepl.middleware.interruptible-eval/evaluate
                        interruptible_eval.clj:171 clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval[fn]
                                     core.clj:2330 clojure.core/comp[fn]
                        interruptible_eval.clj:138 clojure.tools.nrepl.middleware.interruptible-eval/run-next[fn]
                                       AFn.java:24 clojure.lang.AFn.run
                      ThreadPoolExecutor.java:1110 java.util.concurrent.ThreadPoolExecutor.runWorker
                       ThreadPoolExecutor.java:603 java.util.concurrent.ThreadPoolExecutor$Worker.run
                                   Thread.java:722 java.lang.Thread.run




[CLJ-997] max-key and min-key to return the first entry in case of several candidates Created: 18/May/12  Updated: 18/May/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3, Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Oleksandr Shyshko Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Consider the following code:

(def values [[1 2] [3 4] [5] [6 7] [8]])
(apply max-key count values)
; => [6 7]

Which returns the last max entry [6 7]. Why its not the first max entry [1 2]?
Well, truth is "max-key" gives no warranty on which max value will be returned.

Consider the following example in Scala:

println(List(List(0, 1, 2), List(2, 3, 4), List(1), List(1, 2, 3)).maxBy(_.length))
> List(0, 1, 2)

The very same function in Scala returns the first max entry (by default).

The code from relase 1.4 file "clojure/core.clj#4419-4426" looks is:
=======================
4419: (defn max-key
4420: "Returns the x for which (k x), a number, is greatest."
4421: {:added "1.0"
4422: :static true}
4423: ([k x] x)
4424: ([k x y] (if (> (k x) (k y)) x y))
4425: ([k x y & more]
4426: (reduce1 #(max-key k %1 %2) (max-key k x y) more)))
=======================

I am unsure what is the motivation in returning the last candidate, but

I suggest two following things:

1. Make "max-key" and "min-key" return the first max/min entry if there are several candidates.

This behavior seems more natural/convenient (to me), because in most cases you want to get first "winner".
(e.g. find the first biggest vector in a sequence) and less often you need to get the last entry – in those cases
you can do "reverse" before feeding a sequence to "max-key", thus it seems having "return first max" behavior more useful.

Line #4424 should have ">=" instead of ">".

2. Make "max-key" and "min-key" make warranty on order, which max/min entry will be returned (either first or last).

Line #4420 should say "Returns the x for which (k x), a number, is greatest. In case of several matches, the first max entry will be returned" or the same doc, but saying "the last max entry will returned".



 Comments   
Comment by Steve Miner [ 18/May/12 4:03 PM ]

The current behavior matches the documentation so I wouldn't consider it a "defect". There doesn't seem to be a compelling reason to impose a tie-breaker rule on the implementation.





[CLJ-995] sorted-set doesn't support IEditableCollection Created: 13/May/12  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Moritz Ulrich Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: collections


 Description   

I think sorted-set (PersistentTreeSet) should implement the transient interface. It's a special-purpose set and should be usable just like every normal set.



 Comments   
Comment by Michel Alexandre Salim [ 04/Jun/12 2:32 AM ]

Note that this would require PersistentTreeMap to implement IEditableCollection as well.





[CLJ-986] Adds an exit function to exit clojure process Created: 06/May/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: dennis zhuang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

There is no standard function to exit the clojure process.
In java implementation,we use (System/exit 0),but in other implementations(CLR), i have to use another function.

Why not add a standard function in clojure.core?
For example:

(defn exit
([] (exit 0)
([status] (System/exit status)))

I think it's useful for us.






[CLJ-979] map->R returns different class when invoked from AOT code Created: 03/May/12  Updated: 29/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3, Release 1.4, Release 1.5, Release 1.6
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Edmund Jackson Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: aot
Environment:

Mac OS X 10.5, lein 1.7 and lein 2.0


Attachments: Text File clj-979-symptoms.patch    

 Description   

Compiling a class via `deftype` during AOT compilation gives different results for the different constructors. These hashes should be identical.

user=> (binding [*compile-files* true] (eval '(deftype Abc [])))
user.Abc
user=> (hash Abc)
16446700
user=> (hash (class (->Abc)))
31966239


 Comments   
Comment by Scott Lowe [ 12/May/12 9:05 PM ]

I can't reproduce this under Clojure 1.3 or 1.4, and Leiningen 1.7.1 on either Java 1.7.0-jdk7u4-b21 OpenJDK 64-Bit or Java 1.6.0_31 Java HotSpot 64-Bit. OS is Mac OS X 10.7.

Edmund, how are you running this AOT code? I wrapped your code in a main function and built an uberjar from it.

Comment by Edmund Jackson [ 13/May/12 2:20 AM ]

Hi Scott,

Interesting.

I have two use cases
1. AOT compile and call from repl.
My steps: git clone, lein compile, lein repl, (use 'aots.death), (in-ns 'aots.death), (= (class (Dontwork. nil)) (class (map->Dontwork {:a 1}))) => false

2. My original use case, which I've minimised here, is an AOT ns, producing a genclass that is called instantiated from other Java (no main). This produces the same error. I will produce an example of this and post it too.

Comment by Edmund Jackson [ 13/May/12 4:23 AM ]

Hi Scott,

Here is an example of it failing in the interop case: https://github.com/ejackson/aotquestion2
The steps I'm following to compile this all up are

git clone git@github.com:ejackson/aotquestion2.git
cd aotquestion2/cljside/
lein uberjar
lein install
cd ../javaside/
mvn package
java -jar ./target/aotquestion-1.0-SNAPSHOT.jar

and it dies with this:

Exception in thread "main" java.lang.ClassCastException: cljside.core.Dontwork cannot be cast to cljside.core.Dontwork
at cljside.MyClass.makeDontwork(Unknown Source)
at aotquestion.App.main(App.java:8)

The error message is really confusing (to me, anyway), but I think its the same root problem as for the REPL case.

What do you see when you run the above ?

Comment by Scott Lowe [ 13/May/12 8:41 AM ]

Ah, yes, looks like my initial attempt to reproduce was too simplistic. I used your second git repo, and can now confirm that it's failing for me with the same error.

Comment by Scott Lowe [ 13/May/12 10:35 PM ]

I looked into this a little further and the AOT generated code looks correct, in the sense that both code paths appear to be returning the same type.

However, I wonder if this is really a ClassLoader issue, whereby two definitions of the same class are being loaded at different times, because that would cause the x.y.Class cannot be cast to x.y.Class exception that we're seeing here.

Comment by Steve Miner [ 03/Sep/13 9:54 AM ]

This could be related to CLJ-1157 which deals with a ClassLoader issue with AOT compiled code.

Comment by Ambrose Bonnaire-Sergeant [ 29/Mar/14 1:11 PM ]

I've tried this patch attached to CLJ-1157 and it did not solve this issue.

Comment by Ambrose Bonnaire-Sergeant [ 29/Mar/14 2:27 PM ]

This bug seems to be rooted in different behaviour for do/let under compilation. Attached a patch showing these symptoms in the hope it helps people find the cause.





[CLJ-969] Symbol/keyword implements IFn for lookup but a non-collection argument produces non-intuitive results Created: 09/Apr/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Sean Corfield Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs


 Description   

('+ 1 2) ;; return 2 because it is treated as (get 1 '+ 2)

Whilst this is "consistent" once you know the lookup behavior, it's confusing for Clojure newbies and it seems to be a non-useful behavior.

Proposal: modify Keyword.invoke() and Symbol.invoke() to restrict first Object argument to instanceof ILookup, Map or IPersistentSet (or null) so that the "not found" behavior doesn't produce non-intuitive behavior.






[CLJ-968] ns emitting gen-class before imports results in imported annotations being discarded. Created: 09/Apr/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3, Release 1.4
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Charles Duffy Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop


 Description   

The following discards the imported annotations:

(ns com.example.BaseXModuleTest
  (:import (org.basex.query QueryModule QueryModule$Deterministic))
  (:gen-class
     :extends org.basex.query.QueryModule
     :methods [
       [^{QueryModule$Deterministic {}}
        addOne [int] int]]))

However, when moving the gen-class call out of the ns declaration, the annotation is correctly applied:

(ns com.example.BaseXModuleTest
  (:import (org.basex.query QueryModule QueryModule$Deterministic)))

(gen-class
  :extends org.basex.query.QueryModule
  :name com.example.BaseXModuleTest
  :methods [
    [^{QueryModule$Deterministic {}}
     addOne [int] int]])

It appears that imported names are not yet in-scope when gen-class is run from a ns declaration.






[CLJ-955] java object reader constructor doesn't work Created: 18/Mar/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Brent Millare Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reader


 Description   

Here is a transcript:

;user=> clojure-version
{:major 1, :minor 4, :incremental 0, :qualifier "beta5"}
;user=> (.getProtocol #java.net.URL["file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"])
java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs, compiling:(null:2)
;user=> (.getProtocol (java.net.URL. "file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"))
"file"

Another transcript from google groups https://groups.google.com/forum/?fromgroups&hl=en#!topic/clojure/vlsFgVaKcSQ

user=> (def x #java.net.URL["file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"])
#'user/x
user=> x
#<URL file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs>
user=> (.getProtocol x)
"file"

user=> (.getProtocol #java.net.URL["file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"])
CompilerException java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs, compiling:(NO_SOURCE_PATH:5)
user=> (defmethod print-dup java.net.URL [o, ^java.io.Writer w] (.write w (str o)))
#<MultiFn clojure.lang.MultiFn@2e694f12>

user=> (.getProtocol #java.net.URL["file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"])
ClassCastException clojure.lang.Symbol cannot be cast to java.net.URL user/eval11 (NO_SOURCE_FILE:7)
user=>

user=> (def x #java.net.URL["file:///home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"])
#'user/x
user=> (printf "(class x)=%s x='%s'\n" (class x) x)
(class x)=class java.net.URL x='file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs'
nil
user=> (let [x #java.net.URL["file:///home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"]]
(printf "(class x)=%s x='%s'\n" (class x) x))
CompilerException java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs, compiling:(NO_SOURCE_PATH:4)

user=> (defmethod print-dup java.net.URL [o, ^java.io.Writer w] (.write w (str o)))
#<MultiFn clojure.lang.MultiFn@3362a63>
user=> (let [x #java.net.URL["file:///home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"]]
(printf "(class x)=%s x='%s'\n" (class x) x))
(class x)=class clojure.lang.Symbol x='file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs'
nil



 Comments   
Comment by Andy Fingerhut [ 19/Mar/12 2:58 AM ]

I've confirmed this behavior with 1.4.0 beta5. I've only tracked it down as far as finding the "Can't embed object in code" message, which is easy to find in Compiler.java in method emitValue. That exception is thrown because printString in RT.java throws an exception.

It isn't clear to me what should be done instead, though.

Comment by Fogus [ 19/Mar/12 9:03 AM ]

What would it mean to construct an arbitrary Java object for the purpose of embedding it in code in a generic way? You could say that it's just a matter of calling its constructor with the right args but very often in Java that is not enough to make an object considered "initialize". Sometimes there are init methods or putters or whatever that are required for object construction. So right there I hope it's clear that even though #some.Klass["foo"] provides a way to call an arbitrary constructor, it's in no way a guarantee that an instance is properly constructed. The reason that (for example) defrecords are embeddable anywhere is because we know for certain that the constructor creates a fully initialized instance. If you need to embed specific instances in your own code then you have two options:

  • Implement a print-dup for the class that guarantees a fully initialized object is built.
  • Use Clojure 1.4's tagged literal feature to do the same.

Quick point of note:

Your code

(defmethod print-dup java.net.URL [o, ^java.io.Writer w] (.write w (str o)))

Doesn't do what you think it does. It spits out exactly file:///home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs that Clojure reads in as a symbol. Something like the following might be more appropriate:

(defmethod print-dup java.net.URL [o, ^java.io.Writer w] (.write w "#java.net.URL") (.write w (str [ (str o) ])))

(let [x #java.net.URL["file:///home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs"]]                             
  (printf "(class x)=%s x='%s'\n" (class x) x) x)                                                                          

; (class x)=class java.net.URL x='file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs'

;=> #<URL file:/home/hara/dj/usr/src/clojurescript/src/cljs/cljs/core.cljs>

(.getProtocol *1)                        
;=> "file"
Comment by Brent Millare [ 19/Mar/12 10:09 AM ]

Fogus, can you please elaborate on using clojure 1.4's tagged literal features. While I understand that you can define data-reader functions, for example

(binding [*data-readers* {'user/f (fn [x] (java.io.File. (first x)))}] (read-string "#user/f [\"hello\"]")) ;=> #<File hello>

however, I feel this is only half a fix, compared with the first mentioned solution (involving print-dup), since clojure's tagged literals only are important for reading, not for printing. Does using tagged literals, so that (read-string (pr-str (read-string ...) works, imply that you must also define print methods per class? If this is true, it seems problematic since if different code wants to define different print methods, this will conflict since defining print-dup methods is global. Is there a good solution for printing objects depending on the context? As an alternative solution, I propose making the default print-method of all objects that didn't already have a printed representation to be a tagged literal, this way, users can customize what it means to read it. (See https://groups.google.com/forum/?fromgroups&hl=en#!topic/clojure/GdT5cO6JoSQ )

Comment by Fogus [ 19/Mar/12 10:18 AM ]

"Fogus, can you please elaborate on using clojure 1.4's tagged literal features."

I can, but it falls outside of the scope of this particular ticket. It might be better to take the broader conversation to the design page at http://dev.clojure.org/display/design/Tagged+Literals and the clojure-dev list.

Has the original motivation for this ticket been addressed?

Comment by Brent Millare [ 19/Mar/12 10:26 AM ]

I think you've explained the underlying problem, so yes. One possible caveat might be that it may be a concern that the current error message is not telling that the underlying problem lies in an improperly initialized object. (or maybe it is, and that's the error message I should expect).





[CLJ-946] eval reader fail to recognize function object Created: 06/Mar/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Naitong Xiao Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reader


 Description   

(defmacro stubbing [stub-forms & body]
(let [stub-pairs (partition 2 stub-forms)
returns (map last stub-pairs)
stub-fns (map constantly returns) ;;(map #(list 'constantly %) returns)
real-fns (map first stub-pairs)]
`(binding [~@(interleave real-fns stub-fns)]
~@body)))

This macro is from Clojure In Action , whith the commented line rewrited (I know that original is better)
I assumed that this macro would work as supposed , if the stub forms use compile time literal, for e.g

(def ^:dynamic $ (fn [x] (* x 10))

(stubbing [$ 1]
($ 1 1))
;; =>
IllegalArgumentException No matching ctor found for class clojure.core$constantly$fn__3656
clojure.lang.Reflector.invokeConstructor (Reflector.java:166)
clojure.lang.LispReader$EvalReader.invoke (LispReader.java:1031)
clojure.lang.LispReader$DispatchReader.invoke (LispReader.java:618)

;;macro can expanded
(macroexpand-all '(stubbing [$ 1]
($ 1 1)))
;; =>
(let* [] (clojure.core/push-thread-bindings (clojure.core/hash-map (var $) #<core$constantly$fn_3656 clojure.core$constantly$fn_3656@161f888>)) (try ($ 1 1) (finally (clojure.core/pop-thread-bindings))))

I thought there is something wrong with eval reader, but i can't figure it out

btw the above code can run on clojure-clr



 Comments   
Comment by Michael Klishin [ 06/Mar/12 11:24 PM ]

As far as I know, Clojure in Action examples are written for Clojure 1.2. What version are you using?

Comment by Naitong Xiao [ 07/Mar/12 1:24 AM ]

I use clojure 1.3

The example in Clojure In Action is Ok on clojure 1.3
I just found this peculiar thing when trying to answer a question from another one in a mail list.





[CLJ-941] NullPointerException possible with seq-zip Created: 26/Feb/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Greg Chapman Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: zip


 Description   

For example:

Clojure 1.3.0
user=> (require '[clojure.zip :as z])
nil
user=> (-> (z/seq-zip (list 1)) z/down z/remove)
NullPointerException clojure.core/with-meta (core.clj:211)

Possibly the make-node function for seq-zip should be:

(fn [node children] (with-meta (or children ()) (meta node)))



 Comments   
Comment by Greg Chapman [ 26/Feb/12 5:54 PM ]

Also the docstring for zipper should probably be updated to indicate that the children parameter can be nil.





[CLJ-938] Output of clojure.reflect is not suitable for type hints Created: 23/Feb/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Brandon Bloom Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reflection, typehints


 Description   

I'm trying to write a macro which generate reify forms using some reflection.

clojure.reflect produces type symbols similar to 'java.util.Iterator<>

Unfortunately, the compiler doesn't accept the <> syntax in type hints:

(reify Iterable (^{:tag java.util.Iterator} iterator [this] nil)) ; works

(reify Iterable (^{:tag java.util.Iterator<>} iterator [this] nil)) ;; fails
CompilerException java.lang.RuntimeException: java.lang.ClassNotFoundException: java.util.Iterator<>, compiling:(NO_SOURCE_PATH:1325)

It seems like the compiler should understand the <> just enough to strip it, rather than reject it. This would make it much easier to write correct macros involving type hinting and reflection.

The workaround I have been using is:

(defn hint
"clojure.reflect demarks generics with angle brackets, but
the compiler does not support generics in type hints"
[obj tag]
(let [tag (-> tag .toString (.replace "<>" "") symbol)]
(with-meta obj {:tag tag}))



 Comments   
Comment by Brandon Bloom [ 24/Feb/12 1:37 AM ]

I'm sorry, I got this wrong earlier. The problem is real, but it's arrays, not generics.

My workaround is useless... :-/





[CLJ-929] Accessing Java property starting with _ has issues in 1.4 Created: 07/Feb/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.4
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Alan Malloy Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: defrecord


 Description   

When attempting to use interop syntax with symbols which aren't legal Java names (such as deleted?), the names are mangled a bit. That's necessary, of course, and the method of munging can be internal to the compiler. However, the behavior when munging changed a little between 1.3 and 1.4 beta1. Obviously the specifics of munging are something I should avoid relying on, but the way it changed looks like an accident or a bug even so.

The use-case I ran into is that defrecords contain a field named __meta for tracking their metadata. In both 1.3 and 1.4 you can get at that field with (. record __meta), which avoids munging. But on 1.3 (. record --meta) also accesses it (translating each - to a _), while on 1.4 (. record -meta) works and (. record --meta) doesn't.

Actually, looking at line 883 of Compiler.java, it looks like this may be related to the (. foo -property) syntax ported from CLJS, and indeed (. record ---meta) works, I guess by reducing to an "old style" (. record --meta). So that clears up why --meta fails: it's looking for __meta. I'm still not clear on why (. record -meta) works, though.

So it looks like the - prefix for properties is not 100% backwards-compatible like it seemed to be. Is this an issue we need to fix, or is the recommendation simply to never have fields that start with - or _?



 Comments   
Comment by Fogus [ 09/Feb/12 2:33 PM ]

Is this a general problem with fields starting with _ or just fields named __meta as in (defrecord [__meta] ...)

Comment by Alan Malloy [ 09/Feb/12 3:01 PM ]

It's a general issue. (defrecord [__meta]) actually breaks immediately, because the record mechanism itself generates a field named __meta, but any field named with a - or _ prefix has this issue.

user=> (defrecord Foo [-blah])
user.Foo
user=> (.-blah (Foo. 1))
IllegalArgumentException No matching field found: blah for class user.Foo clojure.lang.Reflector.getInstanceField (Reflector.java:289)





[CLJ-919] cannot create anonymous primitive functions Created: 27/Jan/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ben Mabey Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: typehints


 Description   

Primitive functions only work (e.g. return primitive types) when defined with `defn`. An equivalent function created with `fn` does not behave the same way as when created with `defn`. For example:

(definterface IPrimitiveTester
(getType [^int x])
(getType [^long x])
(getType [^float x])
(getType [^double x])
(getType [^Object x]))

(deftype PrimitiveTester []
IPrimitiveTester
(getType [this ^int x] :int)
(getType [this ^long x] :long)
(getType [this ^float x] :float)
(getType [this ^double x] :double)
(getType [this ^Object x] :object))

(defmacro pt [x]
`(.getType (PrimitiveTester.) ~x))

(defn with-defn ^double [^double x]
(+ x 0.5))

(pt (with-defn 1.0)) ; => :double

(let [a (fn ^double [^double x] (+ x 0.5))]
(pt (a 0.1))) ; => :object

Please see the discussion on the mailing list for more details and thoughts on what is happening:
http://groups.google.com/group/clojure/browse_thread/thread/d83c8643a7c7d595?hl=en






[CLJ-911] 'proxy' prevents overriding Object.finalize (and doesn't document it) Created: 16/Jan/12  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Norman Gray Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop
Environment:

OS X, Java 1.6.0?



 Description   

It appears to be impossible to override Object.finalize() using proxy. If the method is defined using proxy, then it cannot be called straightforwardly (see below), and it is not called as a finalizer during normal program execution (not demonstrated below).

See extensive discussion at: https://groups.google.com/group/clojure/browse_thread/thread/a1e2fca45af6c1af

user=> (def m (proxy [java.util.HashMap] []
(finalize []
;(proxy-super finalize)
(prn "finalizing..."))
(hashCode []
99)))
#'user/m
user=> (.hashCode m)
99
user=> (.finalize m)
IllegalArgumentException No matching field found: finalize for class user.proxy$java.util.HashMap$0 clojure.lang.Reflector.getInstanceField (Reflector.java:289)

There is at least one of two bugs here (thanks to Cedric Greevey for summarising this way):

  • If the inability to override finalize() is unintentional, that's a bug.
  • If it's intentional for some reason, then (a) that's not documented, and (b) the failure is silent, in the sense that an explicit call produces an apparently completely unrelated error (above), and the failure to call the method during object finalization is completely silent.





[CLJ-903] extend-protocol does not allow classnames as a String Created: 30/Dec/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2, Release 1.3, Release 1.4
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Meikel Brandmeyer Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: interop


 Description   

In various places Clojure accepts classnames as String, eg. in gen-class or type hints. However it does not in extend-protocol. This does not allow simple specification of array types.

See also here: http://groups.google.com/group/clojure/browse_thread/thread/722a0c09d02bb0ac






[CLJ-899] Accept and ignore colon between key and value in map literals Created: 18/Dec/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: reader


 Description   

Original title was 'treat colons as whitespace' which isn't a problem description but a (flawed) implementation approach

For JSON compatibility
known problems when no spaces - x:true and y:false



 Comments   
Comment by Tassilo Horn [ 23/Dec/11 3:22 AM ]

Discussed here: https://groups.google.com/d/msg/clojure/XvJUzaY1jec/l8xEwlFl8EUJ

Comment by Kevin Downey [ 11/Jan/12 2:23 PM ]

please no

Comment by Tavis Rudd [ 16/Jan/12 12:17 PM ]

Alan Malloy raises a good point in the google group discussion (https://groups.google.com/d/msg/clojure/XvJUzaY1jec/aVpWBicwGhsJ) about accidental confusion between trailing (or floating) and leading colons:
"It isn't even as simple as "letting them
be whitespace", because presumably you want (read-string "{a: b}") to
result in (hash-map 'a 'b), but (read-string "{a :b}") to result in
(hash-map 'a :b)."

This issue could be avoided by only treating a colon as whitespace when followed by a comma. As easy cut-paste of json seems the be the key motivation here, the commas are going to be there anyway: valid {"v":, 1234} vs syntax error {a-key: should-be-a-keyword}.

Comment by Alex Baranosky [ 16/Jan/12 5:23 PM ]

This would be visually confusing imo.

Comment by Laurent Petit [ 17/Jan/12 5:01 PM ]

Please, oh please, no.

Comment by Tavis Rudd [ 18/Jan/12 2:40 PM ]

Er, brain fart. I was typing faster than I was thinking and put the comma in the wrong place. In my head I meant the form following the colon would have to have a comma after it. Thus, {"a-json-key": 1234, ...} would be valid while {"a-json-key": was-supposed-to-be-a-keyword "another-json-key" foo} would complain about the colon being an Invalid Token. I don't see the need for it, however.

Comment by Joseph Smith [ 27/Feb/12 10:55 AM ]

Clojure already has reader syntax for a map. If we support JSON, do we also support ruby map literals? Seems like this addition would only add confusion, imo, given colons are used in keywords and keywords are frequently used in maps - e.g., when de-serializing from XML, or even JSON.

Comment by David Nolen [ 27/Feb/12 11:19 AM ]

Clojure is no longer a language hosted only on the JVM. Clojure is also hosted on the CLR, and JavaScript. In particular ClojureScript can't currently easily deal with JSON literals - an extremely common (though problematic) data format. By allowing colon whitespace in map literals - Clojure data structures can effectively become an extensible JSON superset - giving the succinctness of JSON and the expressiveness of XML.

+1 from me.

Comment by Tim McCormack [ 13/Nov/12 7:27 PM ]

Clojure is only hosted on the JVM; ClojureScript is hosted on JS VMs. If this is useful for CLJS, it should just be a CLJS feature.

Comment by Mike Anderson [ 10/Dec/12 11:51 PM ]

-1 for this whole idea: that way madness lies....

If we keep adding syntactical oddities like this then the language will become unmaintainably complex. It's the exact opposite of simple to have lots of special cases and ambiguities that you have to remember.

If people want to use JSON that is fine, but then the best approach use a specific JSON parser/writer, not just paste it into Clojure source and expect it to work.

Comment by Laszlo Török [ 11/Dec/12 4:54 AM ]

-1 for reasons mentioned by Allan Malloy and Mike Anderson





[CLJ-889] Specifically allow '.' inside keywords Created: 01/Dec/11  Updated: 27/Jul/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: keywords, reader


 Description   

The documentation for keywords (on page http://clojure.org/reader) specifically states that '.' is not allowed as part of a keyword name; however '.' is specifically useful. For example, several web frameworks for Clojure use keywords to represent HTML elements, using CSS selector syntax (i.e., :div.important is equivalent to <div class='important'>).

In any case, the use of '.' is not checked by the reader and it is generally useful.

I would like to see '.' officially allowed (in the documentation). Further, I'd like to see additional details about which punctuation characters are allowed (my own web framework uses '&', '?' and '>' inside keywords for various purposes ... again, current reader implementation does not forbid this, but if a future reader will reject it, I'd like to know now).



 Comments   
Comment by Howard Lewis Ship [ 08/Dec/11 3:37 PM ]

To clarify, Hiccup and Cascade both use keywords containing '#' and '.' Cascade goes further, using '&' (to represent HTML entities), '>', and (possibly in the future) '?'.

Comment by Devin Walters [ 20/Oct/12 6:46 PM ]

I think the EDN spec mitigates some of the concern, but as of yet the official clojure.org reader documentation does not reflect the language used in the description of EDN. Where does EDN stand right now? Can the description being used on the github page be pulled over to clojure.org?

References:

Comment by Howard Lewis Ship [ 15/Apr/13 5:56 AM ]

Unfortunately, the EDN specification does not mention '>'.





[CLJ-880] syntax-quote should be a macro instead of implemented inside the reader Created: 17/Nov/11  Updated: 17/Nov/11

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Kevin Downey Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

syntax-quote is currently a somewhat gnarly bit of java code in LispReader.java, would be great to replace it with a (hopefully less gnarly) clojure macro.

LispReader.java would be required to do something similar to 'x => (quote x). `x => (syntax-quote x) , ~x => (unquote x), ~@x => (unquote-splicing x)






[CLJ-878] DispatchReader always calls CtorReader when no dispatch macro found Created: 15/Nov/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Greg Chapman Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reader
Environment:

Windows 7, Java 1.7



 Description   

At the REPL, I accidentally typed:

user=> # "\w+"
RuntimeException Unsupported escape character: \w clojure.lang.Util.runtimeExce
ption (Util.java:156)
#<core$PLUS clojure.core$PLUS@6b7dc78>

You can see the confusing result (the REPL is also left in an unclosed string). After looking at LispReader.java, it seems to me DispatchReader ought to at least check for whitespace before calling CtorReader (perhaps better would be to check for a valid symbol character). Another example:

user=> (defrecord X [x])
user.X
user=> # "user.X"[5]
#user.X{:x 5}

I'm assuming the fact that the above successfully creates an X is accidental.






[CLJ-864] defrecord positional arity factory fn should have an inline version that calls the record constructor Created: 26/Oct/11  Updated: 04/Dec/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Kevin Downey Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: defrecord


 Description   

defrecord positional arity factory fn should have an inline version that calls the record constructor



 Comments   
Comment by Gary Fredericks [ 26/May/13 3:39 PM ]

I had the idea recently that the factory fn was useful partly for being a redefinable var, e.g. that you could wrap with contracts or anything else. This idea would preclude that.

It makes sense though if the only purpose of ->Foo is to avoid having to :import something.

Comment by Kevin Downey [ 13/Aug/13 4:04 PM ]

interesting, that is a good point

Comment by Gary Fredericks [ 04/Dec/13 7:21 AM ]

Another thought – using the factory fns rather than constructors directly gives you a little bit of protection against code-reloading issues, does it not? I don't think I understand the code reloading issues in great detail, so I'm not confident about this. My assumption is that the compiled code refers to vars rather than classes.





[CLJ-859] Built in dynamic vars don't have :dynamic metadata Created: 19/Oct/11  Updated: 24/Feb/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Anthony Simpson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I'm sure 'built in' is probably not the right term here, but I'm not sure what these are called.

I ran into this issue earlier today while fixing a bug in clojail. Built in vars, particularly ones listed here without a source link: http://clojure.github.com/clojure/clojure.core-api.html, do not have :dynamic metadata despite being dynamic. This includes *in*, *out*, and *err* among others. Here are some examples:

user=> (meta #'*err*)
{:ns #<Namespace clojure.core>, :name *err*, :added "1.0", :doc "A java.io.Writer object representing standard error for print operations.\n\n  Defaults to System/err, wrapped in a PrintWriter"}
user=> (meta #'*in*)
{:ns #<Namespace clojure.core>, :name *in*, :added "1.0", :doc "A java.io.Reader object representing standard input for read operations.\n\n  Defaults to System/in, wrapped in a LineNumberingPushbackReader"}
user=> (meta #'*out*)
{:ns #<Namespace clojure.core>, :name *out*, :added "1.0", :doc "A java.io.Writer object representing standard output for print operations.\n\n  Defaults to System/out, wrapped in an OutputStreamWriter", :tag java.io.Writer}
user=> (meta #'*ns*)
{:ns #<Namespace clojure.core>, :name *ns*, :added "1.0", :doc "A clojure.lang.Namespace object representing the current namespace.", :tag clojure.lang.Namespace}


 Comments   
Comment by Ben Smith-Mannschott [ 19/Oct/11 12:03 PM ]

This recent discussion on the users list seems relevant: Should intern obey :dynamic?.

It seems to boil down to this the information that a Var is dynamic (or not) is duplicated. Once as metadata with the key :dynamic, and once as a boolean field on the Var class which implements Clojure's variables. This boolean can be obtained by calling the method isDynamic() on the Var.

The confusion arises because apparently :dynamic and .isDynamic can get out of sync with each other. .isDynamic is the source of truth in this case.

Comment by Ben Smith-Mannschott [ 19/Oct/11 12:18 PM ]

Compiler$Parser.parse(...) finds the :dynamic entry left in the metadata of the symbol by LispReader and passes this on when creating a new DefExpr, which in turn, generates the code that will call setDynamic(...) on the var when it is created at runtime.

As far as I can tell, the :dynamic entry is irrelevant once that has occurred. It seems to be implemented only as a way to communicate (by way of the reader) with the compiler. Once the compiler's gotten the message, it isn't needed anymore. Keeping it around seems to just cause confusion.

Dynamic vars created by the Java layer of Clojure core don't use the :dynamic mechanism, they just setDynamic() directly. That's why they don't have :dynamic in their meta-data map.

  • Perhaps the compiler should elide :dynamic from the metadata map available at runtime, since it has served its purpose.
  • Perhaps Clojure should supply the function dynamic?.
    (defn dynamic? [^clojure.lang.Var v] (.isDynamic v))

Or, perhaps one might consider, for 1.4, replacing :dynamic altogether and just enforcing the established naming convention: *earmuffs* are dynamic, everything-else isn't. (The compile warns about violations of this convention in 1.3.)

Comment by Andy Fingerhut [ 24/Feb/12 11:39 AM ]

I recently noticed several lines like this one in core.clj. Depending upon how many symbols are like this, perhaps this method could be used to add :dynamic metadata to symbols in core, along with a unit test to verify that all symbols in core have :dynamic if and only if .isDynamic returns true?

Comment by Andy Fingerhut [ 24/Feb/12 12:41 PM ]

Ugh. In my previous comment, by "several lines like this one" I meant to paste the following as an example:

(alter-meta! #'agent assoc :added "1.0")





[CLJ-823] Piping seque into seque can deadlock Created: 03/Aug/11  Updated: 12/Apr/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Greg Chapman Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Windows 7; JVM 1.6; Clojure 1.3 beta 1


Approval: Vetted

 Description   

I'm not sure if this is a supported scenario, but the following deadlocks in Clojure 1.3:

(let [xs (seque (range 150000))
ys (seque (filter odd? xs))]
(apply + ys))

As I understand it, the problem is that ys' fill takes place on an agent thread, so when it calls xs' drain, the (send-off agt fill) does not immediately trigger xs' fill, but is instead put on the nested list to be performed when ys' agent returns. Unfortunately, ys' fill will eventually block trying to take from xs, and so it never returns and the pending send-offs are never sent. Wrapping the send-off in drain to:

(future (send-off agt fill))

is a simple (probably not optimal) way to fix the deadlock.



 Comments   
Comment by Peter Monks [ 07/Jan/13 3:43 PM ]

Reproduced on 1.4.0 and 1.5.0-RC1 as well, albeit with this example:

(seque 3 (seque 3 (range 10)))

Comment by Stuart Halloway [ 30/Mar/13 9:16 AM ]

release-pending-sends?





[CLJ-809] fn's created using defn should not lexical shadow the var that holds them Created: 11/Jun/11  Updated: 11/Apr/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Kevin Downey Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

currently (defn foo [x] (foo x)) expands to something like (def foo
(fn foo [x] (foo x))) so the fn is bound to foo lexically in the scope
of the fn body.

because of this lexical shadowing self calls to fns defined with defn
do not incur the overhead of var dereferencing, I assume this is the
reason it was added.

the lexical shadowing also breaks memoization if you create the
memoized functions via (alter-var-root #'foo memoize) and several
macros here and there emit code in this style, since reusing defn is
the easiest way to get all the good juicy metadata bits like arglists,
etc.

1.3 has changes to eliminate some of the cost of going through the var.

since going through the var is much cheaper now it would be nice to
eliminate the lexical shadowing.

http://groups.google.com/group/clojure-dev/browse_thread/thread/33b52b24616967f?hl=en



 Comments   
Comment by Kevin Downey [ 11/Jun/11 2:40 PM ]

tests indicate that accessing the fn through the var vs. accessing through the lexical binding have very similar access times

https://gist.github.com/1013008

results in

$ java -jar clojure.jar ~/src/lexicaltest.clj
lexical binding run 0
"Elapsed time: 86.739 msecs"
lexical binding run 1
"Elapsed time: 8.062 msecs"
lexical binding run 2
"Elapsed time: 16.182 msecs"
var binding run 0
"Elapsed time: 61.136 msecs"
var binding run 1
"Elapsed time: 40.317 msecs"
var binding run 2
"Elapsed time: 11.641 msecs"
$

Comment by Jason Wolfe [ 13/Jun/11 5:05 PM ]

Here's another test that makes sure no JIT funny business (e.g. dead code elimination) is going on, and also looks at primitive hinted versions.

https://gist.github.com/1023816

var-obj result, time: 14930352 ,"Elapsed time: 965.577 msecs"
lex-obj result, time: 14930352 ,"Elapsed time: 957.741 msecs"
var-prim result, time: 14930352 ,"Elapsed time: 770.411 msecs"
lex-prim result, time: 14930352 ,"Elapsed time: 113.412 msecs"
nil

I'm not sure how to properly hint the var in the var-prim version to get truly comparable results. In any case, this use case should definitely be kept in mind.

Comment by Kevin Downey [ 13/Jun/11 5:35 PM ]

I think the point is we want to make sure the jvm can optimize both just as well, so making the code purposefully unoptimizable is kind of silly, yes?

Comment by Jason Wolfe [ 21/Jun/11 4:41 PM ]

@Kevin In your gist, the return value of foo is not used, and foo has no side effects. A sufficiently smart compiler could optimize the calls out entirely, which would not tell us much about the runtime of foo.

Comment by Aaron Bedra [ 28/Jun/11 6:49 PM ]

Rich: Given the performance testing (in the email thread) what is the next step here?

  • more thorough testing
  • patch?

Are there things that this might break that we need to consider?

Comment by Rich Hickey [ 29/Jul/11 7:39 AM ]

Someone needs to make a patch, and then test perf with and without the patch. These simulated tests aren't necessarily indicative. Naive fib is an ok test once you have a patch. And yes, more things might break - needs careful assessment.

Comment by Nicola Mometto [ 11/Apr/14 5:29 PM ]

If I understand this ticket correctly, it looks like this has been done at some point in time.

Clojure 1.7.0-master-SNAPSHOT
user=> (macroexpand-1 '(defn x []))
(def x (clojure.core/fn ([])))




[CLJ-807] hash-maps print-dup as literal, thus can be read as array-maps Created: 07/Jun/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Alexander Taggart Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: print


 Description   

Per Rich in CLJ-799: "The point of print-dup is type preservation"

user=> (hash-map :k :v)
{:k :v}
user=> (type *1)
clojure.lang.PersistentHashMap
user=> (binding [*print-dup* true] (print-str *2))
"{:k :v}"
user=> (read-string *1)
{:k :v}
user=> (type *1)
clojure.lang.PersistentArrayMap

The cause is due to RT.map conditionally creating an array-map if the size is within the PersistentArrayMap.HASHTABLE_THRESHOLD.



 Comments   
Comment by Aaron Bedra [ 28/Jun/11 6:47 PM ]

Rich: do you want a patch for this?

Comment by Rich Hickey [ 29/Jul/11 7:42 AM ]

Only if it important to someone, solves some problem.





[CLJ-792] Refactor method resolution code out of Compiler and into Reflector Created: 11/May/11  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alexander Taggart Assignee: Alexander Taggart
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File clj-792-reorg-reflector-patch2.txt     Text File reorg-reflector.patch    

 Description   

Issues:

  1. Code for obtaining method/constructor instances is duplicated across the Compiler
  2. Code for resolving a preferred overloaded method lives in the Compiler

By consolidating the duplicated code, moving the reflection-related parts into Reflector, and providing a straightforward API, it should be easier to read and understand the method resolution process. Further, improvements to (e.g., CLJ-445) the mechanism for reflecting on class members can largely be isolated from the Compiler. And the few points of coordination (e.g., Compiler emitting same arg and return types as Reflector does when invoking) can be clearly identified and documented.



 Comments   
Comment by Stuart Sierra [ 17/Feb/12 2:28 PM ]

Patch does not apply as of commit f5bcf64.

Comment by Alexander Taggart [ 17/Feb/12 3:14 PM ]

Yeah, year-old patches tend to do that.

Comment by Andy Fingerhut [ 20/Feb/12 1:11 PM ]

I don't know if this is helpful or not, but this updated version of Alexander's patch applies cleanly to latest Clojure head as of Feb 20, 2012. It compiles, but does not pass ant test.





[CLJ-790] Primitive type hints on function names should print error message Created: 10/May/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alan Dipert Assignee: Alan Dipert
Resolution: Unresolved Votes: 0
Labels: errormsgs


 Description   

Functions returning primitives are hinted with metadata on the argument list, not on the function name. Using a primitive type hint on a function name should print an error message.

Currently, misplaced primitive hints are read without error.






[CLJ-776] seque broken for some BlockingQueues Created: 21/Apr/11  Updated: 22/Apr/11

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Pepijn de Vos Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

clojure.core/seque takes an optional argument that can be the initial capacity for a LinkedBlockingQueue or an instance of the BlockingQueue interface.

PriorityBlockingQueue is such a class, but fails to work because seque makes a few assumptions about BlockingQueues that are not true for all BlockingQueues.

user=> (def s (seque (java.util.concurrent.PriorityBlockingQueue. 100) (shuffle (range 100))))
#'user/s
user=> s
java.lang.NullPointerException

user=> (seque (java.util.concurrent.SynchronousQueue.) (shuffle (range 100)))
<never finishes>

user=> (seque (java.util.concurrent.ArrayBlockingQueue. 10) (shuffle (range 100)))
(39 41 27 76 1 24 92 34 72 37 67 99 38 21 5 64 9 26 59 43 82 65 3 11 31 93 50 63 15 90 13 75 40 97 57 88 86 53 19 83 2 12 54 49 71 28 68 73 96 44 8 98 61 6 22 25 78 66 32 84 60 94 70 7 89 4 33 55 74 81 51 56 62 87 69 80 77 20 30 91 52 10 48 18 36 58 47 14 16 42 35 17 95 0 45 85 29 23 46 79)



 Comments   
Comment by Pepijn de Vos [ 21/Apr/11 11:32 AM ]

A working implementation with some tests.

Comment by Pepijn de Vos [ 22/Apr/11 9:27 AM ]

Attachement removed, I'm working on it here: https://gist.github.com/934781





[CLJ-764] (partition 0 seq) and (parition-all 0 seq) are infinite sequences of empty sequences Created: 21/Mar/11  Updated: 21/Jan/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.1, Release 1.2
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Paul Stadig Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

(partition n seq) and (partition-all n seq) are implemented by taking n and dropping n from seq. When n=0 this becomes an infinite sequence of empty sequences.

While conceptionally this makes sense, I think practically it may surprise people, and perhaps it should return an empty sequence or a sequence of a single empty sequence.



 Comments   
Comment by Paul Stadig [ 21/Mar/11 9:22 AM ]

Ugh! I didn't mean for this to be major priority, and I can't seem to edit it after the fact.

Comment by Mike Anderson [ 21/Jan/13 5:40 AM ]

I think the current behaviour is logically more correct than returning a empty sequence or a single empty sequence.

In particular, I would expect (partition n infinite-seq) to return an infinite sequence of sequences of length n, for any value of n >= 0. Anything else would be very surprising.

The only other sensible option would be throwing an exception, on the philosophical grounds that it isn't possible to partition a non-empty sequence into sub-sequences of length 0. But I think that changing behaviour in this way would break backwards compatibility for no obvious gain.

I suggest closing this issue.





[CLJ-761] print-dup generates call to nonexistent method for APersistentVector$SubVector Created: 19/Mar/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Backlog
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Stuart Sierra Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: print


 Description   

Originally reported by Carson

print-dup on any collection type generates code to call the create method of the collection's class. APersistentVector$SubVector has no create method.

Example with Clojure at commit ecae8ff08a298777c365a261001adfe9bfa4d83c :

Clojure 1.3.0-master-SNAPSHOT
user=> (read-string (binding [*print-dup* true] (pr-str (subvec [1 2 3] 1))))
IllegalArgumentException No matching method found: create  clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:50)


 Comments   
Comment by Kevin Downey [ 04/Nov/11 11:29 AM ]

33.927 hiredman ,(binding [*print-dup* true] (pr-str (first {:a 1})))
33.928 clojurebot "#=(clojure.lang.MapEntry/create [:a 1])"
33.938 hiredman yeah, well, I was busy
33.941 chouser heh
33.949 hiredman ,(clojure.lang.MapEntry/create [:a 1])





[CLJ-760] ClassNotFound when AOT compiling a self-referring deftype extended to a protocol Created: 18/Mar/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ryan Senior Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: aot
Environment:

Clojure 1.2.0, 1.2.1, 1.3.0-alpha6, JDK 1.6.0_24, Ubuntu 10.10


Attachments: Text File stacktraces.txt    

 Description   

If I create a deftype that refers to itself in a protocol extension like below:

(ns type-test)

(defprotocol Foo
  (isa-foo [x]))

(deftype TypeTest []
  Foo
  (isa-foo [x]
           (instance? TypeTest x)))

And use that code via another namespace:

(ns test-type-user
  (:use [type-test :only (isa-foo)])
  (:import [type-test TypeTest]))

(isa-foo (TypeTest.))

When I try to AOT compile the test-type-user namespace with Clojure 1.2.0, I get java.lang.NoClassDefFoundError: compilestub/type-test/TypeTest (test_type_user.clj:5). Full stack trace attached. Running the same code on 1.2.1 and 1.3.0-alpha6 yielded the same exception with a slightly different error message (stacktrace for 1.2.1 is also in the attached file).

This came up in a test at Revelytix. We worked around this issue by not using instance? and instead comparing based on class name. Another workaround is to define the deftype and the extension separately (using extend-type or something similar). This problem also doesn't occur if the usage of the deftype and the definition of it are in the same namespace (i.e. if type-test and test-type-user were in the same file).



 Comments   
Comment by Alex Miller [ 18/Mar/11 10:27 AM ]

The first case where we saw this was actually in having a deftype implement a Java interface (not a protocol) and in that case you cannot extend the interface outside the deftype (although comparing based on class name of course works).





[CLJ-750] clojure.lang.MapEntry violates .equals and .hashCode contracts of HashMap.Entry; leads to non-reflexive .equals, etc. Created: 02/Mar/11  Updated: 29/Jul/11

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2, Release 1.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jason Wolfe Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

user> (let [h (HashMap.)]
(.put h 1 2)
(let [p1 (first h)
p2 (first {1 2})]
(println (.hashCode p1) (.hashCode p2)
(= p1 p2) (= p2 p1)
(.equals p1 p2) (.equals p2 p1))))

3 994 false false true false

clojure.lang.MapEntry does not follow the contracts of hashCode or equals for java.util.Map$Entry. This, among other things, leads to the above non-reflexive behavior of .equals.

I realize this isn't a simple bug; it's a fundamental issue, in that it's impossible to consistently implement both java.util.List and java.util.Map$Entry, which clojure.lang.MapEntry claims to do. On balance I do find it very useful that Clojure map entries are equivalent to vectors, and probably wouldn't want to give that up. But, I think that at minimum this wart should be documented somewhere, since it could lead to some very strange and hard to catch bugs.



 Comments   
Comment by Jason Wolfe [ 02/Mar/11 10:13 PM ]

This might be fixable by changing seq to promise a sequence of pairs rather than MapEntries. I don't know enough about Clojure internals to know if this would entail too large a performance hit, but perhaps it's worth considering.

Comment by Stuart Halloway [ 04/Mar/11 9:11 PM ]

Rich: is there a doc patch you would want for this?

Comment by Rich Hickey [ 29/Jul/11 7:45 AM ]

doc patch ok





[CLJ-740] Unnecessary boxing of primitives in case form Created: 17/Feb/11  Updated: 01/Mar/11

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mikhail Kryshen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Found this while profiling some performance-critical code.

Consider the following Clojure function:

(defn test-case ^double [^long i ^double d1 ^double d2]
  (case (int i)
    0 d1
    d2))

Current Clojure 1.3 snapshot compiles it to:

public final double invokePrim(long, double, double)   throws java.lang.Exception;
  Code:
   0:	lload_1
   1:	invokestatic	#67; //Method clojure/lang/RT.intCast:(J)I
   4:	istore	7
   6:	iload	7
   8:	i2l
   9:	invokestatic	#73; //Method clojure/lang/Numbers.num:(J)Ljava/lang/Number;
   12:	invokestatic	#79; //Method clojure/lang/Util.hash:(Ljava/lang/Object;)I
   15:	iconst_0
   16:	ishr
   17:	iconst_1
   18:	iand
   19:	tableswitch{ //0 to 0
		0: 36;
		default: 58 }
   36:	iload	7
   38:	i2l
   39:	invokestatic	#73; //Method clojure/lang/Numbers.num:(J)Ljava/lang/Number;
   42:	getstatic	#45; //Field const__3:Ljava/lang/Object;
   45:	invokestatic	#83; //Method clojure/lang/Util.equals:(Ljava/lang/Object;Ljava/lang/Object;)Z
   48:	ifeq	58
   51:	dload_3
   52:	invokestatic	#88; //Method java/lang/Double.valueOf:(D)Ljava/lang/Double;
   55:	goto	63
   58:	dload	5
   60:	invokestatic	#88; //Method java/lang/Double.valueOf:(D)Ljava/lang/Double;
   63:	checkcast	#92; //class java/lang/Number
   66:	invokevirtual	#96; //Method java/lang/Number.doubleValue:()D
   69:	dreturn
}

This bytecode contains boxing of primitives (calls to clojure/lang/Numbers.num and java/lang/Double.valueOf) and calls to clojure/lang/Util.hash and clojure/lang/Util.equals that does not seem necessary.

At 60-66 primitive double is boxed into Double only to be converted back into primitive.

The equivalent Java code compiles to much simpler and faster bytecode:

public double testCase(long, double, double);
  Code:
   0:	lload_1
   1:	l2i
   2:	lookupswitch{ //1
		0: 20;
		default: 22 }
   20:	dload_3
   21:	dreturn
   22:	dload	5
   24:	dreturn
}


 Comments   
Comment by Alexander Taggart [ 28/Feb/11 2:16 PM ]

Improved via patch on CLJ-426.

(defn test-case ^double [^long i ^double d1 ^double d2]
  (case (int i)
    0 d1
    d2))

now emits as

 0  lload_1 [i]
 1  invokestatic clojure.lang.RT.intCast(long) : int [67]
 4  istore 7 [G__7903] // let-bound expression
 6  iload 7 [G__7903]
 8  tableswitch default: 32
      case 0: 28
28  dload_3 [d2]
29  goto 34
32  dload 5 [arg2]
34  dreturn

or if the int cast of the expression is omitted:

 0  lload_1 [i]
 1  lstore 7 [G__7903] // let-bound expression
 3  lload 7 [G__7903]
 5  l2i
 6  tableswitch default: 35
      case 0: 24
24  lconst_0           // match, verify long expr wasn't truncated
25  lload 7 [G__7903]
27  lcmp
28  ifne 35
31  dload_3 [d2]
32  goto 37
35  dload 5 [arg2]
37  dreturn




[CLJ-731] Create macro to variadically unroll a combinator function definition Created: 26/Jan/11  Updated: 26/Jul/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Backlog

Type: Enhancement Priority: Minor
Reporter: Fogus Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Approval: Vetted

 Description   

Clojure contains a set of combinators that are implemented in a similar, but slightly different way. That is, they are implemented as a complete set of variadic overloads on both the call-side and also on the functions that they return. Visually, they all tend to look something like:

(defn foo
  ([f]
     (fn
       ([] do stuff...)
       ([x] do stuff...)
       ([x y] do stuff...)
       ([x y z] do stuff...)
       ([x y z & args] do variadic stuff...)))
  ([f1 f2]
     (fn
       ([] do stuff...)
       ([x] do stuff...)
       ([x y] do stuff...)
       ([x y z] do stuff...)
       ([x y z & args] do variadic stuff...)))
  ([f1 f2 f3]
     (fn
       ([] do stuff...)
       ([x] do stuff...)
       ([x y] do stuff...)
       ([x y z] do stuff...)
       ([x y z & args] do variadic stuff...)))
  ([f1 f2 f3 & fs]
     (fn
       ([] do stuff...)
       ([x] do stuff...)
       ([x y] do stuff...)
       ([x y z] do stuff...)
       ([x y z & args] do variadic stuff...))))

To build this type of function for each combinator is tedious and error-prone.

There must be a way to implement a macro that takes a "specification" of a combinator including:

1. name
2. docstring
3. do stuff template
4. do variadic stuff template

And builds something like the function foo above. This macro should be able to implement the current batch of combinators (assuming that http://dev.clojure.org/jira/browse/CLJ-730 is completed first for the sake of verification).



 Comments   
Comment by Stuart Halloway [ 28/Jan/11 9:03 AM ]

This seems useful. Rich, would you accept a patch?

Comment by Stuart Halloway [ 28/Jan/11 9:40 AM ]

Nevermind, just saw that Rich already suggested this on the dev list. Patch away.





[CLJ-705] Make sorted maps and sets implement j.u.SortedMap and SortedSet interfaces Created: 05/Jan/11  Updated: 02/Jun/12

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Rich Hickey Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Comments   
Comment by Andy Fingerhut [ 02/Jun/12 2:29 PM ]

This might be a duplicate of CLJ-248. See that one before working on this one, at least.





[CLJ-704] range function has missing documentation Created: 04/Jan/11  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Maarten Hus Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring
Environment:

All



 Description   

The range function's documentation does indicate the following usage:

(range 10 0 -1) -> (10, 9, 8, 7, 6, 5, 4, 3, 2, 1)

Current doc:

Returns a lazy seq of nums from start (inclusive) to end
(exclusive), by step, where start defaults to 0, step to 1, and end
to infinity.

Suggestion:

Returns a lazy seq of nums from start (inclusive) to end
(exclusive), by step, where start defaults to 0, step to 1, and end
to infinity.

Its also possible to step down rather than up, for example counting
backwards from 10 by -1: (range 10 0 -1).



 Comments   
Comment by Rasmus Svensson [ 15/Jan/11 7:39 AM ]

The current doc actually mentions the 'step' parameter briefly:

"[...] to end (exclusive), by step, where start [...]"

But as this might be easy to miss, an addition to the doc is still a good idea, I think.

My suggestion:

Returns a lazy seq of nums from start (inclusive) to end
(exclusive), by step, where start defaults to 0, step to 1, and end
to infinity. Step may be negative to count backwards.





[CLJ-701] Compiler loses 'loop's return type in some cases Created: 03/Jan/11  Updated: 21/Oct/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Backlog
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Chouser Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Clojure commit 9052ca1854b7b6202dba21fe2a45183a4534c501, version 1.3.0-master-SNAPSHOT


Approval: Vetted

 Description   
(set! *warn-on-reflection* true)
(fn [] (loop [b 0] (recur (loop [a 1] a))))

Generates the following warnings:

recur arg for primitive local: b is not matching primitive, had: Object, needed: long
Auto-boxing loop arg: b

This is interesting for several reasons. For one, if the arg to recur is a let form, there is no warning:

(fn [] (loop [b 0] (recur (let [a 1] a))))

Also, the compiler appears to understand the return type of loop forms just fine:

(use '[clojure.contrib.repl-utils :only [expression-info]])
(expression-info '(loop [a 1] a))
;=> {:class long, :primitive? true}

The problem can of course be worked around using an explicit cast on the loop form:

(fn [] (loop [b 0] (recur (long (loop [a 1] a)))))

Reported by leafw in IRC: http://clojure-log.n01se.net/date/2011-01-03.html#10:31



 Comments   
Comment by a_strange_guy [ 03/Jan/11 4:36 PM ]

The problem is that a 'loop form gets converted into an anonymous fn that gets called immediately, when the loop is in a expression context (eg. its return value is needed, but not as the return value of a method/fn).

so

(fn [] (loop [b 0] (recur (loop [a 1] a))))

gets converted into

(fn [] (loop [b 0] (recur ((fn [] (loop [a 1] a))))))

see the code in the compiler:
http://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L5572

this conversion already bites you if you have mutable fields in a deftype and want to 'set! them in a loop

http://dev.clojure.org/jira/browse/CLJ-274

Comment by Christophe Grand [ 23/Nov/12 2:28 AM ]

loops in expression context are lifted into fns because else Hotspot doesn't optimize them.
This causes several problems:

  • type inference doesn't propagate outside of the loop[1]
  • the return value is never a primitive
  • mutable fields are inaccessible
  • surprise allocation of one closure objects each time the loop is entered.

Adressing all those problems isn't easy.
One can compute the type of the loop and emit a type hint but it works only with reference types. To make it works with primitive, primitie fns aren't enough since they return only long/double: you have to add explicit casts.
So solving the first two points can be done in a rather lccal way.
The two other points require more impacting changes, the goal would be to emit a method rather than a fn. So it means at the very least changing ObjExpr and adding a new subclassof ObjMethod.

[1] beware of CLJ-1111 when testing.

Comment by Alex Miller [ 21/Oct/13 10:28 PM ]

I don't think this is going to make it into 1.6, so removing the 1.6 tag.





[CLJ-699] make sure compile paths shares as much code as possible Created: 31/Dec/10  Updated: 27/Jul/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Issue http://dev.clojure.org/jira/browse/CLJ-697 highlighted the fact that compilation has more than one entry point. Review these entry points and make them share code.






[CLJ-698] class accessible from deftype method bodies is not suitable for instance?, ... Created: 28/Dec/10  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Herwig Hochleitner Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: deftype


 Description   

Example interaction: http://pastebin.com/cTdUCKfp
Which directly contradicts documentation for deftype

In the method bodies, the (unqualified) name can be used to name the class (for calls to new, instance? etc).



 Comments   
Comment by Stuart Halloway [ 29/Dec/10 12:45 PM ]

The problem occurs in 1.2 but is fixed on master. Leaving in backlog in case we ever cut another 1.2 release--if not, then mark as fixed.





[CLJ-668] Improve slurp performance by using native Java StringWriter and jio/copy Created: 01/Nov/10  Updated: 03/Sep/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Jürgen Hötzel Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: io, performance


 Description   

Instead of copying each character from InputReader to StringBuffer.

Performance improvement:

From:
user> (time (count (slurp "/home/juergen/test.dat")))
"Elapsed time: 4269.980863 msecs"
104857600
To:
user> (time (count (slurp "/home/juergen/test.dat")))
"Elapsed time: 1140.854023 msecs"
104857600

Patch: http://github.com/juergenhoetzel/clojure/commit/af1a5713cd485ba5f1db25c37906ebaf7d474b47






[CLJ-451] fn literals lack name/arglists/namespace metadata Created: 05/Oct/10  Updated: 26/Aug/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Anonymous Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

I would expect (meta (fn not-so-anonymous [a b c])) to include {:name not-so-anonymous :arglists ([a b c])} alongside line number information and possibly namespace/file as well, but currently it only includes :line.



 Comments   
Comment by Assembla Importer [ 05/Oct/10 12:29 AM ]

Converted from http://www.assembla.com/spaces/clojure/tickets/451





[CLJ-450] Add default predicate argument to filter, every?, take-while Created: 01/Oct/10  Updated: 26/Aug/13

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Anonymous Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File clj-450-add-default-pred-arg-to-core-fns-patch.txt     Text File clojure-default-every-argument-v1.patch    

 Description   

Some seq processing functions that take predicates could be improved by the addition of a default value of identity for the predicate argument.

This has been discussed on the mailing list, and people seem favorable:
http://groups.google.com/group/clojure/browse_thread/thread/600559b7ee261908/3bc5d144ac54854e?lnk=gst&q=filter+identity#3bc5d144ac54854e
http://groups.google.com/group/clojure-dev/browse_thread/thread/0a9b5750dd7ec4ca

I can put together a patch.



 Comments   
Comment by Assembla Importer [ 01/Oct/10 4:39 PM ]

Converted from http://www.assembla.com/spaces/clojure/tickets/450

Comment by Jason Orendorff [ 13/Mar/12 2:51 PM ]

I independently wanted this. Here's a patch for: some, not-any?, every?, not-every?. If this is roughly what's wanted I'll be happy to add filter, remove, take-while, drop-while.

Comment by Jason Orendorff [ 13/Mar/12 4:57 PM ]

Note that there are a few cases of (every? identity ...) and (some identity ...) in core.clj itself; the patch removes "identity" from those.

Comment by Andy Fingerhut [ 26/Apr/12 7:51 PM ]

clj-450-add-default-pred-arg-to-core-fns-patch.txt dated Apr 26 2012 is identical to Jason Orendorff's, except it is in git format. Jason is not on the list of Clojure contributors as of today. I have sent him an email asking if he has done so, or is planning to.

Comment by Jason Orendorff [ 27/Apr/12 10:35 AM ]

Of course I'd be happy to send in a contributor agreement. ...Is there actually any interest in taking this patch or something like it?

Comment by Andy Fingerhut [ 27/Apr/12 11:38 AM ]

I don't know if there is any interest in taking this patch. Perhaps a Clojure screener will take a look at it and comment, but I am not a screener and can't promise anything.





[CLJ-445] Method/Constructor resolution does not factor in widening conversion of primitive args Created: 29/Sep/10  Updated: 27/Jul/13

Status: In Progress
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Backlog

Type: Enhancement Priority: Major
Reporter: Alexander Taggart Assignee: Alexander Taggart
Resolution: Unresolved Votes: 3
Labels: None

Attachments: Text File clj-445-prim-conversion-update-2-patch.txt     Text File prim-conversion.patch     Text File prim-conversion-update-1.patch     Text File reorg-reflector.patch    
Approval: Vetted

 Description   

Problem:
When making java calls (or inlined functions), if both args and param are primitive, no widening conversion is used to locate the proper overloaded method/constructor.

Examples:

user=> (Integer. (byte 0))
java.lang.IllegalArgumentException: No matching ctor found for class java.lang.Integer (NO_SOURCE_FILE:0)
</code></pre>
The above occurs because there is no Integer(byte) constructor, though it should match on Integer(int).
<pre><code>user=> (bit-shift-left (byte 1) 1)
Reflection warning, NO_SOURCE_PATH:3 - call to shiftLeft can't be resolved.
2

In the above, a call is made via reflection to Numbers.shiftLeft(Object, Object) and its associated auto-boxing, instead of directly to the perfectly adequate Numbers.shiftLeft(long, int).

Workarounds:
Explicitly casting to the formal type.

Ancillary benefits of fixing:
It would also reduce the amount of method overloading, e.g., RT.intCast(char), intCast(byte), intCast(short), could all be removed, since such calls would pass to RT.intCast(int).



 Comments   
Comment by Assembla Importer [ 23/Oct/10 6:43 PM ]

Converted from http://www.assembla.com/spaces/clojure/tickets/445
Attachments:
fixbug445.diff - https://www.assembla.com/spaces/clojure/documents/b6gDSUZOur36b9eJe5cbCb/download/b6gDSUZOur36b9eJe5cbCb

Comment by Assembla Importer [ 23/Oct/10 6:43 PM ]

ataggart said: [file:b6gDSUZOur36b9eJe5cbCb]

Comment by Assembla Importer [ 23/Oct/10 6:43 PM ]

ataggart said: Also fixes #446.

Comment by Stuart Halloway [ 03/Dec/10 12:50 PM ]

The patch is causing a test failure

[java] Exception in thread "main" java.lang.IllegalArgumentException: 
     More than one matching method found: equiv, compiling:(clojure/pprint/cl_format.clj:428)

Can you take a look?

Comment by Stuart Halloway [ 28/Jan/11 12:30 PM ]

The failing test happens when trying to find the correct equiv for signature (Number, long). Is the compiler wrong to propose this signature, or is the resolution method wrong in not having an answer? (It thinks two signatures are tied: (Object, long) and (Number, Number).)

Exception in thread "main" java.lang.IllegalArgumentException: More than one matching method found: equiv, compiling:(clojure/pprint/cl_format.clj:428)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6062)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6050)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.access$100(Compiler.java:35)
	at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5492)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$IfExpr$Parser.parse(Compiler.java:2372)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3277)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6057)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5231)
	at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5527)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5231)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$IfExpr$Parser.parse(Compiler.java:2385)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5231)
	at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5527)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$IfExpr$Parser.parse(Compiler.java:2385)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5231)
	at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5527)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5231)
	at clojure.lang.Compiler$FnMethod.parse(Compiler.java:4667)
	at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3397)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6053)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6043)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.access$100(Compiler.java:35)
	at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:480)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	at clojure.lang.Compiler.analyze(Compiler.java:5866)
	at clojure.lang.Compiler.analyze(Compiler.java:5827)
	at clojure.lang.Compiler.eval(Compiler.java:6114)
	at clojure.lang.Compiler.load(Compiler.java:6545)
	at clojure.lang.RT.loadResourceScript(RT.java:340)
	at clojure.lang.RT.loadResourceScript(RT.java:331)
	at clojure.lang.RT.load(RT.java:409)
	at clojure.lang.RT.load(RT.java:381)
	at clojure.core$load$fn__1427.invoke(core.clj:5308)
	at clojure.core$load.doInvoke(core.clj:5307)
	at clojure.lang.RestFn.invoke(RestFn.java:409)
	at clojure.pprint$eval3969.invoke(pprint.clj:46)
	at clojure.lang.Compiler.eval(Compiler.java:6110)
	at clojure.lang.Compiler.load(Compiler.java:6545)
	at clojure.lang.RT.loadResourceScript(RT.java:340)
	at clojure.lang.RT.loadResourceScript(RT.java:331)
	at clojure.lang.RT.load(RT.java:409)
	at clojure.lang.RT.load(RT.java:381)
	at clojure.core$load$fn__1427.invoke(core.clj:5308)
	at clojure.core$load.doInvoke(core.clj:5307)
	at clojure.lang.RestFn.invoke(RestFn.java:409)
	at clojure.core$load_one.invoke(core.clj:5132)
	at clojure.core$load_lib.doInvoke(core.clj:5169)
	at clojure.lang.RestFn.applyTo(RestFn.java:143)
	at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:77)
	at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
	at clojure.core$apply.invoke(core.clj:602)
	at clojure.core$load_libs.doInvoke(core.clj:5203)
	at clojure.lang.RestFn.applyTo(RestFn.java:138)
	at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:77)
	at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
	at clojure.core$apply.invoke(core.clj:604)
	at clojure.core$use.doInvoke(core.clj:5283)
	at clojure.lang.RestFn.invoke(RestFn.java:409)
	at clojure.main$repl.doInvoke(main.clj:196)
	at clojure.lang.RestFn.invoke(RestFn.java:422)
	at clojure.main$repl_opt.invoke(main.clj:267)
	at clojure.main$main.doInvoke(main.clj:362)
	at clojure.lang.RestFn.invoke(RestFn.java:409)
	at clojure.lang.Var.invoke(Var.java:401)
	at clojure.lang.AFn.applyToHelper(AFn.java:163)
	at clojure.lang.Var.applyTo(Var.java:518)
	at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalArgumentException: More than one matching method found: equiv
	at clojure.lang.Reflector.getMatchingParams(Reflector.java:639)
	at clojure.lang.Reflector.getMatchingParams(Reflector.java:578)
	at clojure.lang.Reflector.getMatchingMethod(Reflector.java:569)
	at clojure.lang.Compiler$StaticMethodExpr.<init>(Compiler.java:1439)
	at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:896)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6055)
	... 115 more
Comment by Alexander Taggart [ 08/Feb/11 6:27 PM ]

In working on implementing support for vararg methods, I found a number of flaws with the previous solutions. Please disregard them.

I've attached a single patch (reflector-compiler-numbers.diff) which is a rather substantial overhaul of the Reflector code, with some enhancements to the Compiler and Numbers code.

The patch notes:

  • Moved reflection functionality from Compiler to Reflector.
  • Reflector supports finding overloaded methods by widening conversion, boxing conversion, and casting.
  • During compilation Reflector attempts to find best wildcard match.
  • Reflector refers to *unchecked-math* when reflectively invoking methods and constructors.
  • Both Reflector and Compiler support variable arity java methods and constructor; backwards compatible with passing an array or nil in the vararg slot.
  • Added more informative error messages to Reflector.
  • Added tests to clojure.test-clojure.reflector.
  • Altered overloaded functions of clojure.lang.Numbers to service Object/double/long params; fixes some ambiguity issues and avoids unnecessary boxing in some cases.
  • Patch closes issues 380, 440, 445, 666, and possibly 259 (not enough detail provided).
Comment by Alexander Taggart [ 10/Feb/11 7:35 PM ]

Updated patch to fix a bug where a concrete class with multiple identical Methods (e.g., one from an interface, another from an abstract class) would result in ambiguity. Now resolved by arbitrary selection (this is what the original code did as well albeit not explicitly).

Comment by Alexander Taggart [ 25/Feb/11 9:29 PM ]

Updated patch to work with latest master branch.

Comment by Stuart Halloway [ 06/Mar/11 1:54 PM ]

patch appears to be missing test file clojure/test_clojure/reflector.clj.

Comment by Alexander Taggart <