<< Back to previous view

[CLJ-2118] extend-type doesn't identify that a protocol is a protocol Created: 23/Feb/17  Updated: 23/Feb/17  Resolved: 23/Feb/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Maurício Eduardo Chicupo Szabo Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: protocols
Environment:

Ubuntu 16.04 x86_64, OpenJDK 8



 Description   

I'm trying to implement this tutorial: http://www.lucagrulla.com/posts/server-sent-events-with-ring-and-compojure/

In one part, I need to extend a core.async type with the following protocol: https://github.com/ring-clojure/ring/blob/master/ring-core/src/ring/core/protocols.clj#L19.

So, I've tried to implement something like this:

src/async_test/protocol_ext.clj
(ns async-test.protocol-ext
  (:require [clojure.core.async :as async]
            [clojure.java.io :as io])
  (:import [ring.core.protocols StreamableResponseBody]
           [clojure.core.async.impl.channels ManyToManyChannel]))

(extend-type ManyToManyChannel
  StreamableResponseBody
  (write-body-to-stream [channel response output-stream]
                        (async/go (with-open [writer (io/writer output-stream)]
                                    (async/loop []
                                                (when-let [msg (async/<! channel)]
                                                  (doto writer (.write msg) (.flush))
                                                  (recur)))))))

But this fails with interface ring.core.protocols.StreamableResponseBody is not a protocol.

Then, I tried to monkey-patch ring: opened a new file with the correct path, and added the following lines on the bottom of the protocol declaration, inside `extend-protocol` call:

src/ring/core/protocols.clj
ManyToManyChannel
  (write-body-to-stream [channel response output-stream]
                        (async/go (with-open [writer (io/writer output-stream)]
                                    (async/loop []
                                                (when-let [msg (async/<! channel)]
                                                  (doto writer (.write msg) (.flush))
                                                  (recur)))))))

Then, it worked. What's happening? Is Clojure ignoring that StreamableResponseBody is a protocol if there's already an `extend-protocol` call?



 Comments   
Comment by Maurício Eduardo Chicupo Szabo [ 23/Feb/17 3:00 PM ]

Okay, my bad. Seems that I need to `:require` the protocol, not `:import` it. Sorry about that...

Comment by Alex Miller [ 23/Feb/17 3:04 PM ]

The issue here is that you are importing StreamableResponseBody as a Java class (really an interface). While the protocol does generate a Java interface, you should use the protocol, not the interface, when you extend. This is confusing because they both have the same name.

So, your ns declaration should instead be:

(ns async-test.protocol-ext
  (:require [clojure.core.async :as async]
            [clojure.java.io :as io]
						[ring.core.protocols :refer [StreamableResponseBody]])
  (:import [clojure.core.async.impl.channels ManyToManyChannel]))

The monkeypatch you tried worked because you were properly referring to the protocol in that case.





[CLJ-2117] Support typical polymorphic maps in spec more directly Created: 23/Feb/17  Updated: 23/Feb/17  Resolved: 23/Feb/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec


 Description   

Problem: A common scenario is that one wants to spec this:

(def a-poly-map
  {::type :t1
   <kvs required/optional in t1>
   <kvs required/optional in all>
   })

To do this, you usually write lots of boiler plate code like this:

(s/def ::type #{:t1 <more-types>})

(defmulti type-dispatch ::type)
(defmethod type-dispatch :t1 [_] (s/keys :req [<t1-keys>]))
<more defmethods for more-types>

(s/def ::poly-map (s/merge (s/keys :req [::type <poly-map-keys-in-all>])
                           (s/multi-spec type-dispatch ::type))

As a workaround one can (and we did) of course write a macro to write all that code.

From my experience this usecase is the 90% multi-spec usecase and thus I'd prefer if a more convenient to use spec existed.

Proposed solution: Implement this spec:

(s/def ::poly-map (s/merge (s/keys :req [<poly-map-keys-in-all>]
                           (s/poly-map ::type
                             :t1 {:req [<t1-keys>]} ; :req, :opt etc. as in s/keys
                             <more requirements for more types>)))

Limitations: This syntax does not support to reuse existing specs in for dispatch values (:t1, :t2, ...) intentionally, because it seems an non-existing usecase to spec incomplete s/keys for such maps separately anyway.

Further enhancement: While the <poly-map-keys-in-all> usecase is rare, this syntax could also directly support it:

(s/def ::poly-map (s/poly-map ::type
                    {:req [<poly-map-keys-in-all>]} ; optional
                    :t1 {:req [<t1-keys>]}
                    <more requirements for more types>))

This new spec would automatically check for the presence of ::type and conform/explain like a regular s/keys.



 Comments   
Comment by Alex Miller [ 23/Feb/17 11:50 AM ]

Thanks, but I don't think we expect to do anything like this.

Comment by Leon Grapenthin [ 23/Feb/17 12:03 PM ]

Why?

Comment by Alex Miller [ 23/Feb/17 12:51 PM ]

Because spec provides the tools to do it already.

Comment by Leon Grapenthin [ 23/Feb/17 1:55 PM ]

This ticket is asking for a syntactic simplification of a typical use case. I understand that there already is s/multi-spec. s/multi-spec however is a generic and in comparison quite advanced tool that is not tied to maps. Thus it requires a substantial amount of boilerplate to "do it".

After much practical experience with spec in my observation - and I do assume others would agree - this is the most common application for it.

There are many things in clojure.core which aren't the only "tools to do it" for the exact same reason, so please reconsider not closing this right away.

Comment by Alex Miller [ 23/Feb/17 2:32 PM ]

My job is to weigh the balance of several factors and decide if an enhancement request seems reasonably likely to remain as an open ticket and at some point be further evaluated.

Some factors I try to think about:

  • Is this a good problem? Does it describe a pain point that many people are likely to encounter? You claim it is "most common" use but seems way too early to judge this (from what I've seen, I wouldn't agree right now).
  • Is the pain significant enough that it prevents you from doing your work? In this case, clearly not - you've already built it and it would be easy to release it independently from core.
  • Is this a fundamental (simple) feature that belongs in core? Clojure has a small library and wishes to keep it that way. There are ample examples where this constraint was not considered enough in Clojure. From what I can tell, this is appears to be a composite of existing things, not a fundamental part.
  • If we were to solve this problem in core, is this the proposed solution a likely way we would do it? Based on things we've talked about during dev, my suspicion is, probably not.

On balance, it does not seem to be something worth leaving open in the tracker right now. But as the saying goes, "no is temporary, yes is forever". If a while down the road it's clear that this is a prominent problem that needs attention, there is nothing preventing us from considering it again (presumably with more data and experience at that time).

Comment by Leon Grapenthin [ 23/Feb/17 3:11 PM ]

Thanks for explaining the decision. I understand that you have priorities and probably a better plan to address this problem in the future.





[CLJ-2110] sorted map returns nil value for existing key Created: 14/Feb/17  Updated: 14/Feb/17  Resolved: 14/Feb/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: jonnik Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: sorted-map
Environment:

Oracle Java 1.8.0_112



 Description   

Then i try to get value by key from sorted map with comparator by value it returns nil.

(def tmap {1 {:v 1} 2 {:v 2} 3 {:v 3}})
(def tmap-sorted (apply sorted-map-by #(let [val-comp (- (compare (get-in tmap [%1 :v]) (get-in tmap [%2 :v])))]
                                         (if (= val-comp 0) 1 val-comp))
                        (flatten (vec tmap))))
; => {3 {:v 3} 2 {:v 2} 1 {:v 1}}
(get tmap-sorted 3)
;=> nil

Expected: {:v 3}
Actual: nil



 Comments   
Comment by Andy Fingerhut [ 14/Feb/17 11:31 AM ]

Your comparison function never returns 0, by the way you have written it. If it never returns 0, it will never 'think' that it has found an equal element when searching for a match using 'get'. By replacing (if (= val-comp 0) 1 val-comp) with val-comp, the get calls will work:

(def tmap-sorted (apply sorted-map-by #(let [val-comp (- (compare (get-in tmap [%1 :v]) (get-in tmap [%2 :v])))]
                                         val-comp)
                        (flatten (vec tmap))))
tmap-sorted
; => {3 {:v 3}, 2 {:v 2}, 1 {:v 1}}
(get tmap-sorted 3)
; => {:v 3}

You are getting a descending sorted order by negating the return value of compare. I would recommend to follow the advice on this page: https://clojure.org/guides/comparators particularly "Reverse the sort by reversing the order that you give the arguments to an existing comparator." That helps avoid corner cases with some integer values.

I would also recommend (into my-empty-sorted-map tmap) in place of your (apply my-empty-sorted-map (flatten (vec tmap)). Putting all of those recommendations together would result in code like this:

(def tmap-sorted2 (into (sorted-map-by #(compare (get-in tmap [%2 :v]) (get-in tmap [%1 :v])))
                        tmap))
tmap-sorted2
; => {3 {:v 3}, 2 {:v 2}, 1 {:v 1}}
(get tmap-sorted2 3)
; => {:v 3}
Comment by saintech [ 14/Feb/17 11:41 AM ]

Ok. But how about this?:

tmap-sorted
; => {3 {:v 3} 2 {:v 2} 1 {:v 1}}
(first tmap-sorted)
; => [3 {:v 3}]
(get tmap-sorted 3)
;=> nil

Is it OK?

Comment by Andy Fingerhut [ 14/Feb/17 12:43 PM ]

I believe that calling clojure.core/first on a sorted-map does not cause its comparison function to be called at all. It is already stored in a sorted tree in memory, and first just finds the earliest one.

clojure.core/get does call the comparison function, perhaps several times, to find an item with an equal key. The original comparison function given in the description never returns equality (i.e. the integer 0) when comparing two items.

Comment by Alex Miller [ 14/Feb/17 1:32 PM ]

Agreed with Andy, declining





[CLJ-2107] s/explain of a predicate defined s/with-gen yields s/unknown Created: 07/Feb/17  Updated: 08/Feb/17  Resolved: 08/Feb/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tamas Herman Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: generator, spec


 Description   

This explanation is clear:

(s/def ::some-string string?)
(s/explain ::some-string 123)

val: 123 fails spec: :boot.user/some-string predicate: string?

However if the failing spec has a custom generator, the explanation is not so helpful:

(s/def ::some-string-with-custom-generator (s/with-gen string? gen/string-alphanumeric))
(s/explain ::some-string-with-custom-generator nil)

=> :boot.user/some-string-with-custom-generator
val: nil fails spec: :boot.user/some-string-with-custom-generator predicate: :clojure.spec/unknown

I would expect the same predicate: string? explanation.

Based on the symptom I suspect this issue is related to http://dev.clojure.org/jira/browse/CLJ-2068



 Comments   
Comment by Tamas Herman [ 08/Feb/17 2:21 AM ]

Explanation of a workaround from Slack from @hiredman:

with-gen is a function, so it's arguments are evaluated, so the string? argument to with-gen is a function object, when spec is trying to find the name to report it does some stuff, which for symbols and keywords reports a good name, but for other Objects (including function objects) you get :clojure.spec/unknown.

wrapping with s/spec allows the spec macro to capture the meaningful, the symbol before evaluation:

(s/def ::some-string-with-custom-generator (s/with-gen (s/spec string?) gen/string-alphanumeric))
(s/explain ::some-string-with-custom-generator nil)

val: nil fails spec: :boot.user/some-string-with-custom-generator predicate: string?

I don't see any simple solution to make the original example yield a good explanation,
so maybe we should just explain how it works in https://clojure.org/guides/spec#_custom_generators
and wrap the with-gen examples with spec.

Comment by Alex Miller [ 08/Feb/17 8:49 AM ]

As you mentioned, this is a dup of CLJ-2068 and is fixed by the patch there.





[CLJ-2106] satisfies? is quite slow when returning false Created: 06/Feb/17  Updated: 09/Feb/17  Resolved: 09/Feb/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Mike Kaplinskiy Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: performance


 Description   
(defprotocol SatisfiesTest (testThing [this]))
(defrecord MyRecord [] Object SatisfiesTest (testThing [this]))
(def r (MyRecord.))

(time (dotimes [_ 30000] (instance? user.SatisfiesTest r)))
"Elapsed time: 1.715 msecs"

(time (dotimes [_ 30000] (satisfies? SatisfiesTest r)))
"Elapsed time: 3.944 msecs"

(time (dotimes [_ 30000] (instance? user.SatisfiesTest {})))
"Elapsed time: 2.304 msecs"

(time (dotimes [_ 30000] (satisfies? SatisfiesTest {})))
"Elapsed time: 718.949 msecs"

It would be nice if satisfies? memoized negative return values by class (though that cache would need to be cleared by `extend-type` and friends).



 Comments   
Comment by Alex Miller [ 06/Feb/17 1:56 PM ]

This is covered in an existing ticket - http://dev.clojure.org/jira/browse/CLJ-1814

Comment by Alex Miller [ 06/Feb/17 2:02 PM ]

Actually, read too quickly - CLJ-1814 covers the positive case only. I don't think we're going to cache a negative return value though. Managing and cleaning that cache is likely to not be worth it. If you need this kind of thing, you should probably consider a different kind of conditional check before-hand.

Comment by Alex Miller [ 06/Feb/17 2:05 PM ]

For example, you can just memoize this call like this:

user=> (def check (memoize #(satisfies? SatisfiesTest %)))
#'user/check
user=> (time (dotimes [_ 30000] (check {})))
"Elapsed time: 3.512 msecs"
Comment by Alex Miller [ 06/Feb/17 2:13 PM ]

I tweaked the test to remove the use of range and the construction of the record so the numbers are more useful.

Comment by Nicola Mometto [ 08/Feb/17 6:52 PM ]

Alex Miller CLJ-1814 handles the negative cases too, and doesn't keep a cache of negative return values. It keeps a cache of all the protocols extended by a certain class and invalidates that cache on every call to extend, the benchmarks on the ticket description showcase both a positive and a negative test

Comment by Alex Miller [ 09/Feb/17 3:09 PM ]

Nicola - cool! I didn't realize that. Will mark this as a dupe then.





[CLJ-2101] Request for a way to forget specs Created: 24/Jan/17  Updated: 25/Jan/17  Resolved: 24/Jan/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Johan Gall Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: spec


 Description   

Hello,

I have a server that loads code, and specs, at runtime in generated namespaces.

Thus I have a spec leak.

It would be nice if there was an official API to forget specs.



 Comments   
Comment by Johan Gall [ 24/Jan/17 7:01 AM ]

(or better, a way to create a temporary spec world, but then that is probably not trivial)

Comment by Alex Miller [ 24/Jan/17 7:51 AM ]

Is this covered by http://dev.clojure.org/jira/browse/CLJ-2060 ?

Comment by Alex Miller [ 24/Jan/17 12:51 PM ]

Marking as a dup for now, but will re-open if this goes beyond what's in CLJ-2060.

Comment by Johan Gall [ 25/Jan/17 5:20 AM ]

ah sorry, indeed it's a dup





[CLJ-2099] Keywords with aliased namespaces cannot be read when the namespace is required in a reader conditional Created: 13/Jan/17  Updated: 25/Jan/17  Resolved: 13/Jan/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Viktor Magyari Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: keywords, reader, readerconditionals


 Description   

The title in itself isn't entirely true, but I couldn't find a way to describe it succintly (feel free to change).

The issue is easier to demonstrate with an example:

(ns foo
  #?(:cljs (:require [clojure.core :as c])))

#?(:cljs ::c/x)

When reading this in a :clj context, the reader cannot read ::c/x ("Invalid token: ::c/x"), despite the code being correct (presumably).
The same thing happens if the reader conditional branches are :clj and the source is read in a :cljs context.



 Comments   
Comment by Alex Miller [ 13/Jan/17 8:07 AM ]

This looks like expected behavior to me. Auto-resolved keywords rely on a resolution context and there isn't one at the point where ::c/x is being read.

Comment by Viktor Magyari [ 13/Jan/17 9:05 AM ]

To me it seems reasonable to expect the resolution context to include the clojure.core alias - more generally, include <platform> specific aliases in the <platform> branches of reader conditionals. Maybe consider this as an enhancement ticket?

Comment by Alex Miller [ 13/Jan/17 9:18 AM ]

To do this would require adding special handling specifically for ns or other things that create aliases, which implies conditional evaluation of some forms at read-time. You would also need some place (other than the current platform's namespace maps) to track other platform's namespace aliases. That's a lot of very special, custom stuff.

We're not going to add this.

Comment by Leon Grapenthin [ 25/Jan/17 2:13 PM ]

1. Why does the reader try to read the :cljs branch in Viktors example?
2. The original intent of reader conditionals was that the code could be read independently of the lang. Have aliases not been considered then due to a lack of popularity of ::a/n syntax?

My suggestion would be a knob that reads unresolvable alias kws as a special object #aliased-keyword{:alias "a", :name "n"}. This would solve both Viktors problem and also pay its debt to the reader conditional intent.

Comment by Alex Miller [ 25/Jan/17 5:16 PM ]

1. The reader reads everything, it's just selective about which parts of an expression gets returned. Without reading, it wouldn't know where the end of that form was.

2. I think this is a little more subtle in that reading the second thing in a conditional requires remembering something read and discarded in a prior conditional.

That said, maybe it would be possible to either read in a mode where this particular case doesn't throw or where this particular exception was known and discarded when inside a conditional.





[CLJ-2096] "Key must be integer" thrown when vector lookup done with short or byte Created: 04/Jan/17  Updated: 06/Jan/17  Resolved: 05/Jan/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8, Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Aaron Cummings Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

clojure-1.9.0-alpha14 and clojure-1.8.0 tested.



 Description   

Looking up a value in a vector with a long or integer index works:
([:a :b :c] (long 1)) => :b
([:a :b :c] (int 1)) => :b

However, lookups with short or byte fail with "IllegalArgumentException Key must be integer"
([:a :b :c] (short 1))
([:a :b :c] (byte 1))

Root cause seems to be clojure.lang.Util.isInteger() which returns true only if the argument is an instance of Integer, Long, clojure.lang.BigInt, or BigInteger. I think it would be more correct for clojure.lang.Util.isInteger() to be consistent with clojure.core/integer? which additionally returns true for both Short and Byte.



 Comments   
Comment by Alex Miller [ 05/Jan/17 8:24 AM ]

I don't see any good reason to make changes in this area so declining.

Comment by Aaron Cummings [ 05/Jan/17 8:44 AM ]

Hi Alex - the case where we ran into this exception was in parsing a binary file where the record descriptor is a byte, and we used this byte to do a lookup in a vector (the vector holding keywords which describe the record type). The workaround is pretty simple though; just cast the byte to an int.

Curiously, a map lookup like ({0 :a, 1 :b, 2 :c} (byte 1)) does work.

I wondering though if the non-support of short and byte lookups in vectors is intentional, and if so, the reason for this choice (I don't see any obvious problems, so perhaps Rich knows something I don't here). If instead this is an oversight, and is deemed not broken enough to fix, then I can accept that.

Comment by Alex Miller [ 06/Jan/17 8:26 AM ]

I would say that given that the check and error message exists, it is intentional. Certainly there is a performance impact to checking for a broader set of types.





[CLJ-2094] clojure.spec: bug with protocols and extend-type Created: 01/Jan/17  Updated: 03/Jan/17  Resolved: 03/Jan/17

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: John Schmidt Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec


 Description   

I have the following two clj files (I've tried to come up with a minimal example):

core.clj
--------------
(ns spec-test.core
  (:require [clojure.spec :as s]))

(defprotocol Game
  (move [game]))

(s/def ::game1 #(satisfies? Game %))
(s/def ::game2 (partial satisfies? Game))

foo.clj
--------------
(ns spec-test.foo
  (:require [spec-test.core :refer [Game]]))

(defrecord Foo [])

(extend-type Foo
  Game
  (move [game]))

Here's a REPL session that explains my problem:

user=> (ns spec-test.core)
nil
spec-test.core=> (require 'spec-test.core :reload)
nil
spec-test.core=> (require 'spec-test.foo :reload)
nil
spec-test.core=> (satisfies? Game (spec-test.foo/->Foo))
true
spec-test.core=> ((partial satisfies? Game) (spec-test.foo/->Foo))
true
spec-test.core=> (s/explain ::game1 (spec-test.foo/->Foo))
Success!
nil
spec-test.core=> (s/explain ::game2 (spec-test.foo/->Foo))
val: #spec_test.foo.Foo{} fails spec: :spec-test.core/game2 predicate: (partial satisfies? Game) <---- WAAAAAT
nil

I would expect ::game1 and ::game2 to be equivalent, but somehow they're not.

Also see: https://groups.google.com/forum/#!topic/clojure/igBlMpqGU3A



 Comments   
Comment by Steve Miner [ 02/Jan/17 12:43 PM ]

I gave some incorrect advice on the mailing list so I'll try to correct myself here. Basically, the protocol is stored in a var, Game. If a predicate captures a particular state of the protocol, it won't respond correctly when additions are made to the protocol (with extend-type, etc.) So the problem with the usage of `partial` is that it evaluates the current value of the protocol at that point in time, before it has been extended to cover Foo.

The #(...) function definition would have used the var itself, not its current value. Naturally, the var allows the protocol to be updated such that the function sees the updated value. Basically, this is just normal Clojure evaluation. A `defn` style function would have worked fine as well. It's just the `partial` that evaluated its args that leads to the problem.

The same kind of issue could come up if you passed any var to partial and captured the current value of the var. Later changes to the var would not affect the result of partial.

I'll say the bug is the confusing error message. It seems that s/explain is implying it is using the var Game, but it really captured an "old" value of Game.

Comment by Alex Miller [ 03/Jan/17 2:10 PM ]

I think Steve's assessment is all correct.

I'm not sure that it's possible for spec to know what's happened here though wrt giving a better error message.

I don't think I really see anything that can be "fixed", so I'm going to mark this as declined.





[CLJ-2093] partial and fn behave differently with eval Created: 28/Dec/16  Updated: 31/Dec/16  Resolved: 31/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: N/A Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

(eval (list map (partial + 1) [0]))
CompilerException java.lang.ExceptionInInitializerError
(eval (list map (fn [x] (+ 1 x)) [0]))
=> (1)



 Comments   
Comment by Kevin Downey [ 29/Dec/16 9:22 PM ]

same issue as http://dev.clojure.org/jira/browse/CLJ-1206 and all the issues connected to that

Comment by Alex Miller [ 31/Dec/16 11:17 AM ]

You should not expect either of these to work as the expressions contain function objects (not function forms).

You should be doing something like this:

(eval (list 'map '(partial + 1) [0]))




[CLJ-2088] 'into' can make maps from collection of lists, but vectors are ok. Created: 19/Dec/16  Updated: 20/Dec/16  Resolved: 19/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Eugene Aksenov Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: bug
Environment:

OS: Ubuntu 16.04, x86_64, Linux 4.2.0-42-generic



 Description   

'into' can make maps from collection of lists, but vectors are ok.

Good behavior:
(into {} [[:a "a"] [:b "b"]])
;;=> {:a "a", :b "b"}

(into {} '([1 "a"] [2 "b"]))
;;=> {1 "a", 2 "b"}

Bad examples:
(into {} ['(:a "a") '(:b "b")])
;;=> ClassCastException clojure.lang.Keyword cannot be cast to java.util.Map$Entry clojure.lang.ATransientMap.conj (ATransientMap.java:44)

(into {} [(list [:a "a"]) [:b "b"]])
;;=> ClassCastException clojure.lang.PersistentVector cannot be cast to java.util.Map$Entry clojure.lang.ATransientMap.conj (ATransientMap.java:44)

(into {} ['(1 "a") '(2 "b")])
;;=> ClassCastException java.lang.Long cannot be cast to java.util.Map$Entry clojure.lang.ATransientMap.conj (ATransientMap.java:44)

(into {} '('(1 "a") '(2 "b")))
;;=> ClassCastException clojure.lang.Symbol cannot be cast to java.util.Map$Entry clojure.lang.ATransientMap.conj (ATransientMap.java:44)



 Comments   
Comment by Alex Miller [ 19/Dec/16 11:53 AM ]

into is built on conj (takes a sequential collection of elements to invoke with conj). conj for maps takes a map entry - a subclass of java.util.Map$MapEntry or a 2-element vector. Both of these choices can access keys and vals directly without "traversing" the entry (going through the key to get to the value).

We don't want to support that for performance reasons, so lists or seqs or not valid as map entries (and conj on a map does not support them).

into takes a sequential collection of these entries though, so vector, or list, or seq are all fine as the src collection for into.

So, all of this is working as intended and I am declining the ticket.

Comment by Eugene Aksenov [ 20/Dec/16 2:09 AM ]

Ok, live and learn
I've just added this code case and brief explanation to clojuredocs.org. Hope no one will be confused anymore.





[CLJ-2087] map does not work with core.async/<! Created: 17/Dec/16  Updated: 17/Dec/16  Resolved: 17/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Roy Varghese Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

The construct
(map <! [chan1 chan2 chan3])

does not work for reasons outlined here https://github.com/clojure/core.async/wiki/Go-Block-Best-Practices

This leads to very obscure bugs.

If this cannot be fixed in map, please consider throwing an exception when it cannot handle the passed function.



 Comments   
Comment by Alex Miller [ 17/Dec/16 5:08 PM ]

It's unlikely that we're going to "fix" this as it part of the design of Clojure.

When I try your example I see:

AssertionError Assert failed: <! used not in (go ...) block




[CLJ-2086] A macro in a nested syntax-quote causes an exception Created: 15/Dec/16  Updated: 03/Jan/17  Resolved: 15/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: N/A Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

(defmacro foo
[x y]
``(~~x ~~y))

(foo and true)

CompilerException java.lang.RuntimeException: Can't take value of a macro: #'clojure.core/and



 Comments   
Comment by Alex Miller [ 15/Dec/16 10:15 PM ]
user=> (macroexpand '(foo and true))
(clojure.core/seq (clojure.core/concat (clojure.core/list and) (clojure.core/list true)))

What do you expect this to do?

Comment by Alex Miller [ 15/Dec/16 10:36 PM ]

Declining till more info about a) what problem you are trying to solve and b) what the expected behavior is.

Comment by N/A [ 22/Dec/16 8:30 PM ]

a) I'm trying to define a macro that defines a macro.
(defmacro make-defmacro
[macro-name macro]
`(defmacro ~macro-name
x# & more#
(do (~macro x#)
`(~~macro-name ~@more#))))

(make-defmacro bar and)
CompilerException java.lang.RuntimeException: Can't take value of a macro

b) (defmacro foo
[x y]
``(~~x ~~y))

(macroexpand '(foo and true))
=> (and true)

Comment by Viktor Magyari [ 03/Jan/17 10:52 AM ]

Try

(defmacro foo [x y]
  ``(~'~x ~~y))




[CLJ-2084] Support MapEquivalence for defrecords Created: 13/Dec/16  Updated: 13/Dec/16  Resolved: 13/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Herwig Hochleitner Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: defrecord


 Description   

defrecord equality implies matching types. This has been discussed and decided upon such that people should only bother with defrecords when they want an equality partition. There are, however, use cases for defrecords as small structmaps to save memory and those require map-based equivalence. Clojure has a marker interface for map equivalence: clojure.lang.MapEquivalence, which gets used in = to decide whether something qualifies as a persistent map. This can be reused to change the equiv implementation of a defrecord.

Current Behavior

When MapEquivalence is implemented in a defrecord, = becomes non-reflexive

(= {} (->Rec)), (not= (->Rec) {})

Proposed Behavior

Add clojure.lang.MapEquivalence to implemented interfaces in a defrecord, to treat it as a PersistenMap in equality, disregarding the type tag.
APersistentMap.equiv already does this, defrecord implementation can call APersistentMap/mapEquals



 Comments   
Comment by Alex Miller [ 13/Dec/16 10:27 AM ]

Wouldn't this be a breaking change for anyone relying on records to compare not equals based on type?

Comment by Herwig Hochleitner [ 13/Dec/16 4:35 PM ]

The key phrase is "Add clojure.lang.MapEquivalence to implemented interfaces". Any existing defrecords would not be affected. Au contraire, having a defrecord implement MapEquivalence right now results in a non-reflective-equals clojure-gigo scenario, so it's completely uncharted territory and ripe for definition. Sorry for not being clearer in the description, would you consider reopening?

Comment by Alex Miller [ 13/Dec/16 5:15 PM ]

It's not uncharted territory. Records intentionally compare different for equality based on type.

If you don't want that, don't use records. Or pour your record into a map before comparing via into or select-keys.

Comment by Herwig Hochleitner [ 13/Dec/16 5:51 PM ]

Yes, and I'm not proposing to change a single bit of that, agreed?
I am proposing the possibility to have specific defrecords implement map-like equality by using an option that a) naturally fits with what's already there b) compiles but yields GIGO right now:

(defrecord Rec []
  clojure.lang.MapEquivalence)

How is this already charted? If it is, then this ticket needs to be refiled as a Defect, because then the current behavior makes no sense.

So, if you're not declining this on technical grounds, what is it then? If it is not wanting to add use cases to defrecords for philosophical reasons, you're using the term "breaking change" very loosely here. There can't be any existing expectations on defrecords implementing MapEquivalence right now, because such defrecords are invalid programs right now.
Are you arguing some sort of "general expectation" on defrecords? One that would stand without looking at specific definitions? What use would that be to anybody. Who works with a defrecord without having a general idea of what it definition is?

Comment by Herwig Hochleitner [ 13/Dec/16 6:19 PM ]

Just to be clear, I did my due diligence on this and I'm aware that the current equality situation for records is completely intentional, hence I'm proposing growing it. The argument might still be that we don't want to grow it in that particular direction, but it's definitely not breakage.

Comment by Alex Miller [ 13/Dec/16 6:34 PM ]

I thought you were asking to add MapEquivalence to all records, which would be a breaking change.

The docstring for defrecord says:

{pre}
In addition, defrecord will define type-and-value-based =,
and will defined Java .hashCode and .equals consistent with the
contract for java.util.Map.{pre}

The important thing there is "type-and-value-based", and that's part of the design of records. Asking for MapEquivalence on a map means value-based only. These two things are in conflict and thus I would agree that asking for MapEquivalence on a record makes no sense.

deftype exists so that you can make your own types with whatever equality/hash/behavior you want - if you want something like this, then build it on top of deftype.

Comment by Herwig Hochleitner [ 13/Dec/16 6:41 PM ]

yeah, changing defrecord to implement MapEquivalence by default would make no sense.

However, declaring a (specific) defrecord to be in the MapEquivalence partition makes perfect sense, doesn't it?
That way, we can type our records an structmap them too.

Comment by Herwig Hochleitner [ 13/Dec/16 6:49 PM ]

Anyway, I'll build this on deftype for data.xml in any case, because we want this to work on current and older clojure versions. I just thought it would fit neatly with clojure itself (since there is already a marker interface for that purpose, only its use currently leads to breakage) and wanted to make you aware of the idea. I'll recycle it through the mailing list, as soon as the deftype based thing is in data.xml





[CLJ-2078] Add a maven profile to connect with an nrepl client Created: 07/Dec/16  Updated: 07/Dec/16  Resolved: 07/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Herwig Hochleitner Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: development

Attachments: Text File clj-2078-0.patch    
Patch: Code

 Description   

Opening an nrepl port into a project is a popular development strategy with clojure projects. It should also work with the clojure project.

Maven profiles make it possible to add development tools without affecting the rest of the build.

clojure-maven-plugin has this tool built right in as the clojure:nrepl goal and cider provides middlewares for many advanced ide functionalities.

Attached patch provides and documents a -P cider profile that lets you jack into a clojure source checkout.



 Comments   
Comment by Herwig Hochleitner [ 07/Dec/16 7:38 PM ]

Assuming that non-contrib covered code is OK for optional dev tools.

Comment by Alex Miller [ 07/Dec/16 8:44 PM ]

Thanks, I don't think we're going to do this.





[CLJ-2071] Unexpected behavior with clojure.spec/tuple and clojure.spec.test/instrument Created: 01/Dec/16  Updated: 01/Dec/16  Resolved: 01/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ben Rady Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

Clojure 1.9, JDK 8



 Description   

It looks like stest/instrument is comparing a sequence of actual args to a vector created by spec/tuple and it doesn't match, however clojure.spec.test/instrument appears to work fine. Reading the clojure.spec guide I would think the two approaches would be equivalent.

(ns sample
  (:require [clojure.spec.test :as stest]
            [clojure.spec :as spec]))

    (defn myinc [i]
      (inc i))

    (spec/fdef myinc
               :args (spec/tuple integer?) ; Fails with the error below
               ; :args (spec/cat :i integer?) ; This works
               :ret integer?)

    (stest/instrument `myinc)
    (myinc 1)
    ; clojure.lang.ExceptionInfo: Call to #'specific.core-spec/myinc did not conform to spec:
    ; val: (1) fails at: [:args] predicate: vector?
    ; :clojure.spec/args  (1)
    ; :clojure.spec/failure  :instrument
    ; :clojure.spec.test/caller  {:file "core_spec.clj", :line 28, :var-scope specific.core-spec/fn--6046}


 Comments   
Comment by Ben Rady [ 01/Dec/16 8:11 AM ]

Note that the namespace in this example was changed. It used to be specific.core-spec, which is shown in the error.

Comment by Alex Miller [ 01/Dec/16 10:22 AM ]

Spec will create a list or a seq of the args for checking the :args spec. Tuples can only be used on vectors (because they match by index). So, this is not currently expected to work. It is recommended that you use a regex spec (s/cat) instead.





[CLJ-2058] s/keys doesn't check non-keyword elements in :req and :req-un vectors Created: 15/Nov/16  Updated: 15/Nov/16  Resolved: 15/Nov/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Eugene Pakhomov Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: spec


 Description   

As can be seen here https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L430 the s/keys macro filters out any non-keyword element from :req and :req-un before checking it, but later it still uses them in the arguments to map-spec-impl.
Compare the behavior when passing non-keyword elements to :opt and :opt-un.



 Comments   
Comment by Alex Miller [ 15/Nov/16 9:29 AM ]

:req and :req-un support `and` and `or` connectives, :opt and :opt-un do not. So this all seems right to me.

I don't see any bug here?

Comment by Eugene Pakhomov [ 15/Nov/16 9:58 AM ]

I see your point. But how are `and` and `or` related to "all keys must be namespace-qualified keywords"?
And why to do the check at all for :opt and :opt-un that are not even used, when the check for :req and :req-un just allows any forms, not just documented `and` and `or`?

Comment by Alex Miller [ 15/Nov/16 10:08 AM ]

and and or are connectors, not keys.

I'd be fine if the ticket showed something invalid - no actual bug is being shown here. If you improve the ticket, I will re-open.





[CLJ-2052] .class vs .clj isn't picked correctly when either .class or .clj are not in a jar Created: 03/Nov/16  Updated: 04/Nov/16  Resolved: 04/Nov/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mike Kaplinskiy Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None


 Description   

The code that figures out the last modification date (https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L389) uses URLConnection.getLastModified. For `file://` URLs this always returns 0.

One of the resulting bugs: when both .class and .clj files are present on a directory-based classpath, .clj files are always preferred, regardless of modification time.



 Comments   
Comment by Alex Miller [ 03/Nov/16 6:15 PM ]

Can you provide OS and JVM info? This might be env-specific.

Comment by Mike Kaplinskiy [ 03/Nov/16 6:20 PM ]

Sure - I'm on macOS Sierra on Oracle Java 8:

$ java -version
java version "1.8.0_74"
Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.74-b02, mixed mode)
$ uname -a
Darwin mikekap-mbp.local 16.1.0 Darwin Kernel Version 16.1.0: Thu Oct 13 21:26:57 PDT 2016; root:xnu-3789.21.3~60/RELEASE_X86_64 x86_64 i386 MacBookPro11,5 Darwin
Comment by Alex Miller [ 03/Nov/16 6:28 PM ]

It's prob not too fun but some example code would be awfully handy.

Comment by Mike Kaplinskiy [ 03/Nov/16 7:13 PM ]

Sorry this looks like it was my fault - the path I was looking at when testing didn't exist (seems .getLastModified always returns 0 on those instead of throwing an exception).

Sorry for the noise.

Comment by Alex Miller [ 04/Nov/16 9:26 AM ]

Closed per comments.





[CLJ-2045] bean function not working Created: 17/Oct/16  Updated: 17/Oct/16  Resolved: 17/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Roy Varghese Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Environment:

(defproject hello-jcloud "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [#_[org.clojure/clojure "1.8.0"]
[org.clojure/clojure "1.9.0-alpha13"]
[org.clojure/tools.logging "0.1.0"]
[org.apache.jclouds/jclouds-all "2.0.0-SNAPSHOT"]]
:repositories {"jclouds-snapshots" "https://repository.apache.org/content/repositories/snapshots"})



 Description   

user> (bean (java.util.Date.))
UnsupportedOperationException empty clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.empty (:-1)
user>

It works when clojure/1.8.0 is uncommented.



 Comments   
Comment by Alex Miller [ 17/Oct/16 7:38 AM ]

This is a dupe of http://dev.clojure.org/jira/browse/CLJ-2027 which will probably be in the next alpha.





[CLJ-2042] s/form of s/? does not resolve pred Created: 14/Oct/16  Updated: 28/Oct/16  Resolved: 28/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Attachments: Text File clj-2042.patch    
Patch: Code
Approval: Ok

 Description   
user=> (require '[clojure.spec :as s])
nil
user=> (s/form (s/? int?))
(clojure.spec/? int?)

Patch: clj-2042.patch






[CLJ-2034] comment macro doesn't ignore namespace keyword Created: 04/Oct/16  Updated: 04/Oct/16  Resolved: 04/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nuttanart Pornprasitsakul Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

In fresh started REPL:

user=> (comment (x/y))
nil
user=> (comment ::x/y)
RuntimeException Invalid token: ::x/y  clojure.lang.Util.runtimeException (Util.java:221)
RuntimeException Unmatched delimiter: )  clojure.lang.Util.runtimeException (Util.java:221)


 Comments   
Comment by Alex Miller [ 04/Oct/16 11:37 PM ]

This code is not valid because x is an invalid alias (not bound to anything).

This works for example:

user=> (alias 'x 'clojure.string)
nil
user=> (comment ::x/y)
nil

comment still reads the code, so code that is unable to be read is still invalid.

Comment by Alex Miller [ 04/Oct/16 11:38 PM ]

Note that CLJ-2030 would make this code work (by auto-creating x as a new namespace).





[CLJ-2033] s/conformer conversion loss in (very common) s/merge edge case Created: 04/Oct/16  Updated: 04/Oct/16  Resolved: 04/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alexander Kahl Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec
Environment:

1.9.0-alpha14



 Description   

When using s/conform on a spec generated by s/merge that uses non-namespaced keys from s/keys, keys that use a custom s/conformer lose their conversion.

Steps to reproduce:

Unable to find source-code formatter for language: clojure. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
(require '[clojure.spec :as s])
(defn convert [n] (if (double? n) n (double n)))
(s/def ::value (s/conformer convert))
(s/conform (s/keys :req [::value]) {::value 5}) => #:user{:value 5.0} ; correct
(s/conform (s/keys :req-un [::value]) {:value 5}) => {:value 5.0} ; correct
(s/conform (s/merge (s/keys :req [::value]) (s/keys)) {::value 5}) => #:user{:value 5.0} ; correct
(s/conform (s/merge (s/keys :req-un [::value]) (s/keys)) {:value 5}) => {:value 5} ; WRONG

While this seems like an edge case, it is very likely to occur since using s/merge with unqualified keys is a typical use case for configuration files. I first spotted this behavior in alpha13 and it still occurs in alpha14.



 Comments   
Comment by Alex Miller [ 04/Oct/16 8:36 AM ]

alpha14 isn't out yet, so thanks for traveling back in time to report this!

I think this is not a bug, just a subtlety in how merge conform works. Specifically, merge does not flow conformed results, you only get the conformed result of the last spec in the merge.

In this case:

(s/conform (s/merge (s/keys :req [::value]) (s/keys)) {::value 5})

the spec determining the conform value is (s/keys), which will conform all namespaced keys, including ::value.

In this case:

(s/conform (s/merge (s/keys :req-un [::value]) (s/keys)) {:value 5})

the spec determining the conform value is also (s/keys), but s/keys only conforms namespaced keys and there aren't any here, so you get the original map.

If you want the conformed value, you can swap the order in the merge because the first spec is conforming the unqualified key:

(s/conform (s/merge (s/keys) (s/keys :req-un [::value])) {:value 5})
;;=> {:value 5.0}
Comment by Alexander Kahl [ 04/Oct/16 8:50 AM ]

Oh geez, I meant 13 (and first observed in 12), wish I could actually travel back in time!

If what you're saying is right, why doesn't this work, as both (s/keys) use only unqualified keys?

(s/def ::other string?)
(s/conform (s/merge (s/keys :req-un [::value]) (s/keys :req-un [::other])) {:value 5 :other "5"}) => {:value 5, :other "5"}
Comment by Alexander Kahl [ 04/Oct/16 9:08 AM ]

I just read again your comment Alex Miller and finally started to understand how (s/keys) conforms all namespaced keys, so please disregard my other comment.
I still wish there was a solution that worked for multiple maps of unqualified keys. Otherwise, I'd have to expect my users to use qualified keys throughout configuration files which looks a lot like redundancy, unless I convert all the keys first.





[CLJ-2032] Confusing error conforming fspec with missing arg spec Created: 03/Oct/16  Updated: 28/Oct/16  Resolved: 28/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: errormsgs, spec
Environment:

1.9.0-alpha13


Attachments: Text File clj-2032.patch    
Patch: Code
Approval: Ok

 Description   
(require '[clojure.spec :as s])
(def my-spec (s/fspec :ret string?))
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil  clojure.core/-cache-protocol-fn (core_deftype.clj:568)
	clojure.core/-cache-protocol-fn (core_deftype.clj:583)
	clojure.spec/fn--13560/G--13555--13569 (spec.clj:121)
	clojure.spec/specize (spec.clj:138)
	clojure.spec/gensub (spec.clj:262)
	clojure.spec/gen (spec.clj:275)
	clojure.spec/gen (spec.clj:275)
	clojure.spec/validate-fn (spec.clj:1664)
	clojure.spec/fspec-impl/reify--14270 (spec.clj:1686)
	clojure.spec/conform (spec.clj:150)
	clojure.spec/conform (spec.clj:146)

Proposed: When conforming, throw if no args spec specified for the fspec:

Can't conform fspec without args spec: (fspec :args nil :ret string? :fn nil)

Alternatives

  • absence of args spec always conforms ::invalid
  • absence of args spec always conforms any args
  • disallow fspecs without args (still potentially useful for other uses like documentation, so not sure we want to do that)

Patch: clj-2032.patch






[CLJ-2029] (nth nil <anything>) does not throw an exception Created: 28/Sep/16  Updated: 28/Sep/16  Resolved: 28/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Hans Hübner Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

If `nth` is being passed `nil` as `coll` argument, no exception is thrown for arbitrary indices. This does not match the expected behavior (all indices should throw an "Index out of bounds" exception) and also not documented.



 Comments   
Comment by Alex Miller [ 28/Sep/16 9:21 AM ]

It is by design that nth works on nil to return nil for any index and changing this would likely break many existing programs. An enhancement for the docstring would be considered.





[CLJ-2027] bean printing regression from namespace map printing Created: 24/Sep/16  Updated: 28/Oct/16  Resolved: 28/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Trey Sullivan Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: print, regression
Environment:

Clojure 1.9.0-alpha12


Attachments: Text File clj-2027.patch    
Patch: Code and Test
Approval: Ok

 Description   

The new namespace map printing is causing a failure in printing bean maps (which are proxies that don't support empty):

user=> (bean (java.util.Date.))
UnsupportedOperationException empty  clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.empty (:-1)

user=> (pst *e)
UnsupportedOperationException empty
	clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.empty (:-1)
	clojure.core/empty (core.clj:5151)
	clojure.core/lift-ns (core_print.clj:237)

Cause: The internal lift-ns function calls empty on the map too early (here it doesn't need to call it at all).

Proposed: Defer calling (empty m) until we know map has namespace keywords and namespace maps will be used for printing.

Patch: clj-2027.patch (note that into is not used in the change b/c it has not yet been defined at this point)






[CLJ-2024] Check should specize function specs before checking Created: 19/Sep/16  Updated: 28/Oct/16  Resolved: 28/Oct/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: James Reeves Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Attachments: Text File clj-2024-2.patch     Text File clj-2024.patch    
Patch: Code
Approval: Ok

 Description   

This code works fine in 1.9.0-alpha12:

(defn f [x] (+ x 1))
(s/def f (s/fspec :args (s/cat :x number?) :ret number?))
(stest/check `f)

But if we factor the fspec out into its own keyword:

(defn f [x] (+ x 1))
(s/def ::inc (s/fspec :args (s/cat :x number?) :ret number?))
(s/def f ::inc)
(stest/check `f)

The check fails with the exception:

({:failure #error {
 :cause "No :args spec"
 :data #:clojure.spec{:failure :no-args-spec}
 :via
 [{:type clojure.lang.ExceptionInfo
   :message "No :args spec"
   :data #:clojure.spec{:failure :no-args-spec}
   :at [...]}]
 :trace
 [...]}, :sym user/f, :spec :user/inc})

The check function doesn't seem to be resolving ::inc, when presumably it should.

Patch: clj-2024-2.patch



 Comments   
Comment by Rich Hickey [ 28/Oct/16 7:44 AM ]

this should be fixed in fspec, not its use by test

Comment by Alex Miller [ 28/Oct/16 8:23 AM ]

fspec is not the problem as far as I can tell - it is already making specs of its args.

The problem is that f is registered as an alias of ::inc. I don't think you want to resolve that at registration time (as ::inc might not exist yet).

The problem as far as I understand it is that at the time of use (by check), f is not resolved to it's final spec and that's what the patch does.

Comment by Alex Miller [ 28/Oct/16 8:43 AM ]

Added new patch that uses `spec` instead of private `specize` function.





[CLJ-2023] clojure.spec edge case failure Created: 14/Sep/16  Updated: 14/Sep/16  Resolved: 14/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Brian Noguchi Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

Using clojure 1.9.0-alpha12



 Description   

I stumbled onto an odd edge case. The following is a minimal example you can run in the REPL:

```
(require '[clojure.spec :as s])
(s/def :tv/star (s/or :starring/ernie #{:char/ernie}
:starring/big-bird #{:char/big-bird}))

; The following behaves as expected.
(s/explain
(s/and (s/keys :req [:tv/star]
:opt [:tv/co-star])
(s/or
:ernie-and-bert (s/and #(do (println "!!!!!") (println %) (= (:tv/star %) :char/ernie))
#(= (:tv/co-star %) :char/bert))
:just-the-bird #(not (contains? % :tv/co-star))))
{:tv/star :char/big-bird})
; This outputs the following:
; !!!!!
; #:tv{:star [:starring/big-bird :char/big-bird]}
; Success!
; => nil

; The following is unexpected
(s/explain
(s/and (s/keys)
(s/or
:ernie-and-bert (s/and #(do (println "!!!!!") (println %) (= (:tv/star %) :char/ernie))
#(= (:tv/co-star %) :char/bert))
:just-the-bird #(not (contains? % :tv/co-star))))
{:tv/star :char/ernie :tv/co-star :char/bert})
; This outputs the following:
; !!!!!
; #:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert}
; !!!!!
; #:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert}
; !!!!!
; #:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert}
; !!!!!
; #:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert}
; val: #:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert} fails at: [:ernie-and-bert] predicate: (do (println "!!!!!") (println %) (= (:tv/star %) :char/ernie))
; val: #:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert} fails at: [:just-the-bird] predicate: (not (contains? % :tv/co-star))
; => nil
```

I would have expected the second `(s/explain ...)` to succeed, given my understanding of `clojure.spec` semantics. However, it seems as though the argument inside `#(do (println "!!!!!") (println %) (= (:tv/star %) :char/ernie))` does not resolve `%` to the original map `{:tv/star :char/ernie :tv/co-star :char/bert}` but rather the transformed map `#:tv{:star [:starring/ernie :char/ernie], :co-star :char/bert}` that seems to mix in the output of applying conform against the spec definition of `:tv/star`.

Miscellaneous edge case observations:

  • If I replace the sibling spec `(s/keys)` with a simple predicate like `some?`, then it succeeds.

```
(s/explain
(s/and some?
(s/or
:ernie-and-bert (s/and #(do (println "!!!!!") (println %) (= (:tv/star %) :char/ernie))
#(= (:tv/co-star %) :char/bert))
:just-the-bird #(not (contains? % :tv/co-star))))
{:tv/star :char/ernie :tv/co-star :char/bert})
```

  • If I elide the first, solitary registered clojure.spec definition `(s/def :tv/star ...)`, then it succeeds.

I haven't investigated a solution yet.



 Comments   
Comment by Brian Noguchi [ 14/Sep/16 4:20 AM ]

It looks like the observed behavior is the expected correct behavior. I just noticed that successive conformed values are supposed to propagate through the rest of the predicates.

https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L439-L440

Comment by Brian Noguchi [ 14/Sep/16 4:21 AM ]

This issue can be ignored and closed. Sorry!





[CLJ-2020] New prohibited field names (__hash __hasheq) break existing software Created: 07/Sep/16  Updated: 08/Sep/16  Resolved: 08/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: lvh Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: defrecord


 Description   

The most recent Clojure alpha (1.9.0-alpha12) contained a patch closing CLJ-1224. This had the unfortunate side effect of breaking some existing software, notably datascript. I've filed an issue upstream as well: https://github.com/tonsky/datascript/issues/176

It's not clear to me what the best resolution is; IIUC the behavior datascript was trying to accomplish is what records now do automagically, although I might have misunderstood. Ideally, datascript wouldn't have the serious performance regression on >=1.8.0, but it definitely should compile on 1.9.0, regardless of how that's resolved.



 Comments   
Comment by Alex Miller [ 07/Sep/16 9:02 PM ]

My initial reaction is that datascript should not be using fields starting with __ as that is at least implied to be "internal stuff" in the defrecord docstring. But, I reserve the right to think about it more.

Comment by Alex Miller [ 08/Sep/16 9:51 AM ]

After thinking about it more, I will double down to say that extending a defrecord to IHashEq is also bad form. Records are intended to have the hash semantics of maps and to be controlled by the language, not by a record extender. I did quite a bit of searching and have found no other project that does this.

Implementing IHashEq for deftypes is a normal and perfectly acceptable thing to do as deftypes are considered to be opaque from Clojure's point of view - you give it the semantics.

Comment by Alex Miller [ 08/Sep/16 9:58 AM ]

Rich concurs so I am declining. The lib code should be changed.

Comment by lvh [ 08/Sep/16 9:59 AM ]

IIUC the options for datascript are either moving to deftype, or removing the IHashEq impl (and living with the perf penalty). Is there an option that lets them keep the perf, conditionally implementing only on 1.9.0? (That may not be desirable at all.)

Comment by Alex Miller [ 08/Sep/16 10:41 AM ]

It's conceivable to conditionally load different versions of a namespace that defines the records based on Clojure version. I'm not sure whether that's worth doing or not (and how it plays with AOT). I don't know whether it even affects perf or by how much - seems like that's the first question to answer. If it doesn't matter, then just remove it.





[CLJ-2018] clojure.spec.gen lazy-loaded fns should contain wrapped thing's docstring Created: 03/Sep/16  Updated: 06/Sep/16  Resolved: 06/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: lvh Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: docstring, spec


 Description   

Right now, using clojure.spec.gen's lazily loaded fns is harder than using the test.check versions because the former doesn't have useful docstrings and the latter does.

clojure.spec.gen lazy-loaded fns should contain docstring of the thing they refer to, or at least what it was at compile time, or at least a universally-understood reference to that thing that IDEs can follow. I don't think the latter exists, so perhaps the docstring at compile time should be embedded into the lazy-loaded fn if possible.

(As a general note, and I don't know if my experience generalizes, but I find myself grabbing for stuff in e.g. test.chuck so often that the "no runtime dependency" thing doesn't really work out anyway.)



 Comments   
Comment by Alex Miller [ 03/Sep/16 6:44 PM ]

I don't think they can contain those docstrings without actually loading the remote var to get its meta, which defies the whole point of lazy loading. So while I sympathize with the request, I'm not sure that I understand how it is possible to achieve?

Comment by lvh [ 04/Sep/16 10:28 AM ]

I was hoping a macro would be able to load the var, but only at compile time (e.g. when building the jar), not affecting a runtime consumer of the same ns; the same way that some of my ClojureScript code is built with macros that read resource files at compile time. (Perhaps that is not technologically possible right now in Clojure.)

Comment by Alex Miller [ 06/Sep/16 11:20 AM ]

I'm going to decline this as I'm not sure how it would be done while still satisfying the other goals of this code. But if someone finds a reasonable way to do it, would reconsider.





[CLJ-2016] function contains? return value error Created: 30/Aug/16  Updated: 30/Aug/16  Resolved: 30/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: wenjixiao Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

ubuntu 14.04



 Description   

hello.core=> (contains? [1 2 3] 1)
true
hello.core=> (contains? [1 2 3] 2)
true
hello.core=> (contains? [1 2 3] 3)
false

i am not sure, it's a bug or not, because it's so simple.
But, [1 2 3] should contains 3 right?!



 Comments   
Comment by Alex Miller [ 30/Aug/16 9:18 AM ]

This is a common question and this is the expected behavior.

contains? checks for containment of a key in an indexed collection. In a map, contains? works on keys. In a set, it works on the (hashed) elements. In a vector, it vector on the vector indexes (not the values).

So asking (contains? [1 2 3] 1) is asking whether there is an element at index 1 in [1 2 3], which there is (here it's 2). When you ask (contains? [1 2 3] 3) you are asking if [1 2 3] has index 3 (0-based), which it does not.

Hope that helps.

Comment by Alex Miller [ 30/Aug/16 9:27 AM ]

Also, more info here http://insideclojure.org/2015/01/27/clojure-tips-contains/





[CLJ-2014] (keyword "@type") can be printed, but not read Created: 26/Aug/16  Updated: 26/Aug/16  Resolved: 26/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.2
Fix Version/s: Backlog

Type: Defect Priority: Minor
Reporter: David Smith Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Approval: Incomplete
Waiting On: Rich Hickey

 Description   

user=> (keyword "")
:
user=> (prn-str *1)
":\n"
user=> (read-string *1)
java.lang.RuntimeException: java.lang.Exception: Invalid token: : (NO_SOURCE_FILE:0)

This obviously isn't a huge defect, but I'd argue that anything that can be printed should be readable.



 Comments   
Comment by David Smith [ 26/Aug/16 5:02 AM ]

This is a clone of http://dev.clojure.org/jira/browse/CLJ-732 which it appears was closed with no explanation. I have recently come up against this when deserializing json. IMO it doesn't make sense for the keyword function to be able to produce non-valid keywords. What is the reason for rejecting this?

Comment by Alex Miller [ 26/Aug/16 7:47 AM ]

This is a feature used by a lot of Clojure programs. See:

http://clojure.org/guides/faq#unreadable_keywords

Comment by David Smith [ 26/Aug/16 7:49 AM ]

Thank you for the explanation, this can therefor be closed.





[CLJ-2012] ns spec breaks gen-class forms that use strings as class names Created: 24/Aug/16  Updated: 07/Sep/16  Resolved: 07/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Alex Miller
Resolution: Completed Votes: 0
Labels: regression, spec

Attachments: Text File clj-2012-2.patch     Text File clj-2012.patch    
Patch: Code
Approval: Ok

 Description   

The following valid `ns` gen-class form is reported as invalid by spec:

(ns foo
  (:gen-class
     :name ^{org.apache.logging.log4j.core.config.plugins.Plugin
             {:name "LogstashConverter" :category "Converter"}
             org.apache.logging.log4j.core.pattern.ConverterKeys ["logstashJson"]} social.common.logging.LogstashPatternConverter
     :constructors {["[Ljava.lang.String;"] [String String]}
     :factory newInstance
     :init init
     :state state
     :extends org.apache.logging.log4j.core.pattern.LogEventPatternConverter))

This is because the ns spec assumes that all class names can be represented by simple-symbols, while in reality some can only be represented by strings.

Approach: Pull out spec for class identifiers which can be either a simple-symbol (class name) or a string and use that in signature (which is used for both gen-class constructors and methods.

Patch: clj-2012-2.patch



 Comments   
Comment by Stuart Halloway [ 26/Aug/16 7:23 AM ]

Nicola: Good catch, thanks for the report!

Alex: argtype is a pretty general name to grab at the top level spec ns. How about something like class-ident?





[CLJ-2010] clojure.spec/fdef does not add specs to doc Created: 22/Aug/16  Updated: 22/Aug/16  Resolved: 22/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: lvh Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec


 Description   

fdef docstring claims:

Once registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests, and (if
a macro) used to explain errors during macroexpansion.

When specifying an fdef for a fn, (:doc (meta my-fn) does not include any information about the spec, which is what I was expecting.



 Comments   
Comment by Alex Miller [ 22/Aug/16 10:52 PM ]

You are misinterpreting the intent there. What we mean there is that the clojure.repl/doc function will emit the spec as part of its output once you have declared a spec.

There is no intention to update the meta for a var when you declare an fdef spec.





[CLJ-2009] 'symbol and 'keyword turn "" into unreadable symbol and keyword respectively Created: 22/Aug/16  Updated: 23/Aug/16  Resolved: 22/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Aaron Brooks Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

In playing with clojure.spec for our upcoming Clojure meetup exercise and I found that (symbol "") returns an empty symbol which is unprintable and unreadable but can still be converted back to an empty string (name (symbol "")) => "". Similarly, (keyword "") returns ":" (which is invalid Clojure and cannot be read) but does round-trip as an object (name (keyword "")) => ""

I'm happy to provide a patch if we can determine the correct behavior. I'll start by making the assertion that the current behavior seems tricksy and prone to cause Great Mystery followed by Great Sadness.



 Comments   
Comment by Alex Miller [ 22/Aug/16 9:32 PM ]

Hey Aaron, symbols and keyword are programmatically constructable (and usable) without being constrained to what the reader can read and the printer can print, and that's often a useful feature (and something we don't consider to be a bug). It is possible that we might in the future have some kind of delimited keyword or symbol (| has been reserved for this) that would allow the reader to read these. I've done some research on this in the past and there were more complexities to it than initially thought so we ended up shelving it, but it's not dead, just in the deep freeze.

I've answered this question so many times that it's silly I haven't put it in the http://clojure.org/guides/faq, so I will do so for the next time.

I think when this comes up, what people most commonly want is some way to avoid making a non-readable keyword or symbol. So some kind of readable-keyword and readable-symbol functions that did validation and either threw an exception or created the keyword or symbol would maybe be a good enhancement idea.

Comment by Aaron Brooks [ 22/Aug/16 10:50 PM ]

Ships passing... I just saw the FAQ update and rechecked here.

The FAQ update is helpful. It might be worth noting in the docstrings of both 'symbol and 'keyword — that they are able to produce keywords and symbols which are unprintable/readable. I do also like the idea of readable-symbol and readable-keyword. Having a defined grammar for those would be great too.

Thanks for taking the time with this.

I am satisfied with my care. ;-D

Comment by Alex Miller [ 22/Aug/16 10:55 PM ]

If you want to file an enhancement for the readable-symbol/keyword, go for it. There is a grammar (kind of) on the http://clojure.org/reference/reader page, but it does not exactly line up with what's actually accepted in the regex, and there are several tickets already out there about the details of that misalignment. I think due to the intent that readable-symbol/keyword could take a conservative approach though and this would be pretty handy.

Comment by Aaron Brooks [ 22/Aug/16 11:03 PM ]

I meant grammar ala [E]BNF or similar. Something you can data rather than human. Also, as you note, the Java regex is pretty loose. I ran into this pretty directly when creating https://github.com/abrooks/clj-chopshop.

I've given some thought about proposing a grammar that would be used by the compiler (or other tools). Would patches for that be generally welcome or are we really attached to the regexes as they are? I understand that there are natural performance and compatibility concerns.

Comment by Alex Miller [ 23/Aug/16 8:23 AM ]

I don't think we're looking to replace the general reader strategy currently in use unless you could demonstrate significantly better performance.





[CLJ-2008] clojure.spec.test/check without args tests fns from clojure.core, erroring out with #:clojure.spec{:failure :no-fn} Created: 21/Aug/16  Updated: 07/Sep/16  Resolved: 07/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: lvh Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec
Environment:

N/A


Attachments: Text File clj-2008.patch    
Patch: Code
Approval: Ok

 Description   

When running (clojure.spec.test/check (in CIDER, but I doubt that's relevant other than it changes the formatting of the following snippet):

0. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/when-let, :spec clojure.spec$fspec_impl$reify__13891@6ee9efaf }
  1. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/let, :spec clojure.spec$fspec_impl$reify__13891@4ff7da85 }
  2. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/if-let, :spec clojure.spec$fspec_impl$reify__13891@35daec9c }
  3. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/fn, :spec clojure.spec$fspec_impl$reify__13891@98fcf1f }
... (some elided)
  8. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/ns, :spec clojure.spec$fspec_impl$reify__13891@283add16 }
  9. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/defn, :spec clojure.spec$fspec_impl$reify__13891@59681966 }
... (some elided)
  12. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/defn-, :spec clojure.spec$fspec_impl$reify__13891@3b34b0db }

It doesn't seem appropriate to tell me that these macros have no fn to spec. Perhaps it should know about macros.

Problem: checkable-syms does not omit spec'ed macros so when they flow down to check-1, they throw errors. Since these can't be checked, they should be omitted from checkable-syms. I put the change in fn-spec-name?, which is also used (and has the same problem in) instrumentable-syms.

Approach: Skip spec'ed macros.

Patch: clj-2008.patch



 Comments   
Comment by Alex Miller [ 22/Aug/16 10:10 AM ]

That's interesting as there were changes in alpha11 designed to catch and omit macros from check. So, something still amiss there.

Comment by Stuart Halloway [ 26/Aug/16 7:31 AM ]

lvh: thanks for the report!

Not sure that we want to give up on generatively testing macros, but it certainly doesn't work atm, so this seems good until we take it on.





[CLJ-2006] clojure.spec/fdef mentions nonexistent clojure.spec.test/run-tests Created: 21/Aug/16  Updated: 07/Sep/16  Resolved: 07/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Minor
Reporter: lvh Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: docstring, spec
Environment:

N/A


Attachments: Text File clj-2006.patch    
Patch: Code
Approval: Ok

 Description   

Once registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests, and (if
a macro) used to explain errors during macroexpansion.

Should be: clojure.spec.test/check



 Comments   
Comment by lvh [ 21/Aug/16 6:55 AM ]

Crud, I messed up the title of this issue copy-pasting from the docstring, and I appear to lack the permissions to resolve it.

Comment by Alex Miller [ 21/Aug/16 3:34 PM ]

lvh I've added you to the groups with edit permission.

Comment by Stuart Halloway [ 26/Aug/16 7:34 AM ]

Thanks again lvh!





[CLJ-2004] s/multi-spec doesn't include :retag in `s/form` Created: 18/Aug/16  Updated: 07/Sep/16  Resolved: 07/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Allen Rohner Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Attachments: Text File clj-2004.patch    
Patch: Code
Approval: Ok

 Description   

`s/form` on a multispec doesn't include the :retag, so it's impossible to recover that information from an existing spec. This is important for tooling that interacts with specs.

(require '[clojure.spec :as s])
(defmulti mm :mm/type)
(s/def ::foo (s/multi-spec mm :mm/type))
(s/form ::foo)
;; (clojure.spec/multi-spec user/mm) ;; :mm/type is missing

Approach: Include retag in form

Patch: clj-2004.patch



 Comments   
Comment by Alex Miller [ 18/Aug/16 6:18 PM ]

Yeah, this is one of many known form issues.





[CLJ-2000] (transduce) does an unexpected extra step Created: 06/Aug/16  Updated: 07/Aug/16  Resolved: 07/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Vadim Liventsev Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

https://stackoverflow.com/questions/38809642/applying-a-transducer-directly-and-with-transduce-yield-different-results/38809928#38809928

(reduce) calls (reducer aggregate element) for every element in the collection. A total of n calls for a collection of n elements.

(transduce) calls (reducer aggregate element) for every element and then for some reason calls (reducer aggregate) again, making n+1 calls. As a result, (transduce) doesn't work as expected with .

Is it a bug?



 Comments   
Comment by Kevin Downey [ 06/Aug/16 10:21 PM ]

this is a feature of transduce. the docstring for transduce says "f should be a reducing step function that accepts both 1 and 2 arguments, if it accepts only 2 you can add the arity-1 with 'completing'." which, if you know about the completing arity of reducing functions clues you in, but if you don' the docstring is not very helpful about it.

Comment by Alex Miller [ 07/Aug/16 11:42 AM ]

As Kevin said in the comments, the question assumes things that are not accurate. Everything here is working as expected/designed.





[CLJ-1999] Infinity and -Infinity do not equal literal versions if within a data structure Created: 03/Aug/16  Updated: 03/Aug/16  Resolved: 03/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Darrick Wiebe Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Description   

The first statement below is correctly true, but the second one is false.

(= [0 Double/POSITIVE_INFINITY] [0 Double/POSITIVE_INFINITY])

(= [0 Double/POSITIVE_INFINITY] (clojure.edn/read-string "[0 Infinity]"))



 Comments   
Comment by Alex Miller [ 03/Aug/16 1:37 PM ]

The problem here is that Infinity is read as a symbol, not as Double/POSITIVE_INFINITY.

I think these issues are captured better in http://dev.clojure.org/jira/browse/CLJ-1074 so I'm going to mark this as a dupe of that one.





[CLJ-1998] clj.spec: improve boolean kw option naming Created: 03/Aug/16  Updated: 09/Dec/16  Resolved: 09/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Max Penet Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec


 Description   

We have a mix of boolean keyword options with and without trailing "?" at the moment. It would be good to settle to 1 style, hopefully the one with the trailing "?".

Ex: in map-of we have :conform-keys, in double-in: NaN? and :infinite? and possibly others.



 Comments   
Comment by Alex Miller [ 09/Dec/16 4:21 PM ]

I don't think we're going to make any changes for this, thanks





[CLJ-1994] the-ns / clojure.lang.Namespace/find should handle nil values Created: 29/Jul/16  Updated: 31/Jul/16  Resolved: 31/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

After some refactoring, I had the (broken) code:

(test-all-vars (find-ns 'missing.ns))

when I ran my tests, I got this error:

...
Caused by: java.lang.NullPointerException
	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
	at clojure.lang.Namespace.find(Namespace.java:188)
	at clojure.core$find_ns.invokeStatic(core.clj:4000)
	at clojure.core$the_ns.invokeStatic(core.clj:4030)
	at clojure.core$ns_interns.invokeStatic(core.clj:4077)
	at clojure.test$test_all_vars.invokeStatic(test.clj:736)
	at clojure.test$test_all_vars.invoke(test.clj:736)
...

I expected there would be an exception thrown that the namespace could not be found.

Looking into it further, it seems like the issue is that find-ns was called twice in this code, once at the top level on the non-existent namespace which returned nil, and then a second time in the-ns with nil as the value. The second nil blows up inside clojure.lang.Namespace/find. It seems like there should be a nil check somewhere along the way, probably either in the-ns, or clojure.lang.Namespace/find, which would then throw a descriptive error message.



 Comments   
Comment by Alex Miller [ 29/Jul/16 7:58 AM ]

find-ns says "Returns the namespace named by the symbol or nil if it doesn't exist." and you're getting nil in this case.

So can't this just be simplified to (test-all-vars nil)? Why should we expect that to work?

Comment by Daniel Compton [ 31/Jul/16 5:15 AM ]

You're right, I'm not expecting it to work, I'm suggesting that we throw a more descriptive error. Thinking about this over the last few days, there are a few options:

Add a nil check in the-ns so that an exception is also thrown on nil passed as a namespace:

;; Something with this intent:
(if (instance? clojure.lang.Namespace x)
    x
    (or (and x (find-ns x)) (throw (Exception. (str "No namespace: " x " found")))))

Add a nil check in find-ns:

(defn find-ns
  "Returns the namespace named by the symbol or nil if it doesn't exist."
  {:added  "1.0"
   :static true}
  [sym] (when (some? sym) (clojure.lang.Namespace/find sym)))

Add a nil check in clojure.lang.Namespace/find:

public static Namespace find(Symbol name) {
        // If nil throw more descriptive error than NPE
        return (Namespace)namespaces.get(name);
    }

Thinking some more about it, all of these changes could be breaking to people who relied on the old behaviour. It's not clear to me whether this is worth continuing. Thoughts?

Comment by Alex Miller [ 31/Jul/16 8:16 AM ]

I think find-ns, Namespace/find, and the-ns are behaving as documented and intended. If anything, having a spec for test-all-vars would catch this more explicitly.

(s/fdef clojure.test/test-all-vars :args #(instance? clojure.lang.Namespace %))
(st/instrument 'clojure.test/test-all-vars)

;; results in:
user=> (clojure.test/test-all-vars (find-ns 'foo))
ExceptionInfo Call to #'clojure.test/test-all-vars did not conform to spec:
val: (nil) fails at: [:args] predicate: (instance? clojure.lang.Namespace %)
:clojure.spec/args  (nil)
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "NO_SOURCE_FILE", :line 11, :var-scope user/eval23}
  clojure.core/ex-info (core.clj:4724)

which would have pointed you pretty precisely to the problem. But I don't think it's worth keeping this ticket open re the spec as we are working on specs via other means.





[CLJ-1993] Print flag to suppress namespace map syntax Created: 28/Jul/16  Updated: 19/Aug/16  Resolved: 16/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Completed Votes: 0
Labels: print

Attachments: Text File clj-1993-2.patch     Text File clj-1993.patch    
Patch: Code and Test
Approval: Ok

 Description   

Add new print flag *print-namespace-maps* to optionally suppress namespace map syntax. Default flag to false.

Set flag to true as part of the standard REPL bindings. This allows repl users to (set! *print-namespace-maps* false) if desired.

Patch: clj-1993-2.patch



 Comments   
Comment by Rich Hickey [ 16/Aug/16 7:33 AM ]

the plan/approach should be somewhere in the description and not just the code please. Also, I wonder if the default should be false other than at the REPL?

Comment by Alex Miller [ 16/Aug/16 6:53 PM ]

Committed for next alpha





[CLJ-1992] Add explanation for clojure.test-clojure.metadata/public-vars-with-docstrings-have-added failure Created: 26/Jul/16  Updated: 26/Jul/16  Resolved: 26/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: docstring

Patch: Code and Test

 Description   

While I was adding the async macro patch, I ran the tests and clojure.test-clojure.metadata/public-vars-with-docstrings-have-added failed. As someone unfamiliar to the codebase, it wasn't very obvious why it had failed, or what I had to do to fix it. I traced it back to understand I needed to add {:added "1.9"}. This is obvious in hindsight, but wasn't apparent from the test failure message. The attached patch adds an explanation of why the test failed.

Compare the test output from before and after:

[java] FAIL in (public-vars-with-docstrings-have-added) (metadata.clj:46)
     [java] expected: (= [] (remove (comp :added meta) public-vars-with-docstrings-not-generated))
     [java]   actual: (not (= [] (#'clojure.test/async)))
[java] FAIL in (public-vars-with-docstrings-have-added) (metadata.clj:46)
     [java] All public vars with docstrings must have :added metadata, e.g. {:added "1.0"}
     [java] expected: (= [] (remove (comp :added meta) public-vars-with-docstrings-not-generated))
     [java]   actual: (not (= [] (#'clojure.test/async)))


 Comments   
Comment by Alex Miller [ 26/Jul/16 5:33 PM ]

"public vars with docstrings have added" is the documentation, not going to bother with this

Comment by Daniel Compton [ 26/Jul/16 5:38 PM ]

It's not the documentation, it's the name of the test, and it wasn't very clear. Changing the test name to public-vars-with-docstrings-have-added-metadata would improve things. This is a small thing, but it tripped me up, and it seems like it could make contributing to Clojure a tiny bit nicer and easier.





[CLJ-1991] ClassCastException in Compiler.java Created: 26/Jul/16  Updated: 27/Jul/16  Resolved: 26/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7, Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jeremy Betts Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

Linux Zulu JDK



 Description   

on line 272 of Compiler.java there is a class cast exception when the value from the system property is an Integer. We cannot upgrade from 1.6 until this is fixed

it is probably best to do a toString() rather than a hard cast.

for (Map.Entry e : System.getProperties().entrySet())
{
String name = (String) e.getKey();
String v = (String) e.getValue();



 Comments   
Comment by Alex Miller [ 26/Jul/16 12:53 PM ]

Hi Jeremy,

This has been filed and declined in the past (see CLJ-1717) on the grounds that System property values are defined to be Strings and to do otherwise is an error in whomever set that property, not on Clojure's assumption.

Alex

Comment by Jeremy Betts [ 26/Jul/16 1:33 PM ]

You realize that you are basically locking my company at version 1.6 right?

This should be fixed from a code quality perspective regardless.

A) why do an unchecked casts to string rather than using toString() and
B) its a one line change!

This type of thinking is only going to stop the adoption of Clojure from being used as anything more than a toy language.

In the 'real world' other code has bugs and it cannot always be fixed. You should be writing robust code that doesn't 'assume' that the world it is in is a pristine environment.

Comment by Sean Corfield [ 26/Jul/16 10:39 PM ]

Jeremy, Java itself defines System property values as Strings: https://docs.oracle.com/javase/7/docs/api/java/util/Properties.html

Can you show how you are "locked" at Clojure version 1.6 because of this?

It's not like you can store non-String values in there:

user=> (System/setProperty "foo" 123)

ClassCastException java.lang.Long cannot be cast to java.lang.String  user/eval1227 (form-init3357403087479620173.clj:1)

That's on Clojure 1.6.0.

Comment by Alex Miller [ 27/Jul/16 8:13 AM ]

Sean, some (bad) Java libs do this - Properties extends Hashtable and thus it's possible to use the super-class put method to put a non-String value into the System properties map after startup - Java calls these "compromised" properties. Creating properties like this breaks many API assumptions in the Properties class and is a known Bad thing to do.

Jeremy, I'm sorry that this is an issue for you, but the problem here is really with the code that is setting the System property, not Clojure.

Comment by Jeremy Betts [ 27/Jul/16 9:00 AM ]

I realize that Clojure has a garbage in garbage out approach - but since this code is in a static {} block, where exactly should I put 'safety code' to ensure that no non String values exist in the system properties? The JVM controls when static blocks get run!

Obviously non stings can get in there since this issue has been reported before - Alex you even new the previous issue number off the top of your head! Any third party code could be doing it.

It would be wise not go blundering into the well known areas of bad design in Java.

The solution is to call toString() instead of doing a unchecked hard cast to (String). This is simply a good coding standard anyway.

can you give any real reason not to make this simple change? You can't even claim performance since this is a static block which gets run exactly once!

We all know that java isn't the best language, that is why we seek other languages like Clojure. But if those other languages force the worst features of Java to the forefront to cause pain why bother?

I would need a reliable work around to this issue that ensures it can never happen. But since the issue can reoccur at the fancy of the JVM executing the static code elsewhere, this is a total show stopper.

Comment by Jeremy Betts [ 27/Jul/16 9:23 AM ]

Also, attempting to remove values from system properties to mitigate this issue has that high risk of throwing a concurrent modification exception.

Comment by Jeremy Betts [ 27/Jul/16 12:20 PM ]

here is a simple fix that represents better java programming practice and makes the code faster and safer.

for (Map.Entry e : System.getProperties().entrySet())
{
String name = (String) e.getKey();
if (name.startsWith("clojure.compiler."))

{ String v = (String) e.getValue(); compilerOptions = RT.assoc(compilerOptions, RT.keyword(null, name.substring(1 + name.lastIndexOf('.'))), RT.readString(v)); }

}

Comment by Alex Miller [ 27/Jul/16 1:25 PM ]

What is putting bad properties in your system? Shouldn't you be spending your effort on the root cause?

Comment by Jeremy Betts [ 27/Jul/16 1:37 PM ]

an ancient mail library. I'm pushing on both bugs.

Why don't you want to fix yours? I'm sure you've spent more time fighting with JIRA issues than what it would take to fix it.

Comment by Alex Miller [ 27/Jul/16 2:34 PM ]

Because it's not our bug.

Comment by Jeremy Betts [ 27/Jul/16 2:41 PM ]

you assign values you never use. You obtain these values from a global static hashmap that anyone can add any value to, even though they "should" only add strings. You do an unchecked assignment. You have a bug.





[CLJ-1988] collection specs conform to reversed list when used on a sequence Created: 24/Jul/16  Updated: 07/Sep/16  Resolved: 07/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Greg Chapman Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Attachments: Text File clj-1988.patch    
Patch: Code and Test
Approval: Ok

 Description   

Example

Clojure 1.9.0-alpha10
user=> (require '[clojure.spec :as s])
nil
user=> (s/conform (s/coll-of int?) (range 5))
(4 3 2 1 0)

Problem: Current code handles vectors, maps, and lists but falls through on sequences to the last case and adds to an empty sequence (at the head).

Approach: Add seqs to the list case.

Patch: clj-1988.patch



 Comments   
Comment by Alex Miller [ 23/Aug/16 2:01 PM ]

In case it helps, a workaround might be: (s/conform (s/coll-of int? :into ()) (range 5))





[CLJ-1987] Update clojure.java.javadoc to open JDK8 docs by default Created: 24/Jul/16  Updated: 24/Jul/16  Resolved: 24/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Richard Hull Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: documentation

Attachments: Text File jdk8_javadoc.patch    

 Description   

The clojure.java.javadoc function opens up JavaSE 7 docs in a browser - looking at the source (https://github.com/clojure/clojure/blob/master/src/clj/clojure/java/javadoc.clj#L21-L24), this hasn't been updated for a few years.

The attached trivial patch to set the default to use the Java 8 docs



 Comments   
Comment by Alex Miller [ 24/Jul/16 6:32 AM ]

There is already a ticket/patch for this at http://dev.clojure.org/jira/browse/CLJ-1398 - any comments should go there.

Comment by Richard Hull [ 24/Jul/16 10:12 AM ]

Ah ok, I did a search in JIRA and didn't spot CLJ-1398 in the results + couldn't find anything committed in git, so assumed that this was something that was worth submitting. Anyway, thanks for the quick response,
regards
Richard





[CLJ-1985] with-gen of conformer loses unform fn Created: 21/Jul/16  Updated: 16/Aug/16  Resolved: 16/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 1
Labels: spec

Attachments: Text File conformer-with-gen.patch    
Patch: Code and Test
Approval: Ok

 Description   
(def ex (s/conformer inc dec))
(s/conform ex 1) ;; 2
(s/unform ex 2)  ;; 1
(def exc
  (s/with-gen
    (s/conformer inc dec)
    (fn [] (s/gen int?))))
(s/conform exc 1) ;; 2
(s/unform exc 2) ;; fails, no unformer

Cause: with-gen doesn't re-apply the unform fn to the new spec

Patch: conformer-with-gen.patch



 Comments   
Comment by Alex Miller [ 16/Aug/16 8:47 AM ]

Committed for next alpha.





[CLJ-1983] clojure.data/diff throws an exception when comparing map keys of different types when used on sorted maps Created: 19/Jul/16  Updated: 19/Jul/16  Resolved: 19/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Thomas Scheiblauer Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Description   

e.g.

(clojure.data/diff (sorted-map :foo 42) (sorted-map 1 42))
(clojure.data/diff (sorted-map :foo 42) (sorted-map "x" 42))
(clojure.data/diff (hash-map :foo 42) (sorted-map 1 42))
(clojure.data/diff (hash-map :foo 42) (sorted-map "x" 42))

will throw
java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.Keyword
while e.g.

(clojure.data/diff (hash-map :foo 42) (hash-map 1 42))
(clojure.data/diff (hash-map :foo 42) (hash-map "x" 2))
(clojure.data/diff (sorted-map :foo 42) (sorted-map :bar 42))

will not.

The same applies to ClojureScript with a different exception (e.g. "Error: Cannot compare :foo to 1")



 Comments   
Comment by Alex Miller [ 19/Jul/16 8:23 AM ]

This is the same root problem as CLJ-1242, so duping to that one.

Comment by Thomas Scheiblauer [ 19/Jul/16 10:30 AM ]

It's not exactly a duplicate since diff should work in any case regardless of (compare x y) not working in this situation (possibly by design?).
(= (sorted-map :foo 42) (sorted-map 1 42)) works by the way.
(compare (sorted-map :foo 42) (sorted-map 1 42)) throws the exception.
In my opinion this could (and maybe should) be fixed in diff.

Comment by Alex Miller [ 19/Jul/16 12:41 PM ]

The stack traces for the two tickets are identical. diff is not using compare, it's using =. (= (sorted-map :foo 42) (sorted-map 1 42)) throws.

user=> (clojure.data/diff (hash-map :foo 42) (sorted-map "x" 42))
ClassCastException java.lang.String cannot be cast to clojure.lang.Keyword  clojure.lang.Keyword.compareTo (Keyword.java:114)
user=> (pst *e)
ClassCastException java.lang.String cannot be cast to clojure.lang.Keyword
	clojure.lang.Keyword.compareTo (Keyword.java:114)
	clojure.lang.Util.compare (Util.java:153)
	clojure.lang.RT$DefaultComparator.compare (RT.java:280)
	clojure.lang.PersistentTreeMap.doCompare (PersistentTreeMap.java:311)
	clojure.lang.PersistentTreeMap.entryAt (PersistentTreeMap.java:298)
	clojure.lang.PersistentTreeMap.containsKey (PersistentTreeMap.java:94)
	clojure.lang.APersistentMap.equiv (APersistentMap.java:87)
	clojure.lang.Util.pcequiv (Util.java:124)
	clojure.lang.Util.equiv (Util.java:32)
	clojure.data/diff (data.clj:134)
	clojure.data/diff (data.clj:120)
	user/eval20 (NO_SOURCE_FILE:11)
Comment by Thomas Scheiblauer [ 19/Jul/16 1:28 PM ]

You are of course right as I can see clearly now.
I did overlook the asymmetrical behavior of '=' in context of a sorted map.
Please excuse my ignorance.





[CLJ-1981] `spec/merge` does not flow conformed values across preds per docstring Created: 13/Jul/16  Updated: 18/Jul/16  Resolved: 18/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Moritz Ulrich Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec
Environment:

[org.clojure/clojure "1.9.0-alpha10"]


Approval: Vetted

 Description   

The order of specs passed to spec/merge affect the spec/conform behavior of the keys specified. This seem to happen only with non-prefixed keys via (spec/keys :req-un [..])

The following code snippet shows the broken/non-intuitive behavior:

(require '[clojure.spec :as s])

(s/def ::id (s/conformer str))
(s/def ::m (s/keys :req-un [::id]))

;; Correct behavior on ::id
(s/conform ::id 42)
;;=> "42"

;; Fine if unmerged
(s/conform ::m {:id 42})
;;=> {:id "42"}

;; Fine if merged with ::m in the *last* position
(s/conform (s/merge map? ::m) {:id 42})
;;=> {:id "42"}

;; Broken because `map?` is the last arg
(s/conform (s/merge ::m map?) {:id 42})
;;=> {:id 42}

;; Broken because another `s/keys` is used as the last argument
(s/conform (s/merge ::m (s/keys :req-un [::foo]))
           {:id 42 :foo 23})
;;=> {:id 42, :foo 23}


 Comments   
Comment by Alex Miller [ 13/Jul/16 8:36 AM ]

Perhaps a simpler pair of examples - the first should return the result of the second if conformed values are flowing through the predicates.

(s/conform
  (s/merge (s/map-of keyword? (s/or :s string? :n number?))
           map?)
  {:x "s"})
=> {:x "s"}
(s/conform
  (s/merge map?
           (s/map-of keyword? (s/or :s string? :n number?)))
  {:x "s"})
=> {:x [:s "s"]}
Comment by Alex Miller [ 18/Jul/16 9:04 AM ]

This is working as designed. s/merge should not flow conformed values. The docstring has been corrected in https://github.com/clojure/clojure/commit/d920ada9fab7e9b8342d28d8295a600a814c1d8a





[CLJ-1977] Printing a Throwable throws if Throwable has no cause / stacktrace Created: 06/Jul/16  Updated: 08/Jul/16  Resolved: 08/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: regression
Environment:

alpha9


Attachments: Text File clj-1977.patch    
Patch: Code and Test
Approval: Ok

 Description   

Throwable->map in core_print.clj doesn't handle Throwable.getCause returning null in L463. This results in a NPE in StrackTraceElement->vec in the same file in some cases, so printing a stacktrace results in a new exception being thrown which is a bit confusing.

Repro:

(def t (Throwable.))
(.setStackTrace t (into-array StackTraceElement []))
(Throwable->map t) ;; throws npe during conversion
(pr t) ;; throws during printing

Approach: Check that at least one StackTraceElement exists before using the top frame. Make printing tolerant of a missing :at value. Add test for this omitted stack trace case.

Patch: clj-1977.patch






[CLJ-1976] using spec/fspec seems to require clojure.test.check Created: 05/Jul/16  Updated: 05/Jul/16  Resolved: 05/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nikita Prokopov Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: spec
Environment:

clojure 1.9.0-alpha8



 Description   
(spec/def ::translate
  (spec/fspec
    :args (spec/cat :locale keyword?
                    :key keyword?
                    :args (spec/* ::spec/any))
    :ret  string?))

(defn tr [l k & args] ...)

(spec/conform ::translate tr)

Uncaught exception, not in assertion.
expected: nil
  actual: java.io.FileNotFoundException: Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.


 Comments   
Comment by Alex Miller [ 05/Jul/16 10:52 AM ]

Dupe of CLJ-1936 I believe.





[CLJ-1974] Clojure.org URLs in docstrings are broken Created: 03/Jul/16  Updated: 04/Jul/16  Resolved: 04/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8, Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Brandon Bloom Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

Many links of the form http://clojure.org/data_structures#hash now have the form http://clojure.org/reference/data_structures#hash with the reference/ subpath in it.

I think the right thing to do is to set up some up some redirects on clojure.org, but if you think it's better to change the docstrings, I can submit a patch.



 Comments   
Comment by Alex Miller [ 03/Jul/16 7:00 PM ]

Hmm, I actually did set up redirects for all old links, so something must be up with the deployment. In the future, filing issues about the site is best at the Clojure-site github issues. We don't plan to change the links in the source.

Comment by Brandon Bloom [ 03/Jul/16 8:26 PM ]

Didn't realize that was on GH. For other looking, I found it: https://github.com/clojure/clojure-site

Comment by Alex Miller [ 04/Jul/16 8:32 AM ]

Last deploy of the site failed so redirects were broken. Site now redeployed and links working.

Comment by Alex Miller [ 04/Jul/16 9:18 AM ]

Deployment failure is due to intermittent AWS errors so I have also added some automatic retry support.





[CLJ-1971] Update docstring of empty? to suggest not-empty instead of seq Created: 27/Jun/16  Updated: 28/Jun/16  Resolved: 28/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

The docstring for empty? says

clojure.core/empty? [coll]
Returns true if coll has no items - same as (not (seq coll)). Please use the idiom (seq x) rather than (not (empty? x))

Would it make more sense to suggest using not-empty, instead of seq here?



 Comments   
Comment by Alex Miller [ 28/Jun/16 9:20 AM ]

No, the recommended idiom is still to use seq as a termination condition in this case.





[CLJ-1970] instrumented macros never conform valid forms Created: 25/Jun/16  Updated: 05/Jul/16  Resolved: 05/Jul/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: OHTA Shogo Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Approval: Ok

 Description   

Although macros can be speced without &form and &env, once they are instrumented they will try to conform the args including &form/&env and fail:

user=> (require '[clojure.spec :as s])
nil
user=> (defmacro m [x] x)
#'user/m
user=> (s/fdef m :args (s/cat :arg integer?) :ret integer?)
user/m
user=> (m 1)
1
user=> (m a)
CompilerException java.lang.IllegalArgumentException: Call to user/m did not conform to spec:
In: [0] val: a fails at: [:args :arg] predicate: integer?
:clojure.spec/args  (a)
, compiling:(NO_SOURCE_PATH:5:1) 
user=> (s/instrument)
[user/m]
user=> (m 1)
ExceptionInfo Call to #'user/m did not conform to spec:
In: [0] val: (m 1) fails at: [:args :arg] predicate: integer?
:clojure.spec/args  ((m 1) nil 1)
  clojure.core/ex-info (core.clj:4718)
user=>

To resolve the situation, I think instrument/instrument-all should avoid speced macros.



 Comments   
Comment by Alex Miller [ 25/Jun/16 9:42 AM ]

This is an issue we're discussing - for the moment, you should not instrument macros. There is no point in instrumenting them as they are automatically checked during macroexpansion.

Comment by OHTA Shogo [ 25/Jun/16 10:00 AM ]

Yes, I know the compiler checks macro specs automatically, but just thought it would be nice if explicit calls to instrument (with no args) and instrument-all would check whether or not each speced var is a macro and filter it out if so.

Comment by Alex Miller [ 25/Jun/16 2:30 PM ]

Totally agreed.

Comment by Alex Miller [ 05/Jul/16 1:58 PM ]

Fixed in https://github.com/clojure/clojure/commit/d8aad06ba91827bf7373ac3f3d469817e6331322 for 1.9.0-alpha9





[CLJ-1969] :as form is unbound when no optional keyword arguments is passed even though :or form is provided Created: 24/Jun/16  Updated: 24/Jun/16  Resolved: 24/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Lars Andersen Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   
(defn fn-with-kw-opts [& {:keys [opt] :or {opt 1} :as options}]
  (println "opt " opt "options " options))

user> (fn-with-kw-opts)
;;=> opt  1 options  nil

user> (fn-with-kw-opts :opt 2)
;;=> opt  2 options  {:opt 2}

I would expect options to be bound to the default value when no keyword argument is passed.



 Comments   
Comment by Alex Miller [ 24/Jun/16 7:36 AM ]

:as binds to the original value passed in and will never include values from :or. :or is used to provide defaults for each local being bound when that local is missing in the input.

In the case of (fn-with-kw-opts), the incoming value is nil so options is bound to nil.

Comment by Alex Miller [ 24/Jun/16 7:38 AM ]

Working as designed.





[CLJ-1967] Enhanced namespaced map pprint support Created: 23/Jun/16  Updated: 31/Aug/16  Resolved: 31/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: print

Attachments: Text File clj-1967.patch    
Patch: Code
Approval: Ok

 Description   

CLJ-1910 added namespaced map syntax for reader but did not include pprint support.

user=> (binding [clojure.pprint/*print-right-margin* 40]
         (pprint {:the.namespace/lajsdflkajsd 1 :the.namespace/aljsdfjasdf 2}))
#:the.namespace{:lajsdflkajsd 1,
                :aljsdfjasdf 2}
nil
user=> (binding [clojure.pprint/*print-right-margin* 40
                 *print-namespace-maps* false]
  (pprint {:the.namespace/lajsdflkajsd 1 :the.namespace/aljsdfjasdf 2}))
{:the.namespace/lajsdflkajsd 1,
 :the.namespace/aljsdfjasdf 2}

Patch: clj-1967.patch






[CLJ-1964] rmap / *recursion-limit* not respected through custom generators Created: 19/Jun/16  Updated: 29/Jun/16  Resolved: 28/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: generator, spec

Approval: Triaged

 Description   

In all cases where a custom generator is used, the recursion limit is not respected.

This limitation becomes clear when one tries to build a recursive spec around e. g. s/coll-of because it uses a custom generator via s/coll-gen. Running s/exercise on it quickly blows the stack.

Here is an example for illustration with s/map-of

(s/def ::map-tree (s/map-of keyword? (s/or :tree ::map-tree :leaf nil?)))

Even though s/or implements recursion checking, it is deceived here and not able to detect itself being called subsequently because the custom generator of s/coll-of (used in s/map-of) doesn't/can't pass rmap (keeping track of recursion calls) through to s/or's gen*.

For the concrete case, coll-of-impl could be implemented that would implement a gen* that passes on rmap.

For the general case of custom generators the challenge would be that test.check generators don't take and pass on rmap to generators of specs they potentially reuse and that there is no well-defined behavior for what they themselves should do when the recursion-limit has been reached. Ideas are:

  • reduce generator size when recursion is detected (this is the strategy used in recursive specs of test.check use(d)?)
  • expose the recursion-limit / rmap mechanism to the user so that custom generators can pass it on to subsequent calls of specs. E. g. a custom generator is passed a context object that it should pass to s/gen as an optional argument


 Comments   
Comment by Leon Grapenthin [ 19/Jun/16 5:08 PM ]

I have changed the issue because in the former description I had made some assumptions that I could prove incorrect by studying the implementation a bit more.

Comment by Alex Miller [ 25/Jun/16 9:37 AM ]

Please re-check this after the next alpha - there are a lot of changes happening in this area.

Comment by Alex Miller [ 28/Jun/16 9:58 PM ]

As of Clojure 1.9.0-alpha8, due to changes in map-of etc, s/exercise now works on this example.

Comment by Leon Grapenthin [ 29/Jun/16 9:15 AM ]

@Alex Miller - I haven't had time yet to check whether latest design changes especially in spec.test solve recursion through custom generators or make it obsolete. The examples given in above description have clearly been solved but they were only examples for a larger problem. Would you like me to change this ticket or should I create a new one?

Comment by Alex Miller [ 29/Jun/16 1:58 PM ]

I would create a new ticket unless it's substantially similar to this one, in which case you can re-open it.





[CLJ-1963] clojure.spec/map-of has confusing error message when input is not a map Created: 15/Jun/16  Updated: 28/Jun/16  Resolved: 28/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Russell Mull Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: errormsgs, spec


 Description   

When using map-of specs, the error message given when checking a non-map value is less than enlightening:

user> (require '[clojure.spec :as s])
nil
user> (s/def ::int-map (s/map-of integer? integer?))
:user/int-map
user> (s/explain ::int-map :not-a-map)
val: :not-a-map fails spec: :user/int-map predicate: (coll-checker (tuple integer? integer?))
nil

This can be worked around to some degree by requiring that the value be a map explicitly:

user> (s/def ::fancy-int-map (s/and map? ::int-map))
:user/fancy-int-map
user> (s/explain ::fancy-int-map :not-a-map)
val: :not-a-map fails spec: :user/fancy-int-map predicate: map?
nil

The definition

map-of
looks like it's trying to do just this, but the
map?
predicate comes second for some reason.



 Comments   
Comment by Alex Miller [ 28/Jun/16 9:54 PM ]

map-of implementation changed a lot in alpha8 and you will now see the error for your first example as:

user=> (s/explain ::int-map :not-a-map)
val: :not-a-map fails spec: :user/int-map predicate: map?




[CLJ-1962] fn-spec only works with a fully ns-qualified quoted symbol Created: 15/Jun/16  Updated: 16/Jun/16  Resolved: 16/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Laszlo Török Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec

Approval: Vetted

 Description   

fn-spec no longer does symbol resolution on its parameter.

However, the following

(sp/fdef + :args (sp/cat :operand (sp/* number?)))

(sp/fn-spec +) ;; => nil (A)
(sp/fn-spec '+) ;; => nil (B)
(sp/fn-spec 'clojure.core/+) ;; this actually returns the fn-specs

Proposal: Either resolve the symbol/var or document that fully-qualified is required.

Also see:
https://gist.github.com/laczoka/acd65028f5a46338e33c940d49d01753



 Comments   
Comment by Laszlo Török [ 15/Jun/16 11:32 AM ]

thanks Alex for making the ticket more palatable

Comment by Alex Miller [ 16/Jun/16 8:39 AM ]

fn-spec has been renamed to get-spec in master and works a bit differently than before. However, it requires a qualified symbol, keyword, or var.

If you want resolution in terms of the local namespace when invoking it, use ` as a helper:

(sp/get-spec `plus)
Comment by Laszlo Török [ 16/Jun/16 4:13 PM ]

Fantastic!





[CLJ-1961] clojure.spec regression bug for 1.9.0-alpha6: ignores :ret function Created: 14/Jun/16  Updated: 14/Jun/16  Resolved: 14/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alan Thompson Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

clojure 1.9.0-alpha6



 Description   

Just noticed that the :ret function in fdef seems to be ignored in 1.9.0-alpha6 (works in 1.9.0-alpha5):

user=> (require '[clojure.spec :as s])
user=> (defn dummy [x] (if x "yes" "no"))
user=> (s/fdef dummy
#_=> :args (s/cat :x integer?)
#_=> :ret integer?)
user=> (s/instrument #'dummy)
user=> (dummy 3) (println clojure-version)
ExceptionInfo Call to #'user/dummy did not conform to spec:
val: "yes" fails at: [:ret] predicate: integer?
:clojure.spec/args (3)
clojure.core/ex-info (core.clj:4703)
{:major 1, :minor 9, :incremental 0, :qualifier alpha5}

;-------------------------------------------------------------------

user=> (dummy 3) (println clojure-version)
"yes"
{:major 1, :minor 9, :incremental 0, :qualifier alpha6}



 Comments   
Comment by Alex Miller [ 14/Jun/16 7:10 PM ]

This was an intentional change in what instrument does.

Instrument is intended to be used to verify that other callers have invoked a function correctly.

Checking that the function works (by verifying that :ret and :fn return valid results) should be done using one of the spec.test functions during testing.

Some other spec features are still to be added as well that relate to this change.





[CLJ-1958] Add uri? generator Created: 12/Jun/16  Updated: 14/Jun/16  Resolved: 14/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: generator, spec

Attachments: Text File clj-1958.patch    
Patch: Code
Approval: Ok

 Description   

uri? was added as a predicate in 1.9 but doesn't have a mapped spec generator.

Proposed: Generate uuids, then produce URIs of the form "http://<uuid>.com".

Patch: clj-1958.patch



 Comments   
Comment by Alex Miller [ 14/Jun/16 11:02 AM ]

Applied for 1.9.0-alpha6.





[CLJ-1957] Add gen support for bytes? Created: 11/Jun/16  Updated: 14/Jun/16  Resolved: 14/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: generator, spec

Attachments: Text File clj-1957.patch    
Patch: Code
Approval: Ok

 Description   

The generator for the new bytes? predicate was overlooked.



 Comments   
Comment by Alex Miller [ 14/Jun/16 11:02 AM ]

Applied for 1.9.0-alpha6.





[CLJ-1947] Add vec-of spec Created: 05/Jun/16  Updated: 28/Jun/16  Resolved: 28/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Maarten Truyens Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec


 Description   

It would be great to add a "vec-of" (and perhaps also a "set-of") Spec, similar to the already existing map-of. I find myself writing (coll-of ::foo []) writing too often.



 Comments   
Comment by Alex Miller [ 28/Jun/16 10:01 PM ]

With 1.9.0-alpha8, you can now get this same effect using:

(s/coll-of ::foo :kind vector?)
(s/coll-of ::foo :kind set?)




[CLJ-1946] improve error messages for `map-of` spec Created: 04/Jun/16  Updated: 29/Jun/16  Resolved: 28/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Chris Price Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: errormsgs, spec


 Description   

When using a map-of spec where the value predicate refers to another spec, the error message if a value does not conform does not seem like it is as helpful as it could be:

(spec/def ::myint integer?)
(spec/explain
 (spec/map-of string? ::myint)
 {"x" 1 "y" "not an int!"})
=> :user.swagger-ui-service/myint
val: {"x" 1, "y" "not an int!"} fails predicate: (coll-checker (tuple string? :user.swagger-ui-service/myint))

The explain result doesn't indicate which key/value pair in the map failed to conform, and it also doesn't make it clear that the integer? predicate is ultimately the one that caused the failure.

From reading through the introduction and docs, it seemed like error messaging was a primary motivation for spec, so any improvements that might be possible to this error message would be extremely valuable.



 Comments   
Comment by Chris Price [ 04/Jun/16 11:22 AM ]

This becomes particularly important with deeply-nested specs and/or large maps.

Comment by Sean Corfield [ 04/Jun/16 11:53 PM ]

In my opinion, the failure should use the spec name, not the underlying predicate function, in the message – isn't that the whole point of using named specs in this?

(as for explaining the value that failed, I agree on the surface but haven't looked at the implementation in detail to see if there might be a good reason)

Comment by Chris Price [ 06/Jun/16 12:23 PM ]

If nothing else, it's inconsistent with what gets logged for other types of specs:

(spec/explain
 (spec/cat :myint ::myint)
 ["hi"])
 
In: [0] val: "hi" fails spec: :puppetlabs.trapperkeeper.main/myint at: [:myint] predicate: integer?
=> nil
(spec/explain
 (spec/tuple ::myint)
 ["hi"])
 
In: [0] val: "hi" fails spec: :puppetlabs.trapperkeeper.main/myint at: [0] predicate: integer?
=> nil

Having spent a good chunk of a day hacking on a project that involved some very deeply-nested specs, I can definitively say that the logging from cat and tuple was much easier to debug. With those, I could keep poking at my "production" code via the REPL, tweaking things until the specs were correct. With map-of, the only way I was able to debug was to copy/paste the entire failed val into the REPL and cobble together a one-off spec/explain form to repro, and then delete things from it until I got past the map-of so that the error was less opaque. Then once I fixed the actual issue I'd need to copy/paste the REPL stuff back into my "production" code. It wasn't impossible, but it was a much more tedious workflow than when dealing with errors from cat or tuple.

Comment by Alex Miller [ 13/Jun/16 3:50 PM ]

The issues here is due to the way map-of and coll-of sample their contents rather than fully conforming all of them. This is done for performance but is not what most people expect. It's likely there will be some more additions in this area.

Comment by Alex Miller [ 28/Jun/16 10:00 PM ]

As of 1.9.0-alpha8, map-of now conforms all entries and the error you'll see is:

In: ["y" 1] val: "not an int!" fails spec: :user/myint at: [1] predicate: integer?
Comment by Chris Price [ 29/Jun/16 12:12 PM ]

\o/ thanks!





[CLJ-1945] provide a way to write a map spec that disallows extra keys Created: 04/Jun/16  Updated: 24/Jul/16  Resolved: 04/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Chris Price Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

After reading the initial docs and tutorials, I was expecting that calling `conform` or `valid` with a `key` spec and a map that contained extra keys would indicate a spec failure, but it doesn't:

```clj
(spec/conform
(spec/keys :req [::foo ::bar])
{::foo "foo" ::bar "bar" ::baz "baz"})
=>
{:user.swagger-ui-service/foo "foo",
:user.swagger-ui-service/bar "bar",
:user.swagger-ui-service/baz "baz"}
```

Obviously this behavior is desirable in many situations, but perhaps there could also be another spec type, called `exact-keys` or something, that would fail in the above example because of the presence of the non-specified `::baz` key.

This seems like it would be particularly useful when specifying the return value for a function that is returning data for an HTTP endpoint, to make sure that the program isn't violating the API specification by including extraneous data in the response.

This can be achieved with the current `spec` by `and`ing together a `keys` spec with a custom predicate that does some set logic on the keys, but that is a little unwieldy and repetitive, and doesn't produce as nice of an error message as what could probably be done if it were built in.



 Comments   
Comment by Alex Miller [ 04/Jun/16 9:05 PM ]

I am declining this ticket as this was considered and intentionally not provided. Rich believes that maps should be open containers and that extra attributes should be allowed (similar philosophy behind having open records).

As you mention, there are other ways to add this constraint if desired.

Comment by Jan-Paul Bultmann [ 24/Jul/16 4:18 PM ]

I hope you guys reconsider at some point, this is the only reason why I'll stick with schema.





[CLJ-1944] (into {}) fails for pairs represented as anything other than vectors Created: 03/Jun/16  Updated: 05/Jun/16  Resolved: 05/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Defect Priority: Major
Reporter: John Napier Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: bug, compiler, exceptions
Environment:

Linux 3.13.0-63-generic #103-Ubuntu SMP x86_64 GNU/Linux



 Description   

This works:

(into {} [[:a 1]])
;=> {:a 1}

This also works:

(into {} (list (vector :a 1)))
;=> {:a 1}

Bizarrely enough, even this works:

(into {} [{:a 1}])
;=> {:a 1}

This produces a ClassCastException:

(into {} [(list :a 1)])
;=> java.lang.ClassCastException: clojure.lang.Keyword cannot be cast to java.util.Map$Entry
	at clojure.lang.ATransientMap.conj(ATransientMap.java:44)
	at clojure.lang.ATransientMap.conj(ATransientMap.java:17)
	at clojure.core$conj_BANG_.invokeStatic(core.clj:3257)
	at clojure.core$conj_BANG_.invoke(core.clj:3249)
	at clojure.lang.PersistentList.reduce(PersistentList.java:141)
	at clojure.core$reduce.invokeStatic(core.clj:6544)
	at clojure.core$into.invokeStatic(core.clj:6610)
	at clojure.core$into.invoke(core.clj:6604)
	at user$eval4419.invokeStatic(form-init625532025826918014.clj:1)
	at user$eval4419.invoke(form-init625532025826918014.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:6927)
	at clojure.lang.Compiler.eval(Compiler.java:6890)
	at clojure.core$eval.invokeStatic(core.clj:3105)
	at clojure.core$eval.invoke(core.clj:3101)
	at clojure.main$repl$read_eval_print__7408$fn__7411.invoke(main.clj:240)
	at clojure.main$repl$read_eval_print__7408.invoke(main.clj:240)
	at clojure.main$repl$fn__7417.invoke(main.clj:258)
	at clojure.main$repl.invokeStatic(main.clj:258)
	at clojure.main$repl.doInvoke(main.clj:174)
	at clojure.lang.RestFn.invoke(RestFn.java:1523)
	at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__663.invoke(interruptible_eval.clj:87)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.core$apply.invokeStatic(core.clj:646)
	at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1881)
	at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1881)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:85)
	at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55)
	at clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__708$fn__711.invoke(interruptible_eval.clj:222)
	at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__703.invoke(interruptible_eval.clj:190)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Likewise, this produces a similar ClassCastException:

(into {} [#{:a 1}])
;=> ClassCastException ....

There doesn't seem to be any documentation on into that implies it only works when kv pairs are represented as vectors (or somehow, maps), so this seems to be a bug. It's extremely surprising that it doesn't work for pairs represented as lists.

For the interested, I found this by writing a function to invert a map in the most natural way I could think of:

(defn invert-map [m]
  (into {} (map (fn [[k v]] [v k]) m)))

(invert-map {:a 1 :b 2})
;=> {1 :a 2 :b}, no alarms and no surprises

; wait, this is pretty stupid, why don't I just use reverse...

(defn invert-map [m]
  (into {} (map reverse m)))

(invert-map {:a 1 :b 2})
;=> :(

Confirmed with Clojure 1.7 on Ubuntu 3.13.0-63-generic 64bit.



 Comments   
Comment by Sean Corfield [ 05/Jun/16 12:03 AM ]

There are definitely some odd edge cases around MapEntry but I would invert a map like this rather than trying to rely on a sequence of MapEntry objects:

(reduce-kv (fn [m k v] (assoc m v k)) {} m)

The fact that a map behaves as a sequence of pairs seems to cause a lot of confusion.

Comment by Alex Miller [ 05/Jun/16 3:05 PM ]

into takes a collection of elements to be conj'ed into the target collection. The differences in your examples are all around what one of those elements is, so this is really a question about conj'ing into a map. Map conj is (lightly) documented at http://clojure.org/reference/data_structures#Maps to take one of:

  • a map whose entries will be added
  • a map entry
  • a vector of 2 items

The examples you mention are lists and sets, which are none of the above. Lists are not supported because the key and value are plucked in constant time where as lists would have to be traversed sequentially to get to the 0th and 1st element. I do not think the time difference is significant but that is the philosophical reason. Sets are not supported because they are not ordered, so that to me makes no sense at all as there is no meaning of the 0th and 1st element at all.

For map-invert, you might try the one that is (obscurely) in clojure.set:

I don't see any bug here - everything is happening as designed, so I'm going to close this ticket.





[CLJ-1943] clojure.spec should implicitly convert classes to specs Created: 03/Jun/16  Updated: 09/Dec/16  Resolved: 09/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Kevin Corcoran Assignee: Unassigned
Resolution: Declined Votes: 3
Labels: spec


 Description   

It would be nice if clojure.spec implicitly converted Java classes to specs, as it does for predicates. As a comparison, plumatic/schema allows classes to be used as schemas directly, and I take advantage of this regularly, as I currently use both schema and interop quite heavily.

For example, the spec guide contains the following:

(import java.util.Date)
(s/valid? #(instance? Date %) (Date.))  ;; true

... and then, later, defines:

(s/def ::date #(instance? Date %))

If classes were implicitly converted to specs, ::date would be unnecessary, and the first example could be simplified to:

(import java.util.Date)
(s/valid? Date (Date.))  ;; true

This would make clojure.spec a lot easier to use and adopt on my projects.



 Comments   
Comment by Alex Miller [ 04/Jun/16 9:07 PM ]

This was proposed and we decided not to include it in the initial release of spec. I do not know that we will in the future though, so leaving this open for now.

Comment by Sean Corfield [ 04/Jun/16 11:37 PM ]

At World Singles we use Expectations and it also automatically treats Java classes as type-based predicates. That said, I don't think a core library should do this. It's convenient "magic" but it doesn't actually feel very Clojure-y. I think I would vote against this being added to clojure.spec.

Comment by Alex Miller [ 09/Jun/16 9:03 AM ]

Note that for this particular example, inst? is now available in core.

Comment by Alex Miller [ 09/Dec/16 4:22 PM ]

We're not going to do this at the current time.





[CLJ-1940] spec has no way to specify a non-fn var should always conform Created: 30/May/16  Updated: 09/Dec/16  Resolved: 09/Dec/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Allen Rohner Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec


 Description   

It appears there's no way to specify that a non-function var should always conform, after e.g. alter-var-root or binding.



 Comments   
Comment by Alex Miller [ 05/Jun/16 3:20 PM ]

I'm not sure it makes sense to do this at all in the case of a def. If you really want to check it on definition you could do so by explicitly calling valid?.

If you want to check changes via alter-var-root, you can do so by setting a var validator using http://clojure.github.io/clojure/clojure.core-api.html#clojure.core/set-validator!

I again don't think it makes a lot of sense to do anything automatic in binding either. You can always validate it explicitly if you want to.

Basically, I think this is outside the use case spec is trying to cover but I'll check with Rich before declining.

Comment by Alex Miller [ 09/Dec/16 4:22 PM ]

I don't think we're going to add anything for this at the current time (but maybe it will be considered again in the future).





[CLJ-1939] clojure.spec evaluates the predicate once, but the conformer several times Created: 29/May/16  Updated: 05/Jun/16  Resolved: 05/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Georgi Danov Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec
Environment:

1.9 alpha-3



 Description   
(defmacro eq [x]
  `(sp/&
     (fn [v#]
       (println "#~" v#)
       (= v# ~x))
     (sp/conformer #(do (println "#" %) %))))
   
;; twice
(sp/conform (sp/cat :a (eq :a)) [:a])

;; 3 times
(sp/conform (sp/cat :a (sp/alt :a-a (eq :a))) [:a])

;; 4 times
(sp/conform (sp/cat :a (sp/alt :a-a (sp/alt :a-a-a (eq :a)))) [:a])


 Comments   
Comment by Sean Corfield [ 01/Jun/16 9:24 PM ]

I raised a similar issue with Rich on Slack (on May 24th) and he said:

"the predicates are presumed to be pure, they will be run an arbitrary # of times and how many is an implementation detail, they get run to determine if a subexpression ​can​ return and again when it ​does​ return for instance, in addition to regular regex speculation"

When I noted "that explain called the predicate a different number of times to conform / valid?" he said:

"explain is a similar but different call path that does more work, doesn;t fail fast, builds paths etc"

s/& considers both arguments to be "predicates" and it just happens to run the second one multiple (arbitrary) times during processing.

Comment by Alex Miller [ 05/Jun/16 3:15 PM ]

I believe this is working as expected, as explained in the comments, so closing.





[CLJ-1938] Namespaced record fields in defrecord Created: 28/May/16  Updated: 01/Jun/16  Resolved: 01/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: J. S. Choi Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: defrecord, keywords, symbols


 Description   

Currently, Clojure records—the preferred Clojure solution to single-dispatch polymorphic data—support only bare, non-namespaced field names. In contrast, the new clojure.spec standard library opinionatedly focuses on fields identified by namespaced keywords.

  • "spec will allow (only) namespace-qualified keywords and symbols to name specs."
  • "Encourage and support attribute-granularity specs of namespaced keyword to value-spec."
  • "People using namespaced keys for their informational maps" is "a practice we'd like to see grow."

The spec guide notes that "unqualified keys can also be used to validate record attributes", using the :req-un and :opt-un options in spec/keys.

In order for records to leverage clojure.spec fully, however, it may be worth somehow adding support namespaced record fields in defrecord.

One example of how this might be done is something like (defrecord Record [x/a y/b] ...). One disadvantage is that it is not clear how to specify that a field belongs to the current namespace. Allowing keywords would allow double-colon :: syntax to be used (defrecord Record [::a] ...), but this may be confusing. (Alternatively, a syntax for namespacing symbols in the current namespace, similarly to double-colon keywords, might instead be implemented, but that would be out of scope of this issue.)

(Also out of scope of this issue, though also related, would be whether CLJ-1910 namespaced maps could somehow be applied to record map literals (e.g., #foo.Record{:a 2}.)



 Comments   
Comment by Alex Miller [ 01/Jun/16 6:25 AM ]

You can use spec with records now via the :req-un and :opt-un support for unqualified map keys (there is an example in the guide). Additional support may still be added that leverages the namespace of the record type itself.

There are no plans to add namespaced keys to records.





[CLJ-1937] spec/fn-specs should behave the same as s/spec w.r.t not-found Created: 28/May/16  Updated: 14/Jun/16  Resolved: 14/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Enhancement Priority: Minor
Reporter: Allen Rohner Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Approval: Vetted

 Description   

s/spec and s/fn-specs behave differently for 'not-found' values:

(s/spec ::bogus)
=> Exception Unable to resolve spec: :user/bogus  clojure.spec/the-spec (spec.clj:95)

(s/fn-specs 'bogus)
=> {:args nil, :ret nil, :fn nil}

fn-specs should throw or return nil

Note: doc uses the return of fn-specs so needs to be checked that it still works properly if this changes



 Comments   
Comment by Alex Miller [ 13/Jun/16 3:44 PM ]

There will be some updates to fn-specs soon and this should be included.

Comment by Alex Miller [ 14/Jun/16 9:20 AM ]

As of https://github.com/clojure/clojure/commit/92df7b2a72dad83a901f86c1a9ec8fbc5dc1d1c7, fn-spec (was fn-specs) now returns nil if no fn spec is found.





[CLJ-1936] instrumented fdef with fspec unnecessarily invokes fspec generator Created: 28/May/16  Updated: 05/Jan/17  Resolved: 03/Nov/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Allen Rohner Assignee: Unassigned
Resolution: Declined Votes: 2
Labels: spec, test

Approval: Triaged

 Description   

With test.check is on the classpath, an instrumented fdef with fspec will invoke the generator for the fspec when invoked:

(require '[clojure.spec :as s] '[clojure.spec.test :as st])

(defn foo [fnn] (fnn 42))
(s/fdef foo :args (s/cat :f (s/fspec :args (s/cat :i integer?)
                                     :ret integer?)))

(foo #(do (println %) (when (even? %) 42)))
42
42

(st/instrument `foo)

(foo #(do (println %) (when (even? %) 42)))
-1
0
-1
0
0
-1
0
-1
ExceptionInfo Call to #'user/foo did not conform to spec:
In: [0] val: nil fails at: [:args :f :ret] predicate: integer?
:clojure.spec/args  (#object[user$eval12$fn__13 0x515c6049 "user$eval12$fn__13@515c6049"])
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "NO_SOURCE_FILE", :line 8, :var-scope user/eval12}
  clojure.core/ex-info (core.clj:4725)

Without test.check, this fails:

user=> (foo #(do (println %) (when (even? %) 42)))
FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.  clojure.lang.RT.load (RT.java:458)


 Comments   
Comment by Zach Oakes [ 28/May/16 9:01 PM ]

I think it would make sense to add something like core.typed's ^:no-check for this. For example:

(s/fdef ^:no-check foo :args (s/cat :f (s/fspec :args (s/cat :i integer?) :ret integer?)))

As a stopgap measure, I made a boot task that has a copy of clojure.spec.test/run-all-tests and modifies it to ignore vars with that metadata. That means I have to add it to the metadata in defn rather than fdef but it still seems to work.

Comment by Sean Corfield [ 01/Jun/16 9:32 PM ]

Yes, there are definitely situations where I would want argument / return spec checking on calls during dev/test but absolutely need the function excluded from generative testing.

Comment by Sean Corfield [ 01/Jun/16 9:38 PM ]

If you don't have test.check on your classpath, the call to s/instrument succeeds but then attempting to call foo fails with:

boot.user=> (defn foo [fnn] (fnn 42))
#'boot.user/foo
boot.user=> (s/fdef foo :args (s/cat :f (s/fspec :args (s/cat :i integer?)
       #_=>                                      :ret integer?)))
boot.user/foo
boot.user=> 

boot.user=> (foo #(do (println %) (when (even? %) 42)))
42
42
boot.user=> (s/instrument 'foo)
#'boot.user/foo
boot.user=> (foo #(do (println %) (when (even? %) 42)))

java.io.FileNotFoundException: Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.

That is certainly unexpected and not very friendly!

Comment by Alex Miller [ 28/Jun/16 10:05 PM ]

There are new options in instrument as of 1.9.0-alpha8 that allow you to stub/mock functions. Those are one potential answer to this and maybe the recommended one, although I haven't used them enough to say that for sure.

Comment by Alex Miller [ 05/Jul/16 10:52 AM ]

See also CLJ-1976.

Comment by Sean Corfield [ 18/Aug/16 4:08 PM ]

Given that recent Alpha builds no longer check :ret or :fn with instrumentation, this issue seems to be resolved Alex Miller?

Comment by Allen Rohner [ 18/Aug/16 4:22 PM ]

fspec still requires generative testing.

Comment by Sean Corfield [ 18/Aug/16 4:24 PM ]

Ah, OK, I thought that had also rolled back to just :args testing at this point (I hadn't retested this since we have test.check as dev/test dependency now anyway).

Comment by Alex Miller [ 03/Nov/16 12:24 PM ]

If an argument takes a function and you pass it a function, instrument cannot validate the fspec other than by generating args for the fspec and trying it out. So, this is the intended behavior. So, this is working as intended.

Comment by Allen Rohner [ 03/Nov/16 12:59 PM ]

Instrument could wrap the function in an fn that checks arguments the same way instrument does. fspec being tied to generators makes it significantly less useful for functions with side effects.

Comment by Alex Miller [ 03/Nov/16 3:54 PM ]

That's an interesting idea, not sure if it satisfies though. Also note that instrument provides a number of options for specifying simpler instrumented replacement specs/stubs.

Comment by Brandon Bloom [ 05/Jan/17 3:33 PM ]

Following up from a discussion occurring now in Slack #clojure-spec. Lots more discussion there, check timestamp.

I found this very surprising. I've been using instrument during dev/repl-time (as recommended? distinguishing form test-time), but just learned that fspec isn't suitable for functions with side-effects due to the use of generators. I expected instrument to only instrument calls to vars, never to use the generators, and either 1) ignore fspec's details (easier) or 2) proxy the fn (many challenges here).

instrument has been very useful for dev (again, vs test), but not as useful as it could be if :ret and :fn were checked, and generators were never used.

Comment by Brandon Bloom [ 05/Jan/17 3:38 PM ]

Whoops submitted a moment too soon. I wanted to add: During dev, I'd rather more things be checked a little bit (ie ret and fn) than fewer things be tested thoroughly (ie higher order functions subjected to generated inputs).

I think a distinction needs to be drawn between what can be conformed with 100% confidence and what cannot. For pure data w/o functions, we can expect conform to make some promises. With fspec etc, we can't rely on valid? => true as a security measure. I'm fine with that, but different contexts/times require different levels of confidence.





[CLJ-1935] clojure.spec/multi-spec ignores the multimethod hierarchy Created: 28/May/16  Updated: 07/Sep/16  Resolved: 07/Sep/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Viktor Magyari Assignee: Unassigned
Resolution: Completed Votes: 1
Labels: multimethods, spec

Attachments: Text File clj-1935-1.patch     Text File clj-1935.patch    
Patch: Code
Approval: Ok

 Description   

Minimal example:

(require '[clojure.spec :as s])

(def h (derive (make-hierarchy) :a :b))

(defmulti spec-type identity :hierarchy #'h)

(defmethod spec-type :b [_]
  (s/spec (constantly true)))

(s/def ::multi (s/multi-spec spec-type identity))

(s/explain ::multi :b)
;; => Success!
;; as expected

(s/explain ::multi :a)
;; => val: :a fails at: [:a] predicate: spec-error/spec-type,  no method
;; fails even though :a derives from :b

Also fails with the default hierarchy. Worked fine in alpha1, broken in alpha2 and alpha3.

Cause: The implementation of multi-spec uses a predicate that invokes the dispatch function, then looks up the result in the method table. However this does not leverage the actual logic used in multimethods for hierarchy resolution.

Approach: Replace the lookup in the method table with a call to getMethod(), which will use the same lookup logic that multimethod uses.

Patch: clj-1935.patch



 Comments   
Comment by Rich Hickey [ 06/Sep/16 12:20 PM ]

dval should change similarly

Comment by Alex Miller [ 06/Sep/16 1:41 PM ]

dval was and is correct, I think? it's just the means of looking up a match for the dispatch value.





[CLJ-1934] (s/cat) with nonconforming data causes infinite loop in explain-data Created: 27/May/16  Updated: 01/Jun/16  Resolved: 01/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Defect Priority: Major
Reporter: Sven Richter Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec
Environment:

Ubuntu 15.10
Leiningen 2.6.1 on Java 1.8.0_91 Java HotSpot(TM) 64-Bit Server VM


Attachments: Text File clj-1934.patch    
Patch: Code and Test
Approval: Vetted

 Description   

The following code yields an infinite loop

(require '[clojure.spec :as s])
(s/explain-data (s/cat) [1]) ;; infinite loop
​

Thread dump:

"main" prio=5 tid=0x00007fb602000800 nid=0x1703 runnable [0x0000000102b3f000]
   java.lang.Thread.State: RUNNABLE
	at clojure.lang.RT.seqFrom(RT.java:529)
	at clojure.lang.RT.seq(RT.java:524)
	at clojure.core$seq__5444.invokeStatic(core.clj:137)
	at clojure.core$concat$cat__5535$fn__5536.invoke(core.clj:715)
	at clojure.lang.LazySeq.sval(LazySeq.java:40)
	- locked <0x000000015885e4e0> (a clojure.lang.LazySeq)
	at clojure.lang.LazySeq.seq(LazySeq.java:56)
	- locked <0x000000015885e2f0> (a clojure.lang.LazySeq)
	at clojure.lang.RT.seq(RT.java:522)
	at clojure.core$seq__5444.invokeStatic(core.clj:137)
	at clojure.core$map$fn__5872.invoke(core.clj:2637)
	at clojure.lang.LazySeq.sval(LazySeq.java:40)
	- locked <0x000000015885e3b0> (a clojure.lang.LazySeq)
	at clojure.lang.LazySeq.seq(LazySeq.java:49)
	- locked <0x000000015885e3b0> (a clojure.lang.LazySeq)
	at clojure.lang.ChunkedCons.chunkedNext(ChunkedCons.java:59)
	at clojure.lang.ChunkedCons.next(ChunkedCons.java:43)
	at clojure.lang.RT.next(RT.java:689)
	at clojure.core$next__5428.invokeStatic(core.clj:64)
	at clojure.core$dorun.invokeStatic(core.clj:3033)
	at clojure.core$doall.invokeStatic(core.clj:3039)
	at clojure.walk$walk.invokeStatic(walk.clj:46)
	at clojure.walk$postwalk.invokeStatic(walk.clj:52)
	at clojure.spec$abbrev.invokeStatic(spec.clj:114)
	at clojure.spec$re_explain.invokeStatic(spec.clj:1286)
	at clojure.spec$regex_spec_impl$reify__11725.explain_STAR_(spec.clj:1305)
	at clojure.spec$explain_data_STAR_.invokeStatic(spec.clj:143)
	at clojure.spec$spec_checking_fn$conform_BANG___11409.invoke(spec.clj:528)
	at clojure.spec$spec_checking_fn$fn__11414.doInvoke(spec.clj:540)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at user$eval8.invokeStatic(NO_SOURCE_FILE:5)
	at user$eval8.invoke(NO_SOURCE_FILE:5)
	at clojure.lang.Compiler.eval(Compiler.java:6942)
	at clojure.lang.Compiler.eval(Compiler.java:6905)
	at clojure.core$eval.invokeStatic(core.clj:3105)
	at clojure.core$eval.invoke(core.clj:3101)
	at clojure.main$repl$read_eval_print__8495$fn__8498.invoke(main.clj:240)
	at clojure.main$repl$read_eval_print__8495.invoke(main.clj:240)
	at clojure.main$repl$fn__8504.invoke(main.clj:258)
	at clojure.main$repl.invokeStatic(main.clj:258)
	at clojure.main$repl_opt.invokeStatic(main.clj:322)
	at clojure.main$main.invokeStatic(main.clj:421)
	at clojure.main$main.doInvoke(main.clj:384)
	at clojure.lang.RestFn.invoke(RestFn.java:397)
	at clojure.lang.Var.invoke(Var.java:375)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)

Cause: This line in op-describe:

(cons `cat (mapcat vector (c/or (seq ks) (repeat :_)) (c/or (seq forms) (repeat nil)))))

is called here with no ks or form, so will mapcat vector over infinite streams of :_ and nil.

Approach: check for this case and avoid doing that

Patch: clj-1934.patch



 Comments   
Comment by Alex Miller [ 01/Jun/16 6:26 AM ]

This was fixed via an alternate change in 1.9.0-alpha4.





[CLJ-1933] please add unless macro for symmetry with when Created: 27/May/16  Updated: 27/May/16  Resolved: 27/May/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Ernesto Alfonso Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: enhancement


 Description   

Is there a reason there is a `when` macro but no `unless`? I think it useful, CL uses it, adds consistency/symmetry and conciseness to code.

(defmacro unless [test & body]
`(when (not ~test) ~@body))



 Comments   
Comment by Ragnar Dahlen [ 27/May/16 2:28 PM ]

There is already when-not: http://clojure.github.io/clojure/clojure.core-api.html#clojure.core/when-not

Comment by Alex Miller [ 27/May/16 3:47 PM ]

As Ragnar says, when-not is equivalent.





[CLJ-1932] Add clojure.spec/explain-str to return explain output as a string Created: 25/May/16  Updated: 26/May/16  Resolved: 26/May/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: Release 1.9

Type: Enhancement Priority: Major
Reporter: D. Theisen Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: spec

Approval: Vetted

 Description   

Currently explain prints to *out* - add a function explain-str that returns the explain output as a string.



 Comments   
Comment by Alex Miller [ 25/May/16 9:51 AM ]

You can easily capture the string with (with-out-str (s/explain spec data)).

Comment by Alex Miller [ 26/May/16 8:35 AM ]

explain-str was added in https://github.com/clojure/clojure/commit/575b0216fc016b481e49549b747de5988f9b455c for 1.9.0-alpha3.





[CLJ-1931] clojure.spec/with-gen throws AbstractMethodError Created: 24/May/16  Updated: 25/May/16  Resolved: 25/May/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tyler van Hensbergen Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None
Environment:

OSX Yosemite 10.10.5



 Description   

An AbstractMethodError is encountered when trying to evaluate a s/def form with the generator-fn overridden using s/with-gen.

(ns spec-fun.core
  (:require [clojure.spec :as s]
            [clojure.test.check.generators :as gen]))

(s/def ::int integer?)

(s/def ::int-vec
  (s/with-gen
    (s/& (s/cat :first ::int
                :rest  (s/* integer?)
                :last  ::int)
         #(= (:first %) (:last %)))
    #(gen/let [first (s/gen integer?)
               rest  (gen/such-that
                      (partial at-least 3)
                      (gen/vector (s/gen integer?)))]
       (concat [first] rest [first]))))
;;=> AbstractMethodError

;; The generator works independently
(gen/generate (gen/let [first (s/gen integer?)
                        rest  (gen/such-that
                               (partial at-least 3)
                               (gen/vector (s/gen integer?)))]
                (concat [first] rest [first])))
;;=> (-13 8593 -33421108 4 6697 0 35835865 -94366552 1 14165115 -4090 42 775 -15238320 233500020 -1 -13)

;; And so does the spec:
(s/def ::int-vec
  (s/& (s/cat :first ::int
              :rest  (s/* integer?)
              :last  ::int)
       #(= (:first %) (:last %))))

(s/conform ::int-vec '(-13 8593 -33421108 4 6697 0 35835865 -94366552 1 14165115 -4090 42 775 -15238320 233500020 -1 -13))
;;=> {:first -13, :rest [8593 -33421108 4 6697 0 35835865 -94366552 1 14165115 -4090 42 775 -15238320 233500020 -1], :last -13}


 Comments   
Comment by Alex Miller [ 25/May/16 10:13 AM ]

Fixed in commit https://github.com/clojure/clojure/commit/ec2512edad9c0c4a006980eedd2a6ee8679d4b5d for 1.9 alpha2.





[CLJ-1930] IntelliJ doesn't allow debugging of clojure varargs from Java Created: 22/May/16  Updated: 22/May/16  Resolved: 22/May/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Mathias Bogaert Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Attachments: PNG File intellij.png    

 Description   

When trying to debug evaluate Datomic's datoms API IntelliJ or the method thows "java.lang.IllegalArgumentException : Invalid argument count: expected 2, received 3". Debugging Java varargs is not an issue.

Using IntelliJ 2016.2 CE.



 Comments   
Comment by Mathias Bogaert [ 22/May/16 9:06 AM ]

Datomic 0.9.5359, JDK 1.8.0_74, OS/X 10.11.5.

Comment by Kevin Downey [ 22/May/16 1:56 PM ]

hi, this is the issue tracker for the Clojure programming language, not Datomic or Intellij. http://www.datomic.com/support.html lists various support options for datomic

Comment by Alex Miller [ 22/May/16 3:55 PM ]

Agreed with Kevin, this is an issue with Cursive and you can find that tracker here:

https://github.com/cursive-ide/cursive/issues

I think this existing ticket is relevant:

https://github.com/cursive-ide/cursive/issues/326





[CLJ-1919] Destructuring support for namespaced keys and syms Created: 27/Apr/16  Updated: 23/Jun/16  Resolved: 23/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Release 1.9

Type: Enhancement Priority: Critical
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Completed Votes: 1
Labels: destructuring

Attachments: Text File clj-1919-2.patch     Text File clj-1919.patch    
Patch: Code and Test
Approval: Ok

 Description   

Expand destructuring to better support a set of keys (or syms) from a map when the keys share the same namespace.

Example:

(def m {:person/first "Darth" :person/last "Vader" :person/email "darth@death.star"})

(let [{:keys [person/first person/last person/email]} m]
  (format "%s %s - %s" first last email))

Proposed: The special :keys and :syms keywords used in associative destructuring may now have a namespace (eg :person/keys). That namespace will be applied during lookup to all listed keys or syms when they are retrieved from the input map.

Example (also uses the new literal syntax for namespaced maps from CLJ-1910):

(def m #:person{:first "Darth" :last "Vader" :email "darth@death.star"})

(let [{:person/keys [first last email]} m]
  (format "%s %s - %s" first last email))
  • The key list after :ns/keys should contain either non-namespaced symbols or non-namespaced keywords. Symbols are preferred.
  • The key list after :ns/syms should contain non-namespaced symbols.
  • As :ns/keys and :ns/syms are read as normal keywords, auto-resolved keywords work as well: ::keys, ::alias/keys, etc.
  • Clarification - the :or defaults map always uses non-namespaced symbols as keys - that is, they are always the same as the locals being created (not the keys being looked up in the map). No change in behavior here, just trying to be explicit - this was not previously well-documented for namespaced key lookup and was broken. The attached patch fixes this behavior.

Patch: clj-1919-2.patch



 Comments   
Comment by Alex Miller [ 06/Jun/16 7:26 AM ]

This patch now needs to be re-worked on top of https://github.com/clojure/clojure/commit/0aa346766c4b065728cde9f9fcb4b2276a6fe7b5

Comment by Alex Miller [ 14/Jun/16 9:34 AM ]

Rebased patch to current master. No semantic changes as they didn't actually conflict, just were close enough to confuse git.





[CLJ-1914] Range realization has a race during concurrent execution Created: 14/Apr/16  Updated: 19/Aug/16  Resolved: 19/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7, Release 1.8
Fix Version/s: Release 1.9

Type: Defect Priority: Critical
Reporter: Ghadi Shayban Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: range

Attachments: Text File clj-1914-2.patch     Text File clj-1914-3.patch     Text File CLJ-1914.patch    
Patch: Code and Test
Approval: Ok

 Description   

When a range instance is enumerated via next concurrently, some threads may not see the entire range sequence. When this happens next will return nil prematurely.

(defn enumerate [r]
 (loop [rr r
        c []]
  (let [f (first rr)]
   (if f
    (recur (next rr) (conj c f))
    c))))

(defn demo [size threads]
 (let [r (range size)
       futures (doall (repeatedly threads #(future (enumerate r))))
       res (doall (map deref futures))]
  (if (apply = res)
   (println "Passed")
   (println "Failed, counts=" (map count res)))))

(demo 300 4)
Failed, counts= (300 64 300 64)

The demo above will reliably produce a failure every few executions like the one above.

The vast majority of the time, range is used either single-threaded or in a non-competing way where this scenario will never happen. This failure only occurs when two or more threads are enumerating rapidly through the same range instance.

Cause:

Each LongRange instance acts in several capacities - as a seq, a chunked seq, and a reducible, all of which represent independent enumeration strategies (multiple of which may be used by the user). LongRange holds 2 pieces of (volatile, non-synchronized) state related to chunking: _chunk and _chunkNext. That state is only updated in forceChunk(). forceChunk() uses the "racy single-check idiom" to tolerate multiple threads racing to create the chunk. That is, multiple threads may detect that the chunk has not been set (based on null _chunk) and BOTH threads will create the next chunk and write it. But both threads have good local values, compute the same next value, and set the same next values in the fields, so the order they finish is unimportant.

The problem here is that there are actually two fields, and they are set in the order _chunk then _chunkNext. Because the guard is based on _chunk, it's possible for a thread to think the chunk values have been set but _chunkNext hasn't yet been set.

Approach:

Moving the set for _chunkNext before the set for _chunk removes that narrow window of race opportunity.

Patch: clj-1914-3.patch

Thanks to Kyle Kingsbury for the initial reproducing case https://gist.github.com/aphyr/8746181beeac6a728a3aa018804d56f6



 Comments   
Comment by Alex Miller [ 14/Apr/16 11:13 PM ]

It's not necessary to synchronize here - just swapping these two lines should address the race I think: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LongRange.java#L131-L132

Comment by Ghadi Shayban [ 14/Apr/16 11:22 PM ]

Good call. That's super subtle.

It appears all mutable assignment occurs in forceChunk() except for https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LongRange.java#L145 which should be OK (famous last words).

Comment by Stuart Halloway [ 27/Jul/16 3:27 PM ]

This ticket would benefit from an explanation of the failure mode caused by the race, plus an explanation of the subtle reasoning behind the fix.

Comment by Alex Miller [ 27/Jul/16 4:27 PM ]

Updated per request.





[CLJ-1910] Namespaced maps Created: 07/Apr/16  Updated: 23/Sep/16  Resolved: 23/Jun/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Release 1.9

Type: Enhancement Priority: Critical
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Completed Votes: 1
Labels: print, reader

Attachments: Text File clj-1910-2.patch     Text File clj-1910.patch    
Patch: Code and Test
Approval: Ok

 Description   

A common usage of namespaced keywords and symbols is in providing attribute disambiguation in map contexts:

{:person/first "Han" :person/last "Solo" :person/ship 
  {:ship/name "Millenium Falcon" :ship/model "YT-1300f light freighter"}}

The namespaces provide value (disambiguation) but have the downside of being repetitive and verbose.

Namespaced maps are a reader (and printer) feature to specify a namespace context for a map.

  • Namespaced maps combine a default namespace with a map and yield a map.
  • Namespaced maps are reader macros starting with #: or #::, followed by a normal map definition.
    • #:sym indicates that sym is the default namespace for the map to follow.
    • #:: indicates that the default namespace auto-resolves to the current namespace.
    • #::sym indicates that sym should be auto-resolved using the current namespace's aliases OR any fully-qualified loaded namespace.
      • These rules match the rules for auto-resolved keywords.
  • A namespaced map is read with the following differences from normal maps:
    • A keyword or symbol key without a namespace is read with the default namespace as its namespace.
    • Keys that are not symbols or keywords are not affected.
    • Keys that specify an explicit namespace are not affected EXCEPT the special namespace _, which is read with NO namespace. This allows the specification of bare keys in a namespaced map.
    • Values are not affected.
    • Nested map keys are not affected.
  • The edn reader supports #: but not #:: with the same rules as above.
  • Maps will be printed in namespaced map form only when:
    • All map keys are keywords or symbols
    • All map keys are namespaced
    • All map keys have the same namespace

Examples:

;; same as above - notice you can nest #: maps and this is a case where the printer roundtrips
user=> #:person{:first "Han" :last "Solo" :ship #:ship{:name "Millenium Falcon" :model "YT-1300f light freighter"}}
#:person{:first "Han" :last "Solo" :ship #:ship{:name "Millenium Falcon" :model "YT-1300f light freighter"}}

;; effects on keywords with ns, without ns, with _ ns, and non-kw
user=> #:foo{:kw 1, :n/kw 2, :_/bare 3, 0 4}
{:foo/kw 1, :n/kw 2, :bare 3, 0 4}

;; auto-resolved namespaces (will use user as the ns)
user=> #::{:kw 1, :n/kw 2, :_/bare 3, 0 4}
:user/kw 1, :n/kw 2, :bare 3, 0 4}

;; auto-resolve alias s to clojure.string
user=> (require '[clojure.string :as s])
nil
user=> #::s{:kw 1, :n/kw 2, :_/bare 3, 0 4}
{:clojure.string/kw 1, :n/kw 2, :bare 3, 0 4}

;; to show symbol changes, we'll quote the whole thing to avoid evaluation
user=> '#::{a 1, n/b 2, _/c 3}
{user/a 1, n/b 2, c 3}

;; edn reader also supports (only) the #: syntax
user=> (clojure.edn/read-string "#:person{:first \"Han\" :last \"Solo\" :ship #:ship{:name \"Millenium Falcon\" :model \"YT-1300f light freighter\"}}")
#:person{:first "Han", :last "Solo", :ship #:ship{:name "Millenium Falcon", :model "YT-1300f light freighter"}}

Patch: clj-1910-2.patch

Screener notes:

  • Autoresolution supports fully-qualified loaded namespaces (like auto-resolved keywords)
  • TODO: pprint support for namespaced maps
  • TODO: printer flag to suppress printing namespaced maps


 Comments   
Comment by Nicola Mometto [ 08/Apr/16 3:57 AM ]

1- yes please. that's consistent with how tagged literals work.
2- no please. that would make the proposed syntax useless for e.g. Datomic schemas, for which I think this would be a good fit to reduce noise
3- yes please
4- yes please, consistency over print methods is important

Comment by Nicola Mometto [ 08/Apr/16 4:00 AM ]

Quoting from a post I wrote on the clojure-dev ML:

  • I really don't like the idea of special-casing `_` here, users are already confused about idioms like `[.. & _]` thinking that `_` is some special token/variable. Making it an actual special token in some occasion wouldn't help.
  • I also don't like how we're making the single `:` auto-qualify keywords when used within the context of a qualifying-map. Auto-qualifying keywords has always been the job of the double `::`, changing this would introduce (IMO) needless cognitive overhead.
  • The current impl treats `#:foo{'bar 1}` and `'#:foo{bar 1}` differently. I can see why is that, but the difference might be highly unintuitive to some.
  • The current proposal makes it feel like quote is auto-qualifying symbols , when that has always been the job of syntax-quote. I know that's not correct, but that's how it's perceived.

Here's an alternative syntax proposal that handles all those issues:

  • No #::, only #:foo or #::foo
  • No auto-resolution of symbols when the namespaced-map is quoted, only when syntax-quoted
  • No special-casing of `_`
  • No auto-resolution of single-colon keywords

Here's how the examples in the ticket description would look:

#:person{::first "Han", ::last "Solo", ::ship #:ship{::name "Millenium Falcon", ::model "YT-1300f light freighter"}}
;=> {:person/first "Han" :person/last "Solo" :person/ship {:ship/name "Millenium Falcon" :ship/model "YT-1300f light freighter"}}

#:foo{::kw 1, :n/kw 2, :bare 3, 0 4}
;=> {:foo/kw 1, :n/kw 2, :bare 3, 0 4}

{::kw 1, :n/kw 2, bare 3, 0 4}
;=> {:user/kw 1, :n/kw 2, :bare 3, 0 4}

Note in the previous example how we don't need `#::` at all – `::` already does that job for us

(require '[clojure.string :as s])
#::s{::kw 1, :n/kw 2, bare 3, 0 4}
;=> {:clojure.string/kw 1, :n/kw 2, :bare 3, 0 4}

`{a 1, n/b 2, ~'c 3}
;=> {user/a 1, n/b 2, c 3}

Again, no need for `#::` here, we can just rely on the existing auto-qualifying behaviour of `.

`#:foo{a 1, n/b 2}
;=> {foo/a 1, n/b 2}

I think this would be more consistent with the existing behaviour – it's basically just making `#:foo` or `#::foo` mean: in the top-level keys of the following map expression, resolve keywords/symbols as if ns was bound to `foo`, rather than introducing new resolution rules and special tokens.

I realize that this proposal wouldn't work with EDNReader as-is, given its lack of support for `::` and "`". I don't have a solution to that other than "let's just bite the bullet and implement them there too", but maybe that's not acceptable.

Comment by Alex Miller [ 08/Apr/16 8:45 AM ]

Nicola, thanks for the proposal, we talked through it. We share your dislike for :_/kw syntax and you should consider that a placeholder for this behavior for the moment - it may be removed or replaced before we get to a published release.

For the rest of it:

  • requiring syntax quote is a non-starter
  • supporting a mixture of default ns and the current ns is important and this is not possible with your proposal. Like #:foo{:bar 1 ::baz 2}.
  • there is a lot of value to changing the scope of a map without modifying the contents, which is an advantage of the syntax in the ticket
Comment by Christophe Grand [ 08/Apr/16 10:31 AM ]

Why restrict this feature to a single namespace? (this doesn't preclude a shorthand for the single mapping) I'd like to locally define aliases (and default ns).

Comment by Alex Miller [ 08/Apr/16 11:02 AM ]

We already have namespace level aliases. You can use :: in the map to leverage those aliases (independently from the default ns):

(ns app 
  (:require [my.domain :as d]
            [your.domain :as y]))

#::{:svc 1, ::d/name 2, ::y/name 3}

;;=> {:app/svc 1, :my.domain/name 2, :your.domain/y 3}
Comment by Christophe Grand [ 11/Apr/16 4:03 AM ]

Alex, if existing namespace level aliases are enough when there's more than one namespace used in the key set I fail to understand the real value of this proposal.

Okay I'm lying a little: there are no aliases in edn, so this would bring aliases to edn (and allows printers to factor/alias namespaces out). And for Clojure code you can't define an alias to a non-existing namespace – and I believe that this implementation wouldn't check namespace existence when resolving the default ns #:person{:name}.

Still my points hold for edn (and that's where the value of this proposal seems to be): why not allows local aliases too?

#:person #:employee/e {:name "John Smith", :e/eid "012345"}
;=> {:person/name "John Smith", :employee/eid "012345"}

I have another couple of questions:

  • should it apply to other datatypes?
  • should it be transitive?
Comment by Alex Miller [ 28/Apr/16 1:33 PM ]

New patch rev supports spaces between the namespace part #:foo and the map in both LispReader and EdnReader.

Comment by Tim Gilbert [ 23/Sep/16 1:12 AM ]

Any news on the flag to prevent this behavior (mentioned in the screener notes)?

I ask because (pr-str) in Clojure 1.9.0-alpha12 currently produces EDN that can't be consumed by (cljs.reader/read-string) in ClojureScript 1.9.229 (the ClojureScript consumption side is tracked in CLJS-1706).

It would be nice to have a way to disable this behavior until ClojureScript reaches parity, and a lot of fairly widely-used open-source libraries use plain (pr-str) to generate EDN output - for example, ring-middleware-format.

Comment by Alex Miller [ 23/Sep/16 8:37 AM ]

Yes, that was implemented in a separate ticket (CLJ-1993) in Clojure 1.9.0-alpha11. There is now a print-namespace-maps dynvar. It defaults to false, but is set to true in the standard REPL.

So at the REPL you can (set! print-namespace-maps false) to turn it off.

If you're doing stuff in your program outside a REPL, it will be off by default so you probably don't need to do anything as of alpha11.

Comment by Alex Miller [ 23/Sep/16 8:38 AM ]

that's *print-namespace-maps* above, sorry for the formatting





[CLJ-1909] Using thrown? in exceptions fails without doall Created: 02/Apr/16  Updated: 02/Apr/16  Resolved: 02/Apr/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Shriphani Palakodety Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

OS: OS X and testing using lein test



 Description   

I have added a small example in this repo: https://github.com/shriphani/thrown-test

See the test in https://github.com/shriphani/thrown-test/blob/master/test/thrown_test/core_test.clj

The first assertion fails, the second passes.

The output I get is: https://gist.github.com/shriphani/d9351d062f2f5c211879ef87c13277ac



 Comments   
Comment by Alex Miller [ 02/Apr/16 10:02 AM ]

In the example without doall, map will return a lazy seq that is not realized and thus you never encounter the exception. This is the expected behavior so I am declining the ticket.





[CLJ-1904] clojure.template/apply-template - 'unsupported binding form' when re-binding the input symbols Created: 29/Mar/16  Updated: 29/Mar/16  Resolved: 29/Mar/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: James Henderson Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   
(clojure.template/apply-template '[s]
                                 '(let [s "foo"]
                                    s)
                                 ["s"])

returns

(let ["s" "foo"] 
  "s")

which then fails with Unsupported binding form: s - whereas it seems that it shouldn't replace the binding symbol in this case?

This came up when using clojure.test, as follows:

(t/are [req resp] (= resp
                     (let [handler (-> (fn [{:keys [uri] :as req}]
                                         {:body (str "You requested: " uri)})
                                       middleware-under-test)]

                       (handler req)))
  {:uri "..."} {:status 200, :body "..."})

macro-expands to

(do
  (t/is
   (=
    {:status 200, :body "..."}
    (let [handler (-> (fn [{:keys [uri], :as {:uri "..."}}]
                        {:body (str "You requested: " uri)})
                      middleware-under-test)]
      (handler {:uri "..."})))))

which, in this case, then threw Bad binding form, expected symbol, got: {:uri "..."}.

An easy work-around is to rename/remove the req parameter in the expr, although this seems like it should be a valid use-case?



 Comments   
Comment by Alex Miller [ 29/Mar/16 6:59 AM ]

It seems to me like the problem here is in 'are', not in apply-template, which is just blindly doing what it's been told to do.

Comment by James Henderson [ 29/Mar/16 8:09 AM ]

Sure, the docstring says that it blindly replaces symbols - but doesn't that mean that all of the callers of apply-template/do-template have to take this issue into account? If so, would it be better to fix it here?

If not, no worries - would you like me to file an issue against clojure.test?

Comment by Alex Miller [ 29/Mar/16 9:59 AM ]

I do not think it is reasonable for a generic utility like apply-template to any special-case thing here (esp when the set of cases is open-ended and hard/impossible to detect).

clojure.test/are points to clojure.template for understanding what will be done and apply-template says "will recursively replace argument symbols in expr with their corresponding values". I think what you are seeing is the expected behavior. That is, there are limits to what are templates do and you have exceeded them. The workaround seems pretty simple.

I'm going to decline this as I don't see anything reasonable that needs to change.





[CLJ-1902] Remove overhead of if-not Created: 16/Mar/16  Updated: 16/Mar/16  Resolved: 16/Mar/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Jeroen van Dijk Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: enhancement

Attachments: Text File clj_1902.patch    

 Description   

The `(if-not x a b)` macro expands to `(if (not x) a b`, but it could be more efficient by just expanding to `(if x b a)`

Why is this important? I've always found it more readable to have the biggest condition to be placed second. This allows you to see the different paths easier.

So one would change:

(if x
  (let [...] 
    .
    .
    .
    a)
  b)

To

(if-not x
  b
  (let [...] 
    .
    .
    .
    a))

I think you would agree that the second one is more readable. However currently with `if-not` there is always the tiny performance counter-argument to not doing this.



 Comments   
Comment by Jeroen van Dijk [ 16/Mar/16 5:58 AM ]

The patch doesn't include any new tests as breaking `if-not` already broke the "compile-clojure" tests.

Comment by Alex Miller [ 16/Mar/16 8:11 AM ]

Why do you think there's a performance difference?

Comment by Alex Miller [ 16/Mar/16 8:23 AM ]

Quick benchmark shows about < 1 ns difference.

vecperf.bench=> (bench (if (odd? 1) 1 2))
Evaluation count : 10214445780 in 60 samples of 170240763 calls.
             Execution time mean : 4.215496 ns
    Execution time std-deviation : 0.025472 ns
   Execution time lower quantile : 4.179194 ns ( 2.5%)
   Execution time upper quantile : 4.272295 ns (97.5%)
                   Overhead used : 1.667409 ns
nil
vecperf.bench=> (bench (if (not (odd? 1)) 2 1))
Evaluation count : 9389004780 in 60 samples of 156483413 calls.
             Execution time mean : 4.768709 ns
    Execution time std-deviation : 0.028476 ns
   Execution time lower quantile : 4.721174 ns ( 2.5%)
   Execution time upper quantile : 4.824708 ns (97.5%)
                   Overhead used : 1.667409 ns
Comment by Jeroen van Dijk [ 16/Mar/16 8:47 AM ]

Yeah sounds I'm a bit pedantic if you put it that way. The benchmark is indeed not very convincing. I'm ok if you close this issue.





[CLJ-1894] Include namespace when stringifying keys in maps Created: 22/Feb/16  Updated: 24/Feb/16  Resolved: 24/Feb/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Rafal Szalanski Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: walk
Environment:

OS X, Java 8, Clojure 1.8


Attachments: Text File full-name.patch    
Patch: Code and Test

 Description   

I noticed that if one wants to stringify keys in map using clojure.walk/stringify-keys and the keywords or symbols contain namespaces, for example:

(clojure.walk/stringify-keys {:a 1 :b/c 2})

then the result will be equal to {"a" 1 "c" 2}. The docstring for clojure.walk/stringify-keys says:

Recursively transforms all map keys from keywords to strings.

which to my mind implies the namespace should be included in the result map so that reverse operation clojure.walk/keywordize-keys can re-create the initial map.

The patch I am proposing adds a function full-name to clojure.core namespace which returns full string representation of a keyword including the namespace (if it's present). I also modify clojure.walk/stringify-keys to use that function instead of clojure.core/name.

The change should be 100% compatible with any Clojure code out there. I am making an assumption that people who came up against this problem found a different way of solving that problem, even re-design everything to use keywords instead of strings. Keywords are one of the most commonly used parts of the language and have clear benefits over strings (i.e. they are functions).



 Comments   
Comment by Alex Miller [ 24/Feb/16 1:53 PM ]

I disagree with your assumption re current use - it's possible that the current behavior is desired (or at least relied-upon) by some existing caller. I'm not willing to change this default behavior.

Instead, I would note that stringify-keys and keywordize-keys are the same function with a different transformation. It would be better to extract that more generic function (transform-keys?), refactor stringify-keys and keywordize-keys in terms of it, and let users supply any transformation function they want.

I'm going to decline this ticket as is, as will not make the suggested change. The other idea would be a reasonable alternative ticket. (Although it does risk running into an expansion of purpose to include shallow/deep alternative and value transformations - all things I think are valuable and in common use).





[CLJ-1893] Clojure returns nil as the empty java.util.Map$Entry Created: 13/Feb/16  Updated: 15/Feb/16  Resolved: 13/Feb/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Herwig Hochleitner Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

1.8



 Description   
(empty (first {1 2}))
;; => nil

empty of a map entry should return the empty vector (as in clojurescript), because then the zipper for edn becomes very elegant:

(def coll-zipper (partial zip/zipper coll? seq #(into (empty %1) %2)))


 Comments   
Comment by Alex Miller [ 13/Feb/16 11:11 PM ]

A map entry is considered to be a 2-tuple of key and value (the particular MapEntry class should be considered an implementation detail), where tuple implies ordered and indexed access like a vector, but also a fixed size. The docstring for empty says "Returns an empty collection of the same category as coll, or nil". To me, having a map entry return a vector violates the "same category" language, although because map entry shares many aspects with vectors this is admittedly open to interpretation.

Overall, I think returning nil is more consistent with the ideals behind map entries and empty. Similar arguments were applied to records (as they have known fields) and that's why empty does not work on records. I concede there is some utility to having map entries empty to a vector.

However, I suspect any of these decisions are more likely to shake out in some future when tuples are reconsidered and the MapEntry classes are replaced with tuples. Because of that, I don't think this ticket is going anywhere now and I'm going to decline it.

Comment by Nicola Mometto [ 15/Feb/16 11:12 AM ]

This feels really counter-intuitive given that

Comment by Nicola Mometto [ 15/Feb/16 11:13 AM ]

This feels really counter-intuitive given that

(vector? (first {:a 1}))
returns true and `empty?` on vectors is supposed to return the empty vector





[CLJ-1892] Subtraction floating point numbers error Created: 12/Feb/16  Updated: 12/Feb/16  Resolved: 12/Feb/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Umarov German Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Attachments: PNG File floating bug.png    

 Description   

Subtraction of floating point numbers gives wrong result.
code:
(- 12.1 42.9)
result:
-30.799999999999997

JRE 1.7 patch 25/1.8 patch 72



 Comments   
Comment by Alex Miller [ 12/Feb/16 9:49 AM ]

By default, Clojure floating point numbers are Java doubles internally. Java doubles are IEEE 754 64-bit floating point numbers. Not all floating points can be represented exactly in this format (because you can't squeeze an infinite number of floats into a finite number of bits). Results like this are typical and expected - http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html.

In Clojure, you can get arbitrary precision floating point (Java BigDecimal) by appending an M to the number:

user=> (- 12.1M 42.9M)
-30.8M




[CLJ-1883] references to a symbol ending with ? that came from a foreign namespace confuses the compiler inside a :cljc block Created: 13/Jan/16  Updated: 13/Jan/16  Resolved: 13/Jan/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7, Release 1.8
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Geraldo Lopes de Souza Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Attachments: File bug.cljc     File bug.tar.bz2    

 Description   

I've found a situation where I'm requiring a om.tempid on a .cljc namespace. The require is
[om.tempid :as tempid :refer [tempid?]]
The reference of tempid? on line 24 of bug.cljc generates
java.lang.IllegalArgumentException: Unable to resolve classname: clojure.core$boolean@7521bffd, compiling/home/geraldo/bug/src/bug/bug.cljc:14:5)

To circumvent I've created a local alias of the tempid? function (line 26). It seems that the ? at the end of the name is causing the trouble because if I reference om.tempid/tempid it does not trigger. Note that the local alias was ended with ? to show that only when is a foreign ns that the problem is present.

Regards and forgive my english.

Geraldo Lopes de Souza



 Comments   
Comment by Alex Miller [ 13/Jan/16 8:20 AM ]

tempid? is:

(defn ^boolean tempid? [x]
  (instance? TempId x))

I suspect the ? has nothing to do with it and it's the ^boolean type hint that's the issue.

Comment by Alex Miller [ 13/Jan/16 8:27 AM ]

This is a bug in Om in using a type hint in non-conditional code that is cljs-specific. David Nolen is fixing in Om.

Comment by Alex Miller [ 13/Jan/16 8:27 AM ]

https://github.com/omcljs/om/commit/cc39f37561455a54153aaef7e5ca36782839aa38

Comment by Geraldo Lopes de Souza [ 13/Jan/16 4:38 PM ]

Alex,

I saw the other issue referencing the ^boolean but I didn't relate with this.

Thank you,

Geraldo Lopes de Souza





[CLJ-1878] destructuring doesn't work on a sorted set Created: 08/Jan/16  Updated: 08/Jan/16  Resolved: 08/Jan/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Anton Gladyshevskiy Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: collections


 Description   

Destructuring doesn't work on a sorted set:

(let [[head :as demoset] (sorted-set 1 2 3)] head)

UnsupportedOperationException nth not supported on this type: PersistentTreeSet clojure.lang.RT.nthFrom (RT.java:933)



 Comments   
Comment by Alex Miller [ 08/Jan/16 4:40 PM ]

Sequential destructuring works on sequential collections - sets (even sorted sets) are unordered (not sequential).

You can make this work with something like (which defines a sequential collection from the set):

(let [[head :as demoset] (seq (sorted-set 1 2 3))] head)
Comment by Anton Gladyshevskiy [ 08/Jan/16 4:57 PM ]

Thank you. It's a pity, because in this case 'demoset' becomes a sequence, not a sorted set.





[CLJ-1877] operation on the sorted-set fails under certain conditions Created: 08/Jan/16  Updated: 08/Jan/16  Resolved: 08/Jan/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Anton Gladyshevskiy Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: collections
Environment:

Windows 8.1



 Description   

Function generates a sequence of prime numbers. Uses a priority queue implemented as a sorted set of vectors [priority val].

On the iteration (x = 10) fails with message:

(defn sieve [[x & t] pq]
  (lazy-seq
    (if (or (empty? pq) (< x (ffirst pq)))
      (cons x (sieve t (conj pq [(* x x) (next (iterate (partial + x) (* x x)))])))
      (sieve t
        (loop [pq pq] (let [[key val :as head] (first pq)]
          (if (= x key) (recur (conj (disj pq head) [(first val) (next val)])) pq) ))))))

(def primes (sieve (iterate inc 2) (sorted-set)))

(take 4 primes)
;; => (2 3 5 7)

(take 5 primes)
ClassCastException clojure.lang.Iterate cannot be cast to java.lang.Comparable  clojure.lang.Util.compare (Util.java:153)

When x = 10 it goes to the (loop ...) section and fails while trying to recur.



 Comments   
Comment by Alex Miller [ 08/Jan/16 4:45 PM ]

Sets are unordered (not sequential) and cannot be used with sequential destructuring. You can wrap (seq ) around the conj in the recur if you wish to make it sequential.

Comment by Anton Gladyshevskiy [ 08/Jan/16 4:55 PM ]

Destructuring is not a subject of this issue. The problem is that the code that have been successfully evaluated several times before fails on the subsequent iteration. I.e. when x is 4, 6, 8, 9 it works fine, but when x = 10 it fails.

Comment by Alex Miller [ 08/Jan/16 8:18 PM ]

Oh, yeah! I totally misread this one after reading the last one. The issue here is that you're trying to put something into a sorted set, but the type of the item (here the sequence produced by `iterate`) is not comparable to the existing items in the set (vectors). Vectors are comparable, but not general sequences.

A simpler example of the same issue is: `(conj (sorted-set) [1] '(2))`

There is a ticket for making lists comparable (CLJ-1467), but I'm not sure it would be a good idea to generically extend this to all sequential data types.





[CLJ-1870] Reloading a defmulti nukes metadata on the var Created: 22/Dec/15  Updated: 19/Aug/16  Resolved: 19/Aug/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Release 1.9

Type: Defect Priority: Critical
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 1
Labels: ft, metadata, multimethods

Attachments: Text File 0001-CLJ-1870-don-t-destroy-defmulti-metadata-on-reload.patch    
Patch: Code
Approval: Ok

 Description   

Reloading a defmulti expression destroys any existing (or new) metadata on that multimethod's var:

user=> (defmulti foo "docstring" :op)
#'user/foo
user=> (-> #'foo meta :doc)
"docstring"
user=> (defmulti foo "docstring" :op)
nil
user=> (-> #'foo meta :doc)
nil

This is highly problematic for tools.analyzer, since it relies on such metadata to convey information to the pass scheduler about pass dependencies.

This means that any code that uses core.async cannot be reloaded using `require :reload-all`, since it will cause tools.analyzer to reload and the passes to scheduled in a random order. See ASYNC-154 for one example.

Cause: defmulti has defonce semantics and the first def does not re-apply meta.

Approach: Re-apply meta before first def.

After patch:

user=> (defmulti foo "docstring" :op)
#'user/foo
user=> (-> #'foo meta :doc)
"docstring"
user=> (defmulti foo "docstring" :op)
nil
user=> (-> #'foo meta :doc)
"docstring"

Patch: 0001-CLJ-1870-don-t-destroy-defmulti-metadata-on-reload.patch

Screened by: Alex Miller



 Comments   
Comment by Alex Miller [ 23/Dec/15 10:16 AM ]

Related: CLJ-900 CLJ-1446

Comment by Alex Miller [ 23/Dec/15 10:42 AM ]

This patch doesn't compile for me: RuntimeException: Unable to resolve symbol: mm in this context, compiling:(clojure/core.clj:1679:17)

Also, please build the patch with more context - use -U10 with git format-patch to expand it.

Comment by Nicola Mometto [ 23/Dec/15 11:06 AM ]

Updated patch fixing the typo & using -U10

Comment by Alex Miller [ 23/Dec/15 11:17 AM ]

Now:

java.lang.RuntimeException: Too many arguments to def, compiling:(clojure/core.clj:3561:1)

Comment by Nicola Mometto [ 23/Dec/15 11:22 AM ]

whoops. Sorry for this, here's the updated (and working) patch

Comment by Alex Miller [ 19/Aug/16 10:58 AM ]

patch doesn't apply with new def spec, needs another look

Comment by Alex Miller [ 19/Aug/16 11:16 AM ]

I had an old version of the patch locally - nvm!





[CLJ-1869] EdnReader allows keywords starting with number Created: 18/Dec/15  Updated: 18/Dec/15  Resolved: 18/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Greg Chapman Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

CLJ-1252 pointed out this problem with the LispReader, but the patch there was rejected because it broke too many things. The same problem exists with the EdnReader, but I would guess that patching it would not cause as much breakage. I suggest applying the CLJ-1252 patch to EdnReader.



 Comments   
Comment by Greg Chapman [ 18/Dec/15 2:35 PM ]

I apologize: for some reason I didn't notice that CLJ-1252 patched EdnReader as well as LispReader. So maybe changing EdnReader did cause breakage. If so, please ignore this report.

Comment by Alex Miller [ 18/Dec/15 2:53 PM ]

At this point, I believe we are likely to continue allowing keywords that start with a number. There is another ticket to sync up the spec with code (and actually it would be nice to fix the regex so it worked intentionally rather than accidentally).





[CLJ-1868] Unknown return type class throws NPE instead of useful exception Created: 17/Dec/15  Updated: 08/Jan/16  Resolved: 08/Jan/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: compiler, errormsgs, regression

Attachments: Text File clj-1868.patch    
Patch: Code and Test
Approval: Ok

 Description   

This is a regression from CLJ-1232 - if you specify a return type class that is not fully-qualified or imported, you now get an NPE instead of a useful error message.

;; 1.7
CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: Closeable, compiling:(NO_SOURCE_PATH:4:1)

;; 1.8
(defn foo ^Closeable [])
NullPointerException   clojure.core/sigs/resolve-tag--4375 (core.clj:247)

Cause: The new code that resolves classes does not handle the possible null return value of Compiler$HostExpr/maybeClass.

Solution: Check for null and fall back to the original argvecs, which will result in the original message.

Patch: clj-1868.patch



 Comments   
Comment by Nicola Mometto [ 17/Dec/15 5:10 PM ]

Dupe of CLJ-1863





[CLJ-1861] Remove unnecessary var interning Created: 02/Dec/15  Updated: 16/Dec/15  Resolved: 16/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Rich Hickey
Resolution: Completed Votes: 1
Labels: compiler

Approval: Vetted

 Description   

Remove unused var interns in static initializer (see https://groups.google.com/d/msg/clojure/731p1NBy2wk/lx98zp9oAAAJ ):

L1 {
             ldc "clojure.core" (java.lang.String)
             ldc "float?" (java.lang.String)
             invokestatic clojure/lang/RT var((Ljava/lang/String;Ljava/lang/String;)Lclojure/lang/Var;);
             checkcast clojure/lang/Var
             putstatic malt/utils$string_to_double.const__0:clojure.lang.Var
             ...
}


 Comments   
Comment by Nicola Mometto [ 02/Dec/15 9:49 AM ]

the compiler emits a lot of unused bytecode in its static initializer, not only vars.
The constant table is also full of unused numbers/keywords

Comment by Alex Miller [ 16/Dec/15 3:10 PM ]

resolved by Rich directly in https://github.com/clojure/clojure/commit/bfe14aec1c223abc3253358bac34b503284467d9

Comment by Alex Miller [ 16/Dec/15 3:33 PM ]

1.8.0-RC4





[CLJ-1856] Direct-linking breaks clojure.test do-report stack-depth assumptions for failure reporting. Created: 24/Nov/15  Updated: 05/Dec/15  Resolved: 05/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Gary Trakhman Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: regression, test

Attachments: Text File clj-1856.patch    
Patch: Code
Approval: Ok

 Description   

Test failure locations are being mis-reported (wrong class/line number).

Given a test ns:

(ns test1856.core-test
  (:require [clojure.test :refer :all]
            [test1856.core :refer :all]))

(deftest fail-test
  ;; 6
  ;; 7
  (is (= nil true)))  ;; 8

The results with 1.8-RC2 (with CLJ-1854 patch included) are:

FAIL in (fail-test) (test.clj:342)    ;; WRONG, expected: (core_test.clj:8)
expected: (= nil true)
  actual: (not (= nil true))

Cause: The location of the error is calculated in test.clj by constructing a Throwable in do-report and then dropping the top 1 frame (which is do-report itself) to find the user frame where the assertion failed. However, with direct linking there are now 2 frames on top of of the failure in user code, so the same (incorrect) location in test.clj is reported every time.

Approach: Change to a different strategy to filter the top frames based on content, not hard-coded depth. This strategy will work regardless of whether direct linking is involved or not.

1. Instead of constructing an exception and using it's stack trace, instead call Thread.getStackTrace() on the current thread. Create a new function that works on stack traces rather than exceptions and no longer needs a depth check.
2. Drop top frames while their class name starts with java.lang (this filters the call to java.lang.Thread.getStackTrace()) or clojure.test. The top frame will then be in the user's code.
3. Deprecated old file-and-line function (not sure if other clojure test reporting frameworks use this, despite it being private).
4. Updated tests that check that these functions work with an empty stack trace, as the JVM may elide it.

Patch: clj-1856.patch



 Comments   
Comment by Gary Trakhman [ 24/Nov/15 4:35 PM ]

'2' works in the standard case of direct-linked clojure with non-direct-linked app code. I'm exploring if there's a way to get the line info via macro-expansion of 'try-expr' and passing the line value into do-report for those cases.

Comment by Gary Trakhman [ 24/Nov/15 4:54 PM ]

I altered a local clojure build to print the stacktrace of the Throwable created by do-report, showing the additional invokeStatic frames during a repl session, showing the first user function frame is at offset 2.

user> (use 'clojure.test)
nil
user> (deftest a []
        (is false))
#'user/a
user> (run-tests)

Testing user

FAIL in (a) (test.clj:342)
expected: false
  actual: false

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}


java.lang.Throwable
    at clojure.test$do_report.invokeStatic(test.clj:355)
    at clojure.test$do_report.invoke(test.clj:342)
    at user$fn__16867.invokeStatic(NO_SOURCE_FILE:74)
    at user$fn__16867.invoke(NO_SOURCE_FILE:73)
    at clojure.test$test_var$fn__7973.invoke(test.clj:706)
    at clojure.test$test_var.invokeStatic(test.clj:706)
    at clojure.test$test_var.invoke(test.clj:697)
    at clojure.test$test_vars$fn__7995$fn__8000.invoke(test.clj:724)
    at clojure.test$default_fixture.invokeStatic(test.clj:676)
    at clojure.test$default_fixture.invoke(test.clj:672)
    at clojure.test$test_vars$fn__7995.invoke(test.clj:724)
    at clojure.test$default_fixture.invokeStatic(test.clj:676)
    at clojure.test$default_fixture.invoke(test.clj:672)
    at clojure.test$test_vars.invokeStatic(test.clj:720)
    at clojure.test$test_all_vars.invokeStatic(test.clj:726)
    at clojure.test$test_ns.invokeStatic(test.clj:747)
    at clojure.test$test_ns.invoke(test.clj:732)
    at clojure.core$map$fn__4781.invoke(core.clj:2648)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.boundedLength(RT.java:1749)
    at clojure.lang.RestFn.applyTo(RestFn.java:130)
    at clojure.core$apply.invokeStatic(core.clj:647)
    at clojure.test$run_tests.invokeStatic(test.clj:757)
    at clojure.test$run_tests.doInvoke(test.clj:757)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.test$run_tests.invokeStatic(test.clj:762)
    at clojure.test$run_tests.invoke(test.clj:757)
    at user$eval16871.invokeStatic(NO_SOURCE_FILE:76)
    at user$eval16871.invoke(NO_SOURCE_FILE:76)
    at clojure.lang.Compiler.eval(Compiler.java:6915)
    at clojure.lang.Compiler.eval(Compiler.java:6878)
    at clojure.core$eval.invokeStatic(core.clj:3107)
    at clojure.core$eval.invoke(core.clj:3103)
    at clojure.main$repl$read_eval_print__7402$fn__7405.invoke(main.clj:240)
    at clojure.main$repl$read_eval_print__7402.invoke(main.clj:240)
    at clojure.main$repl$fn__7411.invoke(main.clj:258)
    at clojure.main$repl.invokeStatic(main.clj:258)
    at clojure.main$repl.doInvoke(main.clj:174)
    at clojure.lang.RestFn.invoke(RestFn.java:1523)
    at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__10251.invoke(interruptible_eval.clj:87)
    at clojure.lang.AFn.applyToHelper(AFn.java:152)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invokeStatic(core.clj:645)
    at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1883)
    at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1883)
    at clojure.lang.RestFn.invoke(RestFn.java:425)
    at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:85)
    at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55)
    at clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__10296$fn__10299.invoke(interruptible_eval.clj:222)
    at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__10291.invoke(interruptible_eval.clj:190)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

This is also with the patch from http://dev.clojure.org/jira/browse/CLJ-1854 applied

Comment by Alex Miller [ 25/Nov/15 1:26 PM ]
(ns test1856.core-test
  (:require [clojure.test :refer :all]
            [test1856.core :refer :all]))

(deftest throw-test
  ;; 6
  ;; 7
  (is (= nil (throw (Exception. "abc")))))  ;; 8

(deftest fail-test
  ;; 11
  ;; 12
  (is (= nil true)))  ;; 13

If I lein test (or run from repl), I see:

;; 1.7
FAIL in (fail-test) (core_test.clj:13)
expected: (= nil true)
  actual: (not (= nil true))

ERROR in (throw-test) (core_test.clj:8)
expected: (= nil (throw (Exception. "abc")))
  actual: java.lang.Exception: abc
 at test1856.core_test/fn (core_test.clj:8)
    clojure.test$test_var$fn__7670.invoke (test.clj:704)
    clojure.test$test_var.invoke (test.clj:704)
    clojure.test$test_vars$fn__7692$fn__7697.invoke (test.clj:722)
    ...
;; 1.8.0-RC2 + CLJ-1854 patch
FAIL in (fail-test) (test.clj:342)    ;; ERROR
expected: (= nil true)
  actual: (not (= nil true))

ERROR in (throw-test) (core_test.clj:8)
expected: (= nil (throw (Exception. "abc")))
  actual: java.lang.Exception: abc
 at test1856.core_test$fn__201.invokeStatic (core_test.clj:8)  ;; OK
    test1856.core_test/fn (core_test.clj:5)
    clojure.test$test_var$fn__7972.invoke (test.clj:703)
    clojure.test$test_var.invokeStatic (test.clj:703)




[CLJ-1855] Add a boolean? function Created: 24/Nov/15  Updated: 24/Nov/15  Resolved: 24/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Description   

It could be handy to have a boolean? function in core to match the checks for most other primitive Clojure types. Is this something that a patch would be considered for



 Comments   
Comment by Alex Miller [ 24/Nov/15 3:47 PM ]

Yes, although there is a bigger ticket (CLJ-1298) with suggestions for a number of type-related predicates. I would prefer to pass that list by Rich and have him yes/no things on it first prior to getting many individual patches.

boolean? is mentioned in the comments, but feel free to add it to the description to make that more prominent.

I'm going to dupe this to that one.

Comment by Daniel Compton [ 24/Nov/15 3:54 PM ]

Sorry about that. I searched for boolean? in JIRA and it didn't match any tickets so I thought this was a new request.





[CLJ-1854] Direct-linking changes lose line-number on invoke() Created: 24/Nov/15  Updated: 01/Dec/15  Resolved: 01/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Gary Trakhman Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: compiler, regression
Environment:

Clojure 1.8RC2, leiningen 2.5.1


Attachments: Text File CLJ-1854-more-context.patch     Text File CLJ-1854.patch    
Patch: Code
Approval: Ok

 Description   
(ns foo)  ;; 1
;; 2
;; 3
;; 4
;; 5
;; 6
;; 7
(defn callstack []                       ;; 8
  [1 2 3]                                ;; 9
  (throw (Exception. "whoopsie!")))      ;; 10

Stack Trace comparison. Only the first two lines of each trace are relevant, the rest is all REPL fluff.

;;; 1.7.0
{:cause "whoopsie!",
 :via
 [{:type java.lang.Exception,
   :message "whoopsie!",
   :at [foo$callstack invoke "foo.clj" 8]}],
 :trace
 [[foo$callstack invoke "foo.clj" 8]
  [user$eval7675 invoke "form-init3342294504880003721.clj" 1]
  [clojure.lang.Compiler eval "Compiler.java" 6782]
  [clojure.lang.Compiler eval "Compiler.java" 6745]
	...

;;; 1.8 RC2
{:cause "whoopsie!",
 :via
 [{:type java.lang.Exception,
   :message "whoopsie!",
   :at [foo$callstack invokeStatic "foo.clj" 8]}],
 :trace
 [[foo$callstack invokeStatic "foo.clj" 8]
  [foo$callstack invoke "foo.clj" -1]    ;; Unexpected: -1
  [user$eval4 invokeStatic "NO_SOURCE_FILE" 28]
  [user$eval4 invoke "NO_SOURCE_FILE" -1]    ;; Unexpected: -1
  [clojure.lang.Compiler eval "Compiler.java" 6913]
  [clojure.lang.Compiler eval "Compiler.java" 6876]
	...

;;; 1.8 RC2 with patch
{:cause "whoopsie!",
 :via
 [{:type java.lang.Exception,
   :message "whoopsie!",
   :at [foo$callstack invokeStatic "foo.clj" 8]}],
 :trace
 [[foo$callstack invokeStatic "foo.clj" 8]
  [foo$callstack invoke "foo.clj" 8]    ;; Fixed
  [user$eval4 invokeStatic "NO_SOURCE_FILE" 3]
  [user$eval4 invoke "NO_SOURCE_FILE" 3]   ;; Fixed
  [clojure.lang.Compiler eval "Compiler.java" 6917]
  [clojure.lang.Compiler eval "Compiler.java" 6880]
	...

Cause: Non-direct linking now calls from invoke() through to invokeStatic(). In invoke(), Compiler does not visitLineNumber() before invoke() calls invokeStatic(), meaning that stack traces end up with -1 instead of a useful line number.

Patch: CLJ-1854-more-context.patch

Screened by: Alex Miller



 Comments   
Comment by Andy Fingerhut [ 24/Nov/15 1:18 PM ]

I have tested with Clojure 1.8.0-RC2 plus patch CLJ-1854.patch, and it does cause all of the -1 line numbers I have seen in stack traces to be filled in with actual source code line numbers.

For an example see this output after the patch: https://github.com/jafingerhut/st/blob/master/out-clj18-rc2-plus-clj1854-patch.txt

compared to this output with unmodified Clojure 1.8.0-RC2: https://github.com/jafingerhut/st/blob/master/out-clj18-rc2.txt

Comment by Alex Miller [ 24/Nov/15 1:30 PM ]

Ghadi, can you re-make the patch with more lines of diff context (use -U15 on format-patch)?

Comment by Ghadi Shayban [ 24/Nov/15 1:54 PM ]

np. -U15 wasn't enough, used -U30

Comment by Alex Miller [ 24/Nov/15 2:18 PM ]

Does it make sense for the two frames for the invoke and invokeStatic to refer to different line numbers in the source?

Comment by Ghadi Shayban [ 24/Nov/15 3:44 PM ]

Example has recursion through walk and is not minimal. Editing the ticket for reproducibility.

Comment by Gary Trakhman [ 24/Nov/15 3:47 PM ]

The current CLJ-1854-more-context.patch just gives me the same line number for all test cases, in my case it's test.clj:342 instead of -1 from before. I think perhaps clojure.test might need an adjustment as well, in particular the hardcoded '1' magic number here: https://github.com/clojure/clojure/blob/master/src/clj/clojure/test.clj#L351

test.clj:342 is do-report: https://github.com/clojure/clojure/blob/master/src/clj/clojure/test.clj#L342

Comment by Alex Miller [ 24/Nov/15 3:56 PM ]

Gary Trakhman I think that issue should be a separate ticket if so.

Comment by Gary Trakhman [ 24/Nov/15 4:03 PM ]

I will make a separate ticket for the potential clojure.test change.

Comment by Gary Trakhman [ 24/Nov/15 5:20 PM ]

Comparison of line numbers between 1.7 and 1.8 with patch here applied, clojure.test/do-report was modified to print stacktraces. It's weird that the numbers are different between parallel invoke/invokeStatic pairs.

diff --git a/src/clj/clojure/test.clj b/src/clj/clojure/test.clj
index 55e00f7..318ef20 100644
--- a/src/clj/clojure/test.clj
+++ b/src/clj/clojure/test.clj
@@ -349,7 +349,10 @@
   (report
    (case
     (:type m)
-    :fail (merge (file-and-line (new java.lang.Throwable) 1) m)
+     :fail (merge (file-and-line (doto (new java.lang.Throwable)
+                                   (.printStackTrace))
+                                 1)
+                  m)
     :error (merge (file-and-line (:actual m) 0) m) 
     m)))

1.7

gary@gary-dell:~/.m2/repository/org/clojure/clojure/1.7.0$ java -jar clojure-1.7.0.jar -r
Clojure 1.7.0
user=> (use 'clojure.test)
nil
user=> (deftest a []
(is false))
#'user/a
user=> (run-tests)

Testing user
java.lang.Throwable
    at clojure.test$do_report.invoke(test.clj:352)
    at user$fn__3.invoke(NO_SOURCE_FILE:3)
    at clojure.test$test_var$fn__7671.invoke(test.clj:707)
    at clojure.test$test_var.invoke(test.clj:707)
    at clojure.test$test_vars$fn__7693$fn__7698.invoke(test.clj:725)
    at clojure.test$default_fixture.invoke(test.clj:677)
    at clojure.test$test_vars$fn__7693.invoke(test.clj:725)
    at clojure.test$default_fixture.invoke(test.clj:677)
    at clojure.test$test_vars.invoke(test.clj:721)
    at clojure.test$test_all_vars.invoke(test.clj:731)
    at clojure.test$test_ns.invoke(test.clj:750)
    at clojure.core$map$fn__4553.invoke(core.clj:2624)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.boundedLength(RT.java:1735)
    at clojure.lang.RestFn.applyTo(RestFn.java:130)
    at clojure.core$apply.invoke(core.clj:632)
    at clojure.test$run_tests.doInvoke(test.clj:765)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.test$run_tests.invoke(test.clj:763)
    at user$eval7.invoke(NO_SOURCE_FILE:4)
    at clojure.lang.Compiler.eval(Compiler.java:6782)
    at clojure.lang.Compiler.eval(Compiler.java:6745)
    at clojure.core$eval.invoke(core.clj:3081)
    at clojure.main$repl$read_eval_print__7099$fn__7102.invoke(main.clj:240)
    at clojure.main$repl$read_eval_print__7099.invoke(main.clj:240)
    at clojure.main$repl$fn__7108.invoke(main.clj:258)
    at clojure.main$repl.doInvoke(main.clj:258)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.main$repl_opt.invoke(main.clj:324)
    at clojure.main$main.doInvoke(main.clj:421)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.lang.Var.invoke(Var.java:379)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)

FAIL in (a) (NO_SOURCE_FILE:3)
expected: false
  actual: false

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}

1.8 with patch here applied

gary@gary-dell:~/.m2/repository/org/clojure/clojure/1.8.0-master-SNAPSHOT$ java -jar clojure-1.8.0-master-SNAPSHOT.jar -r
Clojure 1.8.0-master-SNAPSHOT
user=> (use 'clojure.test)
nil
user=> (deftest a []
(is false))
#'user/a
user=> (run-tests)

Testing user
java.lang.Throwable
    at clojure.test$do_report.invokeStatic(test.clj:355)
    at clojure.test$do_report.invoke(test.clj:342)
    at user$fn__3.invokeStatic(NO_SOURCE_FILE:3)
    at user$fn__3.invoke(NO_SOURCE_FILE:2)
    at clojure.test$test_var$fn__7973.invoke(test.clj:706)
    at clojure.test$test_var.invokeStatic(test.clj:706)
    at clojure.test$test_var.invoke(test.clj:697)
    at clojure.test$test_vars$fn__7995$fn__8000.invoke(test.clj:724)
    at clojure.test$default_fixture.invokeStatic(test.clj:676)
    at clojure.test$default_fixture.invoke(test.clj:672)
    at clojure.test$test_vars$fn__7995.invoke(test.clj:724)
    at clojure.test$default_fixture.invokeStatic(test.clj:676)
    at clojure.test$default_fixture.invoke(test.clj:672)
    at clojure.test$test_vars.invokeStatic(test.clj:720)
    at clojure.test$test_all_vars.invokeStatic(test.clj:726)
    at clojure.test$test_ns.invokeStatic(test.clj:747)
    at clojure.test$test_ns.invoke(test.clj:732)
    at clojure.core$map$fn__4781.invoke(core.clj:2648)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.boundedLength(RT.java:1749)
    at clojure.lang.RestFn.applyTo(RestFn.java:130)
    at clojure.core$apply.invokeStatic(core.clj:647)
    at clojure.test$run_tests.invokeStatic(test.clj:757)
    at clojure.test$run_tests.doInvoke(test.clj:757)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.test$run_tests.invokeStatic(test.clj:762)
    at clojure.test$run_tests.invoke(test.clj:757)
    at user$eval7.invokeStatic(NO_SOURCE_FILE:4)
    at user$eval7.invoke(NO_SOURCE_FILE:4)
    at clojure.lang.Compiler.eval(Compiler.java:6915)
    at clojure.lang.Compiler.eval(Compiler.java:6878)
    at clojure.core$eval.invokeStatic(core.clj:3107)
    at clojure.core$eval.invoke(core.clj:3103)
    at clojure.main$repl$read_eval_print__7402$fn__7405.invoke(main.clj:240)
    at clojure.main$repl$read_eval_print__7402.invoke(main.clj:240)
    at clojure.main$repl$fn__7411.invoke(main.clj:258)
    at clojure.main$repl.invokeStatic(main.clj:258)
    at clojure.main$repl_opt.invokeStatic(main.clj:322)
    at clojure.main$repl_opt.invoke(main.clj:318)
    at clojure.main$main.invokeStatic(main.clj:421)
    at clojure.main$main.doInvoke(main.clj:384)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.lang.Var.invoke(Var.java:379)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)

FAIL in (a) (test.clj:342)
expected: false
  actual: false

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}
user=>




[CLJ-1853] Socket server can't use user-defined accept-fns Created: 24/Nov/15  Updated: 01/Dec/15  Resolved: 01/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: OHTA Shogo Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: server

Attachments: Text File 0001-CLJ-1853-Require-the-ns-before-resolving-the-accept-.patch    
Patch: Code
Approval: Ok

 Description   

In 1.8.0 RC2, if you start a socket server with a user-defined accept-fn like the following (clojure.test/testing-contexts-str is just a 0-arity fn used as an example accept fn here):

$ java -cp clojure-1.8.0-RC2.jar -Dclojure.server.foo='{:port 5555 :accept clojure.test/testing-contexts-str}' clojure.main

And then, if you connect to it with a command like "telnet 127.0.0.1 5555", you'll get an NPE.

Clojure 1.8.0-RC2
user=> Exception in thread "Clojure Connection repl 1" java.lang.NullPointerException
        at clojure.core$apply.invokeStatic(core.clj:645)
        at clojure.core.server$accept_connection.invokeStatic(server.clj:60)
        at clojure.core.server$start_server$fn__7327$fn__7328$fn__7330.invoke(server.clj:104)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.lang.Thread.run(Thread.java:745)

This doesn't happen when starting the server with a pre-defined accept-fn, such as clojure.core.server/repl.

Cause: At the moment, clojure.core.server/accept-connection will require the namespace in which the accept-fn is defined after resolving the accept-fn. However, in order to resolve the accept-fn successfully, requiring the ns should be done prior to it.

Approach: Requiring the ns before resolving the accept-fn.

Patch: 0001-CLJ-1853-Require-the-ns-before-resolving-the-accept-.patch

Screened by: Alex Miller






[CLJ-1851] Add :redef key for vars to avoid being direct linked Created: 20/Nov/15  Updated: 05/Dec/15  Resolved: 05/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File clj-1851.patch    
Patch: Code
Approval: Ok

 Description   

It is useful in some cases to indicate that calls to a var should never be direct linked. That is possible with ^:dynamic but that has additional semantics (and cost). Add a new ^:redef meta for vars that prevents direct invocations but does not have the ^:dynamic semantics.

From CLJ-1845, load was marked dynamic for this reason, now change to redef instead.

Patch: clj-1851.patch (also changes load to be :redef rather than :dynamic)






[CLJ-1850] doseq expansion causes boxed math warning. Created: 13/Nov/15  Updated: 13/Nov/15  Resolved: 13/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6, Release 1.7
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Michael Nygard Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: math


 Description   

When boxed math warnings are enabled, doseq causes a warning.



 Comments   
Comment by Alex Miller [ 13/Nov/15 2:27 PM ]

Example?

Comment by Michael Nygard [ 13/Nov/15 2:58 PM ]

Working on isolating a minimal example.

Comment by Michael Nygard [ 13/Nov/15 4:46 PM ]

With this source:

src/doseq_warning/core.clj
(ns doseq-warning.core
  (:require [clojure.core.async :as async]))

(set! *unchecked-math* :warn-on-boxed)

(defn example
  []
  (async/go-loop [actives []]
    (let [vch (async/alts! actives)]
      (doseq [c (second vch)]
        (async/close! c)))))

Using the following project.clj:

project.clj
(defproject doseq-warning "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.8.0-beta1"]
                 [org.clojure/core.async "0.1.346.0-17112a-alpha"]]
  :global-vars {*unchecked-math* :warn-on-boxed})

Then executing (load-file "src/doseq_warning/core.clj") at a REPL results in:

Boxed math warning, /Users/mtnygard/tmp/doseq-warning/src/doseq_warning/core.clj:8:3 - call: public static boolean clojure.lang.Numbers.lt(java.lang.Object,java.lang.Object). 
Boxed math warning, /Users/mtnygard/tmp/doseq-warning/src/doseq_warning/core.clj:8:3 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_inc(java.lang.Object). 
#'doseq-warning.core/example
Comment by Alex Miller [ 13/Nov/15 4:49 PM ]

I think it's probably unlikely that this is an error with the boxed math warnings and more likely an artifact of using an older core.async with older tools.analyzer in the go block transformation.

Not reproducible with latest core.async, so closing.





[CLJ-1849] Add tests for CLJ-1846 and CLJ-1825 Created: 13/Nov/15  Updated: 16/Nov/15  Resolved: 16/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File clj-1849.patch    
Patch: Code and Test
Approval: Ok

 Description   

Add tests to reproduce CLJ-1846 and CLJ-1825 errors for future testing.






[CLJ-1847] clojure.walk/walk returns a PersistentVector when the input is of type IMapEntry Created: 13/Nov/15  Updated: 15/Nov/15  Resolved: 15/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Yehonathan Sharvit Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: walk

Attachments: Text File fix_walk.patch    
Patch: Code

 Description   

`walk` is supposed to build up a data structure of the same type as its input.
With `clojure.lang.IMapEntry`, it doesn't work as expected.
The output is of type: `clojure.lang.PersisentVector`

For instance,
(class (walk/walk identity identity (first {:a 1})) ); clojure.lang.PersisentVector



 Comments   
Comment by Alex Miller [ 15/Nov/15 12:54 PM ]

Thanks for filing this. I spent some time looking at it and I don't think there is anything that needs to be done at this time.

Re your description, "`walk` is supposed to build up a data structure of the same type as its input." Regarding IMapEntry.... prior to 1.8, IMapEntry concrete types were descendants of AMapEntry, in particular the most common concrete type was MapEntry, which were used in PersistentHashMap.

In 1.8, PHM now returns PersistentVectors, which now implement IMapEntry. So, walk now takes a map entry (which is a PV) and returns a PV, which is an IMapEntry. So I believe the contract is still satisfied.

The new map-entry? predicate can be used to catch only 2-element PVs (not other counts) as map entries, however it is important to still consider whether you will incorrectly snare 2 element vectors that aren't in a map in this particular case. The current clojure.walk code however will have the same effective result in either case so the clojure.walk code does not need to change (and in fact non-PV entries still exist in sorted maps).





[CLJ-1846] VerifyError in Clojure 1.8.0-(beta1..RC1) Created: 11/Nov/15  Updated: 11/Nov/15  Resolved: 11/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Shantanu Kumar Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: compiler, regression
Environment:

Oracle JDK 1.7, Oracle JDK 1.8 on Mac OS X


Approval: Vetted

 Description   

Nicola Mometto provided the below minimal repro case:

user=> (defn foo ^long [] 1)
#'user/foo
user=> (Integer/bitCount ^int (foo))
VerifyError (class: user$eval13, method: invokeStatic signature: ()Ljava/lang/Object;) Expecting to find integer on stack  java.lang.Class.getDeclaredConstructors0 (Class.java:-2)

Full stack trace as found with https://github.com/kumarshantanu/asphalt:

$ lein do clean, with-profile dev,c18 test
Exception in thread "main" java.lang.VerifyError: (class: asphalt/core$invoke_with_transaction, method: invokeStatic signature: (Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;) Expecting to find integer on stack, compiling:(core.clj:201:1)
	at clojure.lang.Compiler$DefExpr.eval(Compiler.java:463)
	at clojure.lang.Compiler.eval(Compiler.java:6918)
	at clojure.lang.Compiler.load(Compiler.java:7360)
	at clojure.lang.RT.loadResourceScript(RT.java:372)
	at clojure.lang.RT.loadResourceScript(RT.java:363)
	at clojure.lang.RT.load(RT.java:453)
	at clojure.lang.RT.load(RT.java:419)
	at clojure.core$load$fn__5673.invoke(core.clj:5895)
	at clojure.core$load.invokeStatic(core.clj:5894)
	at clojure.core$load_one.invokeStatic(core.clj:5694)
	at clojure.core$load_one.invoke(core.clj)
	at clojure.core$load_lib$fn__5622.invoke(core.clj:5739)
	at clojure.core$load_lib.invokeStatic(core.clj:5738)
	at clojure.core$load_lib.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$load_libs.invokeStatic(core.clj:5776)
	at clojure.core$load_libs.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$require.invokeStatic(core.clj:5798)
	at clojure.core$require.doInvoke(core.clj)
	at clojure.lang.RestFn.invoke(RestFn.java:457)
	at asphalt.test_util$eval198$loading__5565__auto____199.invoke(test_util.clj:1)
	at asphalt.test_util$eval198.invokeStatic(test_util.clj:1)
	at asphalt.test_util$eval198.invoke(test_util.clj)
	at clojure.lang.Compiler.eval(Compiler.java:6913)
	at clojure.lang.Compiler.eval(Compiler.java:6902)
	at clojure.lang.Compiler.load(Compiler.java:7360)
	at clojure.lang.RT.loadResourceScript(RT.java:372)
	at clojure.lang.RT.loadResourceScript(RT.java:363)
	at clojure.lang.RT.load(RT.java:453)
	at clojure.lang.RT.load(RT.java:419)
	at clojure.core$load$fn__5673.invoke(core.clj:5895)
	at clojure.core$load.invokeStatic(core.clj:5894)
	at clojure.core$load_one.invokeStatic(core.clj:5694)
	at clojure.core$load_one.invoke(core.clj)
	at clojure.core$load_lib$fn__5622.invoke(core.clj:5739)
	at clojure.core$load_lib.invokeStatic(core.clj:5738)
	at clojure.core$load_lib.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$load_libs.invokeStatic(core.clj:5776)
	at clojure.core$load_libs.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$require.invokeStatic(core.clj:5798)
	at clojure.core$require.doInvoke(core.clj)
	at clojure.lang.RestFn.invoke(RestFn.java:457)
	at asphalt.core_test$eval192$loading__5565__auto____193.invoke(core_test.clj:1)
	at asphalt.core_test$eval192.invokeStatic(core_test.clj:1)
	at asphalt.core_test$eval192.invoke(core_test.clj)
	at clojure.lang.Compiler.eval(Compiler.java:6913)
	at clojure.lang.Compiler.eval(Compiler.java:6902)
	at clojure.lang.Compiler.load(Compiler.java:7360)
	at clojure.lang.RT.loadResourceScript(RT.java:372)
	at clojure.lang.RT.loadResourceScript(RT.java:363)
	at clojure.lang.RT.load(RT.java:453)
	at clojure.lang.RT.load(RT.java:419)
	at clojure.core$load$fn__5673.invoke(core.clj:5895)
	at clojure.core$load.invokeStatic(core.clj:5894)
	at clojure.core$load_one.invokeStatic(core.clj:5694)
	at clojure.core$load_one.invoke(core.clj)
	at clojure.core$load_lib$fn__5622.invoke(core.clj:5739)
	at clojure.core$load_lib.invokeStatic(core.clj:5738)
	at clojure.core$load_lib.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$load_libs.invokeStatic(core.clj:5776)
	at clojure.core$load_libs.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$require.invokeStatic(core.clj:5798)
	at clojure.core$require.doInvoke(core.clj)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:647)
	at clojure.core$apply.invoke(core.clj)
	at user$eval91.invokeStatic(form-init7505432955041312280.clj:1)
	at user$eval91.invoke(form-init7505432955041312280.clj)
	at clojure.lang.Compiler.eval(Compiler.java:6913)
	at clojure.lang.Compiler.eval(Compiler.java:6903)
	at clojure.lang.Compiler.load(Compiler.java:7360)
	at clojure.lang.Compiler.loadFile(Compiler.java:7298)
	at clojure.main$load_script.invokeStatic(main.clj:275)
	at clojure.main$init_opt.invokeStatic(main.clj:277)
	at clojure.main$init_opt.invoke(main.clj)
	at clojure.main$initialize.invokeStatic(main.clj:308)
	at clojure.main$null_opt.invokeStatic(main.clj:342)
	at clojure.main$null_opt.invoke(main.clj)
	at clojure.main$main.invokeStatic(main.clj:421)
	at clojure.main$main.doInvoke(main.clj)
	at clojure.lang.RestFn.invoke(RestFn.java:421)
	at clojure.lang.Var.invoke(Var.java:383)
	at clojure.lang.AFn.applyToHelper(AFn.java:156)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)
Caused by: java.lang.VerifyError: (class: asphalt/core$invoke_with_transaction, method: invokeStatic signature: (Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;) Expecting to find integer on stack
	at java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
	at java.lang.Class.getConstructor0(Class.java:3075)
	at java.lang.Class.newInstance(Class.java:412)
	at clojure.lang.Compiler$ObjExpr.eval(Compiler.java:4902)
	at clojure.lang.Compiler$DefExpr.eval(Compiler.java:450)
	... 95 more


 Comments   
Comment by Nicola Mometto [ 11/Nov/15 8:23 AM ]

Copying a comment I posted on the ML regarding this bug:

To be honest I'm not sure this should even be a valid use of type hints.
You're telling the compiler that the result of (foo) is an int, when it is infact a long.

The correct way to do this should be:

(Integer/bitCount (int (foo))

Again, lack of specification on what the correct type hinting semantics should be make it hard to evaluate if this should be considered a bug or just an user error that previously just got ignored.

Comment by Alex Miller [ 11/Nov/15 3:18 PM ]

The example Nicola gave in the description worked in 1.6 and 1.7 and 1.8 up to 1.8.0-alpha2. It started failing as of https://github.com/clojure/clojure/commit/8c9580cb6706f2dc40bb31bbdb96a6aefe341bd5 for CLJ-1533.

Comment by Alex Miller [ 11/Nov/15 3:20 PM ]

Rich pushed a new commit https://github.com/clojure/clojure/commit/9448d627e091bc010e68e05a5669c134cd715a98 for this in master.

Comment by Alex Miller [ 11/Nov/15 3:20 PM ]

The commit makes this kind of incorrect type hint (previously a no op) now a compile error.

Comment by Shantanu Kumar [ 11/Nov/15 8:59 PM ]

I tested with the latest master and it correctly reports the "Caused by: java.lang.UnsupportedOperationException: Cannot coerce long to int, use a cast instead" error now. However, the reported line number in the exception is that of the defn (first line of the fn) instead of where the coercion was attempted in the fn body.





[CLJ-1845] Allow load to be redefined Created: 10/Nov/15  Updated: 05/Dec/15  Resolved: 05/Dec/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File clj-1845.patch     Text File clj-1845-test.patch     Text File ctyp1845-direct-linking-test.patch    
Patch: Code
Approval: Ok

 Description   

With direct linking of core, we have lost the ability to easily redef load (as calls to load inside Clojure are direct linked).

Proposed: make load dynamic (dynamic vars are not direct linked)

Patch: clj-1845.patch
See: https://groups.google.com/d/msg/clojure/_AGdLHSg41Q/q8LeplkrBQAJ

------------------------------------------

Re-opened because the initial declare of load is not declared as ^:dynamic and thus functions that use load between the initial forward declare and the later actual declaration were still allowing direct linking.

Because we are adding ^:redef, I rolled the changes into CLJ-1851 instead. The only thing here is a new test (which will fail till CLJ-1851 is applied).

Test patch: clj-1845-test.patch (NEW)
See also: CLJ-1851
Screened by: Alex Miller



 Comments   
Comment by Alex Miller [ 20/Nov/15 9:18 AM ]

Reopening...

Comment by Alex Miller [ 20/Nov/15 9:19 AM ]

Test (that doesn't work):

user=> (alter-var-root #'load (fn [f] (fn [& args] (prn "patched") (apply f args))))
#object[user$eval1241$fn__1242$fn__1243 0x1c857e6 "user$eval1241$fn__1242$fn__1243@1c857e6"]
user=> (load)
"patched"
nil
user=> (require 'clojure.core :reload)   ;; expect to see "patched"
nil
user=> (require 'clojure.tools.analyzer :reload)   ;; expect to see "patched"
nil
user=> (require 'clojure.tools.analyzer :reload-all)   ;; expect to see "patched"
nil
Comment by Alex Miller [ 20/Nov/15 9:20 AM ]

The issue is that load is forward-declared and the forward declaration does not declare dynamic. Replacing (declare load) with (def ^:declared ^:dynamic load) will fix it.

Comment by Ambrose Bonnaire-Sergeant [ 20/Nov/15 9:47 AM ]

Interested in a patch with a test?

Comment by Ambrose Bonnaire-Sergeant [ 20/Nov/15 9:52 AM ]

Confirmed that (declare ^:dynamic load) fixes the problem

Comment by Alex Miller [ 20/Nov/15 9:55 AM ]

No patch - this interacts with another change and I may just roll them together.

Comment by Alex Miller [ 20/Nov/15 9:56 AM ]

Actually, if you wanted to make a patch for a test, that would be useful.

Comment by Ambrose Bonnaire-Sergeant [ 20/Nov/15 10:12 AM ]

Attached direct linking test.

Comment by Alex Miller [ 20/Nov/15 11:00 AM ]

New test patch that applies to master

Comment by Andy Fingerhut [ 05/Dec/15 3:13 PM ]

It appears that CLJ-1845, CLJ-1851, and CLJ-1856 are committed now, and can be closed as complete?





[CLJ-1842] Use code generation to support more than 4 primitive arguments in function calls Created: 02/Nov/15  Updated: 02/Nov/15  Resolved: 02/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Richard Davies Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: compiler
Environment:

All



 Description   

The current restriction that "fns taking primitives support only 4 or fewer args" is apparently to prevent an explosion of possible interfaces https://groups.google.com/forum/#!topic/clojure/MI7iakMqEXo . Could the same code generation approach to "Unrolled small vectors" (http://dev.clojure.org/jira/browse/CLJ-1517) be used to increase the supported arities of primitive functions in IFn.java? I have bumped into this restriction a few times when trying to tune my code for high performance.

Each additional arity would require (1 + arg-arity)^3 interfaces to be generated. I understand that it is possible to embed primitives within deftypes instead of passing more parameters. However, the main use case for type-hinting to generate primitive interfaces is when an increase in performance is required so the deftype work-around is not optimal. Likewise, it is possible to drop out to Java to implement the primitive functions but this complicates the development cycle/tools/etc.



 Comments   
Comment by Alex Miller [ 02/Nov/15 7:53 PM ]

The issue is not generating the interfaces (the existing interfaces were themselves generated), but whether the result is manageable in terms of code size etc. There are other ways to approach it though and we may do something in the future. Declining the ticket as we would not work it off of here.





[CLJ-1835] Fix minor typos in documentation Created: 28/Oct/15  Updated: 28/Oct/15  Resolved: 28/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.7
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Artur Cygan Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: documentation
Environment:

Not relevant


Attachments: Text File 0001-PATCH-Fix-typos-iff-if-in-docstrings-and-comment.patch    

 Description   

iff -> if



 Comments   
Comment by Nicola Mometto [ 28/Oct/15 6:07 AM ]

Those are not typos, iff means "if and only if"

Comment by Artur Cygan [ 28/Oct/15 6:11 AM ]

Ah okay, didn't know that.





[CLJ-1834] Support for test matrix build of direct linking on / off Created: 26/Oct/15  Updated: 27/Oct/15  Resolved: 27/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: build

Attachments: Text File clj-1834-2.patch     Text File clj-1834-3.patch     Text File clj-1834.patch    
Patch: Code
Approval: Ok

 Description   

Enable build box test matrix build of direct linking on and off.

Modified build to do the following:

  • Maven build - add user property "directlinking" with default value "true"
  • Maven build - add two profiles: "direct" and "nodirect" which force this property to either "true" or "false"
  • Ant build - defines new property "directlinking"
  • Ant build - inherits this property value from Maven automatically
  • Ant build - echoes the setting of the property during test compilation so you can tell which is activated
  • Ant build - compiles and runs tests with clojure.compiler.direct-linking set to the value of ${directlinking}

The Maven build can be invoked with one of these as follows:

mvn clean test -Ptest-direct
mvn clean test -Ptest-no-direct

The Hudson clojure-test-matrix will have a new axis with values "test-direct" and "test-no-direct" and a modified command line that will set the profile with -P based on the axis value.

Patch: clj-1834-3.patch



 Comments   
Comment by Alex Miller [ 26/Oct/15 5:10 PM ]

I may have broken the default ant build behavior with this patch, need to check on that still but taking a break for now...

Comment by Alex Miller [ 27/Oct/15 9:03 AM ]

Ant behavior fixed in -2 patch





[CLJ-1833] pretty-print fix needs type hint Created: 26/Oct/15  Updated: 26/Oct/15  Resolved: 26/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Steve Miner Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: pprint

Attachments: Text File CLJ-1833-type-hint-in-pretty-writer.patch    

 Description   

A recent fix to the pretty-print code is missing a type hint. Other recent fixes turned on reflection warnings so now you get this warning when building Clojure:

Reflection warning, clojure/pprint/pretty_writer.clj:419:12 - call to method write can't be resolved (target class is unknown).



 Comments   
Comment by Steve Miner [ 26/Oct/15 9:46 AM ]

The original fix was CLJ-1390. It was missing a type hint. (My fault.)

Comment by Steve Miner [ 26/Oct/15 9:47 AM ]

added type hint

Comment by Alex Miller [ 26/Oct/15 9:47 AM ]

Dupe of CLJ-1827





[CLJ-1831] Add map-entry? predicate Created: 19/Oct/15  Updated: 27/Oct/15  Resolved: 27/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File clj-1831.patch    
Patch: Code and Test
Approval: Ok

 Description   

Due to changes in 1.8 with making vector implement IMapEntry for 2-element vectors, checking whether something is a map entry has become a bit trickier. This enhancement proposes a new function `map-entry?` to encapsulate that check and any future updates to it.

The check for map-entry? will return true if the instance implements java.util.Map$Entry and if it is also a vector, if it's size is exactly 2.

Patch: clj-1831.patch

Screened by Joe Smith.



 Comments   
Comment by Nicola Mometto [ 24/Oct/15 3:33 PM ]

Joe R. Smith Only members of the clojure core team are allowed to screen tickets

Comment by Ghadi Shayban [ 24/Oct/15 3:39 PM ]

Nicola, Joe Smith is core team.

Comment by Nicola Mometto [ 24/Oct/15 5:21 PM ]

Sorry about that then, I restored the ticket status

Comment by Nicola Mometto [ 24/Oct/15 5:23 PM ]

http://dev.clojure.org/display/community/Screeners should probably be updated then

Comment by Alex Miller [ 24/Oct/15 9:08 PM ]

Yep, will do.





[CLJ-1830] Test equality ignore decimal Created: 18/Oct/15  Updated: 19/Oct/15  Resolved: 18/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Nick DeCoursin Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   
user> (= {:a 1.00} {:a 1.0})
true
user> (= {:a 1} {:a 1.0})
false

This ^ is expected because (not (= 1 1.0)), so instead == is used to compare number equivalence: (== 1 1.0). But, == fails on collections:

user> (== {:a 1} {:a 1.0})
ClassCastException clojure.lang.PersistentArrayMap cannot be cast to java.lang.Number  clojure.lang.Numbers.equiv (Numbers.java:208)

In summary, there's not way to make this assertion (= {:a 1} {:a 1.0})



 Comments   
Comment by Alex Miller [ 18/Oct/15 4:50 PM ]

This would require a significant number of changes across the collection hierarchy to define a new additional kind of equivalence. I do not expect that we will add this functionality. If we did embark on this, it would be done through a design effort, not through a ticket like this. Thanks for the suggestion.

Comment by Andy Fingerhut [ 18/Oct/15 5:13 PM ]

Nick: Clojure core members make the final calls on things like this, but I am pretty sure that (= 1 1.0) is false by design. The inability to use == to compare anything other than numbers is also by design.

If you want a function, which for sake of example I will call "deep==" here, that uses == on numbers inside of collections, or collections nested inside of other collections, etc., I don't think it would be difficult to write one, as long as the values you wanted to compare using == were the values in the maps only, and not the keys.

If you wanted a function where (deep== {1 :a} {1.0 :a}) returned true, you would have to do something other than the normal key lookup functions built into Clojure, because they use clojure.core/hash to put items into 'hash buckets'. (clojure.core/hash x) and (clojure.core/hash (* 1.0 x)) are in general different from each other, again I believe by design.

Comment by Nick DeCoursin [ 19/Oct/15 1:34 AM ]

Thank you for looking at this, for your input, and the details.

I think a design change might be worth it. This would probably need to happen at 2.0 or something. But this is pretty fundamental that can serious hidden bugs.

Anyways, I don't mean to start a debate, but it's something that I think deserves some consideration. Thank you.





[CLJ-1829] VerifyError on Android Created: 16/Oct/15  Updated: 11/Jan/16  Resolved: 11/Jan/16

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Konstantin Mikheev Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: android, compiler
Environment:

Android API >= 21


Attachments: Text File clj-1829.patch    
Patch: Code
Approval: Ok

 Description   

Android Lollipop (API level 21) introduced an advanced bytecode checker that does not allow Clojure 1.8 to run.

Here is an example repo that reproduces the issue:
https://github.com/konmik/try_clojure_on_android/blob/master/app/src/main/java/com/clojure_on_android/TestInvokeUnit.java#L8

It uses 'org.clojure:clojure:1.8.0-beta1' dependency.

To reproduce the exception you need to install Android Studio
https://developer.android.com/sdk/index.html
and an android emulator https://www.genymotion.com/

Run the emulator, open the project and press "run" in the IDE.

The expected result that I get on Android API < 21 is:
https://github.com/konmik/try_clojure_on_android/blob/master/expected.png

On Android versions >= 21 I get:

10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: java.lang.VerifyError: Rejecting class clojure.core.server$stop_server because it failed compile-time verification (declaration of 'clojure.core.server$stop_server' appears in /data/app/com.clojure_on_android-1/base.apk)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core.server__init.load(Unknown Source)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core.server__init.<clinit>(Unknown Source)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at java.lang.Class.classForName(Native Method)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at java.lang.Class.forName(Class.java:309)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.classForName(RT.java:2168)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.classForName(RT.java:2177)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.loadClassForName(RT.java:2196)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.load(RT.java:443)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.load(RT.java:419)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load$fn__5669.invoke(core.clj:5885)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load.invokeStatic(core.clj:5884)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_one.invokeStatic(core.clj:5685)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_one.invoke(core.clj)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_lib$fn__5618.invoke(core.clj:5730)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_lib.invokeStatic(core.clj:5729)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_lib.doInvoke(core.clj)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RestFn.applyTo(RestFn.java:142)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$apply.invokeStatic(core.clj:647)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_libs.invokeStatic(core.clj:5767)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$load_libs.doInvoke(core.clj)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RestFn.applyTo(RestFn.java:137)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$apply.invokeStatic(core.clj:647)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$require.invokeStatic(core.clj:5789)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.core$require.doInvoke(core.clj)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RestFn.invoke(RestFn.java:408)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.Var.invoke(Var.java:379)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.doInit(RT.java:480)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.RT.<clinit>(RT.java:331)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.Namespace.<init>(Namespace.java:34)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.Namespace.findOrCreate(Namespace.java:176)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.lang.Var.intern(Var.java:146)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.java.api.Clojure.var(Clojure.java:82)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at clojure.java.api.Clojure.<clinit>(Clojure.java:96)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at com.clojure_on_android.TestInvokeUnit.invokePlus(TestInvokeUnit.java:8)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at com.clojure_on_android.MainActivity.onCreate(MainActivity.java:15)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.Activity.performCreate(Activity.java:5990)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2278)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2387)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.ActivityThread.access$800(ActivityThread.java:151)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1303)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:102)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.os.Looper.loop(Looper.java:135)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:5254)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at java.lang.reflect.Method.invoke(Method.java:372)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903)
10-16 14:49:26.801 2008-2008/? E/AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)

Cause: In Clojure 1.8, the socket server code is loaded during startup, which causes classes to be loaded that are compiled with the locking macro. The locking macro generates monitorenter and monitorexit instructions (and exception handlers) that do not conform to the (optional) structured locking section of the JVM spec. While this code is considered valid in OracleJDK, OpenJDK, etc the new Android bytecode checker will fail with it. Other versions of Clojure also have this verification issue, but the use of the locking macro during language boot time changes this potential issue to always being a problem.

Approach: The proposed patch sidesteps this issue by avoiding the locking macro and replaces it with a similar macro that uses ReentrantLock instead. This approach has been verified on the provided test case.

Patch: clj-1829.patch

Alternatives: There is a patch available for the locking macro (CLJ-1472) that replaces the locking macro by a synchronized block instead.



 Comments   
Comment by Alex Miller [ 16/Oct/15 2:32 PM ]

This could be a result of CLJ-1809 (hard to tell). The clojure.core.server/stop-server fn is a new fn with the socket server feature and should be direct-linked, which could implicate 1809.

Comment by Alex Miller [ 16/Oct/15 3:14 PM ]

I tried to reproduce this (using Run -> Run 'build' in Android Studio). The build was successful. I suspect I'm missing one or more steps in how to run correctly. Do I need to add a virtual device in Genymotion and if so, does it matter which one?

Comment by Konstantin Mikheev [ 16/Oct/15 3:17 PM ]

Yes, the build is successful.

The issue appears when you run it on a device.

You need to add a device in the emulator with API level >= 21 and to run it there.

Comment by Konstantin Mikheev [ 16/Oct/15 3:20 PM ]

When you run it, the logcat (Android logging panel) appears that is showing you the system log. The application's crash log can be found there.

Comment by Alex Miller [ 16/Oct/15 4:39 PM ]

Well, I tried pretty hard but I still can't figure out how to make Android Studio run the project in Genymotion. This was helpful https://www.genymotion.com/#!/developers/user-guide#genymotion-plugin-for-android-studio and I installed the Genymotion plugin etc but I never seem to get the opportunity to choose a device when I run the build.

What I would like to try (in case anyone else is able to do this) is to apply the patch from CLJ-1809 to clojure master, do a build, and then see if that fixes the problem in the Android emulator. If so, then this is just a dupe of CLJ-1809. If not, then probably some more digging is needed.

Comment by Konstantin Mikheev [ 16/Oct/15 4:46 PM ]

Oh no, you don't need the geny plugin for android studio.
It is sad you wasn't able to run it.
I'll run any test builds for you, just let me know when the next version become available.

Comment by Konstantin Mikheev [ 28/Oct/15 3:12 PM ]

I've just tried the org.clojure:clojure:1.8.0-beta2 release and the bug is still there.

Comment by Alex Miller [ 28/Oct/15 4:17 PM ]

I believe you, but I will need more explicit instructions on how to reproduce. (Or someone else needs to track this down.)

Comment by Konstantin Mikheev [ 28/Oct/15 4:25 PM ]

OK, I'll make a series of screenshots.

Comment by Konstantin Mikheev [ 15/Nov/15 3:33 PM ]

Sorry for the delay.

1. At first you need to setup Android Studio: http://developer.android.com/sdk/installing/index.html?pkg=studio

2. And setup the Android SDK: http://developer.android.com/sdk/installing/adding-packages.html

you will need to install:
Tools -> Android SDK Tools,
Tools -> Android SDK Platform Tools,
Tools -> Android SDK Build Tools 23.0.1,
Android 6.0 -> SDK Platform
Extras -> Google Repository (not sure if it is needed)

3. Run Android Studio and open the project.

4. Running the project on the genymotion emulator: https://github.com/konmik/try_clojure_on_android/tree/master/run_with_genymotion

Comment by Alex Miller [ 20/Nov/15 10:03 AM ]

Have you tried 1.8.0-RC2 with this problem?

Comment by Alex Miller [ 20/Nov/15 10:22 AM ]

stop-server uses the locking macro which reminds me of CLJ-1472.

Comment by Konstantin Mikheev [ 20/Nov/15 11:57 AM ]

Yes I've tried, it still doesn't work.

There is something with the new Android bytecode validator.
We had similar problems with validation while using retrolambda.

Comment by Konstantin Mikheev [ 16/Dec/15 3:51 PM ]

Does not work with org.clojure:clojure:1.8.0-RC4 still.

Comment by Alex Miller [ 21/Dec/15 9:29 AM ]

I was able to reproduce this and verify the hypothesis I had above - this is a duplicate of CLJ-1472. The clojure.core/locking macro seems to generate bytecode that fails on the latest ART bytecode validator (that ticket has more detail on this and some links to issues filed on Android in this regard).

Clojure 1.8 is not actually new in this regard - any version of Clojure would fail in the same way as the locking macro has not changed. The difference here is that the new Clojure socket server code in 1.8 causes it to be used during runtime startup, so the failure occurs during bootstrapping when it did not previously.





[CLJ-1828] Remove trailing whitespace in clojure.test Created: 13/Oct/15  Updated: 13/Oct/15  Resolved: 13/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Attachments: Text File clojure-test-whitespace.patch    

 Description   

Removes trailing whitespace from clojure.test.



 Comments   
Comment by Daniel Compton [ 13/Oct/15 9:00 PM ]

Declined by Alex on Slack.





[CLJ-1827] Reflection warning introduced in CLJ-1259 Created: 13/Oct/15  Updated: 27/Oct/15  Resolved: 27/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: print, reflection, regression
Environment:

1.8.0-beta1


Attachments: Text File clj-1827-v1.patch    
Patch: Code
Approval: Ok

 Description   

The patch for CLJ-1259 addressed the reflection warnings in pprint. However, a different ticket introduced some new code in between when this patch was written and applied and thus there is a new reflection warning produced in the Clojure build:

[java] Reflection warning, clojure/pprint/pretty_writer.clj:419:12 - call to method write can't be resolved (target class is unknown).

This ticket is to address that one newly introduced case to remove the warning.

Patch: clj-1827-v1.patch
Screened by: Alex Miller



 Comments   
Comment by Andy Fingerhut [ 15/Oct/15 10:23 AM ]

Patch clj-1827-v1.patch dated Oct 15 2015 eliminates the one reflection warning in pretty_writer.clj.





[CLJ-1825] Compilation errors on anonymous recursive function Created: 12/Oct/15  Updated: 12/Nov/15  Resolved: 12/Nov/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.8
Fix Version/s: Release 1.8

Type: Defect Priority: Major
Reporter: Nicolas Modrzyk Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None
Environment:

OS X, Yosemite, jdk 1.8.0_60


Approval: Vetted

 Description   

Seems the below does not compile with 1.8:

(def lazy-fib
  "Lazy sequence of fibonacci numbers"
  ((fn rfib [a b]
     (lazy-seq (cons a (rfib b (+' a b)))))
   0 1))

(defn even-lazy-fib[n]
  (filter even? (take n lazy-fib)))

(even-lazy-fib 10)

Status:

  • 1.7.0 - works
  • 1.8.0-alpha2 - works
  • 1.8.0-alpha3-1.8.0-beta1 - VerifyError, see below
  • 1.8.0-beta2 - NPE, see below
  • 1.8.0-RC1 - ClassCastException, see below
  • 1.8.0 master - NPE, see below

1.8.0-alpha3:

CompilerException java.lang.VerifyError: (class: vecperf/bench$rfib__1233, method: invokeStatic signature: (Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;) Illegal local variable number, compiling:(form-init3780016918836504993.clj:3:3)
	clojure.lang.Compiler$InvokeExpr.eval (Compiler.java:3661)
	clojure.lang.Compiler$DefExpr.eval (Compiler.java:455)
	clojure.lang.Compiler.eval (Compiler.java:6948)
	clojure.lang.Compiler.eval (Compiler.java:6906)
	clojure.core/eval (core.clj:3084)
	clojure.core/eval (core.clj:-1)
	clojure.main/repl/read-eval-print--7081/fn--7084 (main.clj:240)
	clojure.main/repl/read-eval-print--7081 (main.clj:240)
	clojure.main/repl/fn--7090 (main.clj:258)
	clojure.main/repl (main.clj:258)
	clojure.main/repl (main.clj:-1)
	clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn--630 (interruptible_eval.clj:58)
Caused by:
VerifyError (class: vecperf/bench$rfib__1233, method: invokeStatic signature: (Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;) Illegal local variable number
	java.lang.Class.getDeclaredConstructors0 (Class.java:-2)
	java.lang.Class.privateGetDeclaredConstructors (Class.java:2658)
	java.lang.Class.getConstructor0 (Class.java:2964)
	java.lang.Class.newInstance (Class.java:403)
	clojure.lang.Compiler$ObjExpr.eval (Compiler.java:4943)
	clojure.lang.Compiler$InvokeExpr.eval (Compiler.java:3652)
	clojure.lang.Compiler$DefExpr.eval (Compiler.java:455)

1.8.0-beta2 / 1.8.0 master:

NullPointerException
	clojure.lang.Numbers.ops (Numbers.java:1013)
	clojure.lang.Numbers.addP (Numbers.java:132)
	user/rfib--1250/fn--1251 (form-init4987495233354047014.clj:4)

1.8.0-RC1:

ClassCastException java.lang.Long cannot be cast to clojure.lang.IFn
	user/rfib--1250/fn--1251 (form-init1118919529313120594.clj:4)


 Comments   
Comment by Alex Miller [ 12/Oct/15 10:07 PM ]

Dupe of CLJ-1809

Comment by Nicolas Modrzyk [ 11/Nov/15 8:51 PM ]

With 1.8-RC1, and the code above, I now get the following:

java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn
/Users/niko/projects/maths/src/maths/fastfib.clj:41 maths.fastfib/rfib[fn]
          LazySeq.java:40 clojure.lang.LazySeq.sval
          LazySeq.java:49 clojure.lang.LazySeq.seq
              RT.java:521 clojure.lang.RT.seq
             core.clj:137 clojure.core/seq
            core.clj:2777 clojure.core/take[fn]
          LazySeq.java:40 clojure.lang.LazySeq.sval
          LazySeq.java:49 clojure.lang.LazySeq.seq
              RT.java:521 clojure.lang.RT.seq
             core.clj:137 clojure.core/seq
            core.clj:2702 clojure.core/filter[fn]
          LazySeq.java:40 clojure.lang.LazySeq.sval
          LazySeq.java:49 clojure.lang.LazySeq.seq
Comment by Alex Miller [ 12/Nov/15 10:26 AM ]

The generated bytecode for rfib seems fishy to me. In 1.7 for example, it does aload_0 to load the this reference to self-invoke, but in 1.8 that winds up in an invokestatic where aload_0 is a, not this, so the stack is messed up when invokespecial is invoked.

1.7:

public java.lang.Object invoke(java.lang.Object, java.lang.Object);
    Code:
       0: new           #14                 // class clojure/lang/LazySeq
       3: dup           
       4: new           #16                 // class fib$rfib__25$fn__26
       7: dup           
       8: aload_0       
       9: aload_1       
      10: aconst_null   
      11: astore_1      
      12: aload_2       
      13: aconst_null   
      14: astore_2      
      15: invokespecial #19                 // Method fib$rfib__25$fn__26."<init>":(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V

In 1.8:

public static java.lang.Object invokeStatic(java.lang.Object, java.lang.Object);
    Code:
       0: new           #14                 // class clojure/lang/LazySeq
       3: dup           
       4: new           #16                 // class fib$rfib__25$fn__26
       7: dup           
       8: aload_0
       9: aload_0
      10: aconst_null
      11: astore_0
      12: aload_1
      13: aconst_null
      14: astore_1
      15: invokespecial #19                 // Method fib$rfib__25$fn__26."<init>":(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)V
Comment by Alex Miller [ 12/Nov/15 4:36 PM ]

Rich made a commit to fix this in master:

https://github.com/clojure/clojure/commit/7faeb3a5e1fb183539a8638b72d299a3433fe990

Comment by Nicolas Modrzyk [ 12/Nov/15 6:46 PM ]

Confirming, master with commit "7faeb3a5e1fb183539a8638b72d299a3433fe990" fixes it for me too.





[CLJ-1824] Splicing macros Created: 12/Oct/15  Updated: 14/Oct/15  Resolved: 14/Oct/15

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Mike Anderson Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

In many cases, it would be convenient for a macro to "splice" its return value into the resulting form.

Example use cases:

  • `comment` can return no forms, so that it can be inserted in places where nil would be disruptive
  • a macro can return two forms to create a condition in a `cond` block
  • a macro can create multiple forms to support a variable-arity function (e.g. passing a set of keyword arguments)
  • a macro can create one or more complete `binding`, `let` or `loop` bindings

Proposed syntax and usage:

(defmacro-splicing multi-test [v values exp]
  (mapcat 
    (fn [value] `[(= ~v ~value) ~exp]`
    values))

(let [v (compute-some-result)]
  (cond 
    (multi-test v [1 3 5 7 9] "Odd digit")
    (multi-test v [0 2 4 6 8] "Even digit")
    :else "Not a digit"))


 Comments   
Comment by Ghadi Shayban [ 13/Oct/15 7:08 PM ]

These sorts of things need a design discussion prior to a JIRA ticket.

Comment by Alex Miller [ 14/Oct/15 8:00 AM ]

I'm declining this as a ticket as it does really need design evaluation in some way before it gets to this point (either clojure-dev mailing list or a des