patches! (Clojure JIRA)

[CLJ-803] IAtom interface

Mon, 4 Jul 2011 21:18:48 -0500

Atom and the other reference types do not have interfaces and are marked final.

Use cases for interfaces for the reference types include database wrappers. CouchDB behaves exactly like compare-and-set! and is shared, synchronous, independent state, so it makes sense to use the Atom interface to update a CouchDB document.

I talked to Rich about this, and he said "patch welcome for IAtom", complete conversation: http://clojure-log.n01se.net/date/2010-12-29.html#10:04c

[CLJ-821] should reify merge rather than replace on repeated specs?

Tue, 9 Aug 2011 14:18:40 -0500

reify, deftype, and the like fail silently if you specify a class twice:

(macroexpand '(reify Map (size [this] 0),
Counted (count [this] 0),
Map (keySet [this] nil)))
;=> (reify* [Counted Map] (count [this] 0) (keySet [this] nil))

The later Map section entirely supersedes the former, which I discovered when I wrote a macro that injects some automated method bodies into a reify for you.

I've attached a fix to make the above expand to the expected [for me, anyway] output.

[CLJ-735] Improve error message when a protocol method is not found

Fri, 9 Dec 2011 10:05:55 -0600

If you call a protocol function but pass the wrong arity (forget an argument for example), you currently a message that says "No single method ... of interface ... found for function ... of protocol ...". The code in question is getting matching methods from the Reflector and creates this message if the number of matches != 1.

There are really two cases there:

matches == 0 - this happens frequently due to typos
matches > 1 - this presumably happens infrequently

I propose that the == 0 case instead should have slightly different text at the beginning and a hint as to the intended arity within it:

"No method: ... of interface ... with arity ... found for function ... of protocol ...".

The >1 case should have similar changes: "Multiple methods: ... of interface ... with arity ... found for function ... of protocol ...".

Patch is attached. I used case which presumably should have better performance than a nested if/else. I was not sure whether the reported arity should match the actual Java method arity or Clojure protocol function arity (including the target). I did the former.

I did not add a test as I wasn't sure whether checking error messages in tests was appropriate or not. Happy to add that if requested.

[CLJ-891] make (symbol foo "bar") work with foo being a namespace

Fri, 2 Mar 2012 16:53:07 -0600

I've run across the need to do (symbol ns "bar") a few times, and the existing approach (symbol (name (ns-name ns)) "bar") just doesn't seem like it ought to be the only way to do the job. Includes a patch to make this work, by adding a new arity to Symbol.intern().

Some discussion on this idea, here: https://groups.google.com/forum/#!topic/clojure/n25aZ5HA7hc/discussion

[CLJ-884] Reflector error messages can be improved when no matching method is found.

Fri, 23 Mar 2012 01:14:18 -0500

When accessing a java method with an arity mismatch or a mismatched parameter type, Reflector.java returns the following error on REPL:
IllegalArgumentException No matching method found: xyz for class com.abc.MyClass

eventhough method xyz might exist on MyClass, but was being called with the wrong number of arguments.

Attached is a patch that fixes that problem.

[CLJ-308] protocol-ize with-open

Fri, 13 Apr 2012 12:52:37 -0500

Good use (and documentation example) of protocols: make with-open aware of a Closable protocol for APIs that use a different close convention. See http://groups.google.com/group/clojure/browse_thread/thread/86c87e1fc4b1347c

[CLJ-5] Unintuitive error response in clojure 1.0

Sat, 28 Apr 2012 22:48:40 -0500

The following broken code:

(let [[x y] {}] x)

provides the following stack trace:

Exception in thread "main" java.lang.UnsupportedOperationException: nth not supported on this type: PersistentArrayMap (test.clj:0)
at clojure.lang.Compiler.eval(Compiler.java:4543)
at clojure.lang.Compiler.load(Compiler.java:4857)
at clojure.lang.Compiler.loadFile(Compiler.java:4824)
at clojure.main$load_script__5833.invoke(main.clj:206)
at clojure.main$script_opt__5864.invoke(main.clj:258)
at clojure.main$main__5888.doInvoke(main.clj:333)
at clojure.lang.RestFn.invoke(RestFn.java:413)
at clojure.lang.Var.invoke(Var.java:346)
at clojure.lang.AFn.applyToHelper(AFn.java:173)
at clojure.lang.Var.applyTo(Var.java:463)
at clojure.main.main(main.java:39)
Caused by: java.lang.UnsupportedOperationException: nth not supported on this type: PersistentArrayMap
at clojure.lang.RT.nth(RT.java:800)
at clojure.core$nth__3578.invoke(core.clj:873)
at user$eval__1.invoke(test.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:4532)
... 10 more

The message "nth not supported on this type" while correct doesn't make the cause of the error very clear. Better error messages when destructuring would be very helpful.

[CLJ-248] Add support for subsets and submaps to sorted Sets/Maps

Sat, 2 Jun 2012 19:08:44 -0500

Currently we can have a subseq of a sorted-set|map-by, but this returns a seq and
not a Set or Map.
Implementing the interfaces java.util.SortedSet and java.util.SortedMap would give
us support for calling .subSet and .subMap.

[CLJ-865] Macroexpansion discards &form metadata

Wed, 6 Jun 2012 17:29:38 -0500

As discussed in http://groups.google.com/group/clojure/browse_thread/thread/2690cb6ca0e8beb8 there is a "surprise factor" when type-hinting an expression that represents a macro, such as with (.length ^String (doto (identity "x") prn)). Here the doto macro discards the metadata on &form, causing a reflective lookup. This has the effect that while expressions representing function calls can be type-hinted, expressions representing macros in general cannot. The doto macro could be rewritten to respect its &form metadata, but doing this for every macro in existence would be tedious and error-prone. Instead, I propose a change to the compiler, to cause macroexpansion to hang onto the metadata automatically.

The first patch attached adds a test for the behavior I propose: this test fails. After applying the second patch, the test passes.

There are a couple points that merit further consideration before accepting my patch:

I'm not sure I actually got the Java code formatted correctly. My editor is not well-configured to get the clojure/core style right automatically.
My solution is to take the &form metadata, drop :line/:file keys, and then merge with the returned metadata, with &form taking precedence. I'm not sure whether this is the right approach in all cases, even though it works for :tag metadata.
I achieved this with a change to the compiler, which makes it fairly heavy-weight. It should be possible to instead adjust defmacro if changes to the compiler are not desirable. However, I believe this would involve substantially more work and be harder to test (for example, multiple arities complicate things). It seems nicer to treat the macroexpansion as a black box and then make metadata tweaks to the result, rather than modifying their actual defmacro code.
If a macro expands to something that is not an IObj, such as an Integer, then my patch silently discards the caller's metadata. Would it be better to throw an exception?

[CLJ-1010] A left-to-right-variant of `comp`

Thu, 14 Jun 2012 18:27:23 -0500

The function composition function `comp` is quite inefficient in cases like `(apply comp large-seq-of-fns)`, because its arity-greater-than-3 version reverses the seq.

I would be great if there was an alternative `comp*` (or whatever) function which is just like `comp` but composes left-to-right.

[CLJ-1029] ns defmacro allows arbitrary execution of clojure.core fns

Sun, 5 Aug 2012 10:00:07 -0500

The form:

(ns foo (:print "I AM A ROBOT"))

will print "I AM A ROBOT"

This is because the defmacro takes the name of the first element of the reference, looks it up in the clojure.core namespace and invokes it on the rest of the args.

This is minor, but it does mean that an otherwise declarative form is not executing code.

[CLJ-958] Make APersistentVector.iterator slightly more efficient

Tue, 14 Aug 2012 13:43:53 -0500

The attached patch uses a sequence to create an iterator for persistent vectors. It should be slightly more efficient for PersistentVector objects than repeatedly calling nth (for reasons that I outline in the commit message).

[CLJ-1046] Drop-while as a reducer

Sat, 18 Aug 2012 19:11:59 -0500

Implement drop-while as a reducer. Follows the same atom-based strategy as drop and take.

Does not depend on any of my other reducer patches, but there will probably be some minor merge conflicts unless it is merged after CLJ-1045, and before CLJ-992 and CLJ-993.

[CLJ-993] `range` reducer

Sat, 18 Aug 2012 19:18:48 -0500

Rich mentioned in IRC today he'd welcome a reducer implementation of clojure.core/range. Now that I've figured out how to do iterate, I figure I'll knock out range as well by the end of the night. Just opening the issue early to announce my intentions to anyone else interested in doing it.

[CLJ-957] Typehints for variadic methods in deftype fail to compile

Sun, 19 Aug 2012 04:31:33 -0500

When a method defined by a protocol has multiple typehinted signatures, it is impossible to implement them using deftype because deftype throws away the typehints. The compiler then looks for the proper signatures (i.e. with typehints) and throws an exception when it can't find them.

[CLJ-1047] Simplify the process of requiring fj in clojure.core.reducers

Tue, 21 Aug 2012 18:27:01 -0500

this patch removes compile-if in favor of import-if, removing code duplication

[CLJ-700] contains? broken for TransientMaps

Tue, 28 Aug 2012 17:48:49 -0500

(contains? (transient {:x "fine"}) :x)

false

also

(contains? (transient (hash-map :x "fine")) :x)

false

[CLJ-373] update-in with empty key paths

Sat, 8 Sep 2012 06:13:29 -0500

To the topic of get-in and update-in. While I realize this is not a bug it is odd and in my eyes unexpected and unwanted behavior.

get-in called with an empty path returns the hashmap it was given to walk through - this is as I expect and makes a lot of sense when dynamically (with generated pathes ) walking a hash map.

update-in behaves differently and while from the implementation side, it's behavior too makes sense, it does not work as expected (at least not for me) update-in with an empty map creates a new key 'nil' so:

(update-in {...} [] f) ist he same as (update-in {...} [nil] f) while (get-in {...} []) is not the same as (get-in {...} [nil]) and of cause differs from what update-in does.

For automatically walking trees the behavior of get-in makes a lot more sense since the current behavior of update-in forces you to check for empty paths and if they are empty fall back to plain assoc and get (or get-in since this works):

(if-let [r (butlast @path)]
(do
  (alter m update-in r dissoc (last @path))
  (alter m update-in r assoc {:name @sr} c))
(do
  (alter m dissoc (last @path))
  (alter m assoc {:name @sr} c)))

Next argument is that update-in with an empty map working on nil isn't easy to gasp, one needs to know the implementation details to realize that it works, I think 90% of the people reading update-in with [] will not instinctively know that it works on the key nil, so changing this would most likely not break any current code, and if it would the code would be bad anyway .

Chouser has, a very nice solution on the mailing list that would fix the problem I'm not sure if I'm entitled to post it here since I did not wrote it but it can be found in this thread: http://groups.google.com/group/clojure/browse_thread/thread/de5b20b8c3fe498b?hl=en

Regards,
Heinz

[CLJ-1021] (let [i 5] (defmacro m [v] v) m) interprets m as a function, not a macro

Thu, 13 Sep 2012 14:29:20 -0500

(let [i 5] (defmacro m [& args] args) (def x (m (inc 5) (inc 6) (inc 7))) x m (meta #'m))
is
[(8) #<user$eval522$m_523 user$eval522$m_523@11a74355> {:macro true, :ns #<Namespace user>, :name m, :arglists ([& args]), :line 1, :file "NO_SOURCE_PATH"}]

It appears to be interpreting m as a function despite the metadata. This behavior only shows up inside the (let ...).

[CLJ-1020] clojure.inspector/inspect-table gives up when first element of coll is nil

Thu, 13 Sep 2012 14:30:24 -0500

clojure.inspector/inspect-table gives up when first element of coll is nil. The patch provided is rather trivial...instead of blindly choosing the first element (which might be nil), it would be more convenient to choose the first element that is NOT nil and use its keys for columns...a similar issue exists with clojure.pprint/print-table where the keys of the first element are used (if not provided explicitly). The same is not true for 'inspect-table' though. As a result, one cannot 'inspect' a collection of maps where the first element is nil. My (trivial) patch looks for the first element which is NOT nil and uses its keys instead. Maps have to have the same length anyway so no problems there...

[CLJ-994] repeat reducer

Fri, 14 Sep 2012 14:37:35 -0500

i'm working on clojure.core/repeat reducer.

[CLJ-1063] Missing dissoc-in

Mon, 17 Sep 2012 07:04:54 -0500

There is no clojure.core/dissoc-in although there is an assoc-in.
It is correct that dissoc-in can be build with update-in and dissoc but this is an argument against assoc-in as well.
When a shortcut for assoc-in is provided, there should also be one for dissoc-in for consistency reasons.
Implementation is analogical to assoc-in.

[CLJ-1049] Make reducer/folder support reduce-kv

Sun, 23 Sep 2012 20:34:21 -0500

Currently, although rfn makes an effort to support reduce-kv, it is wasted effort since the return values of reducer and folder don't satisfy IKVReduce.

I have a working patch for this, it's quite small. I have printed a CA but not sent it, so I won't reveal the patch yet...

This also applies to ClojureScript.

[CLJ-1077] thread-bound? returns true (implying set! should succeed) even for non-binding thread

Mon, 1 Oct 2012 11:57:10 -0500

thread-bound? returns true for a non-binding thread, this result (according to the docstring) implies that set! should succeed. However, thread-bound? does not check that any binding that might exist was created by the current thread, and calling set! fails with an exception when it is called from a non-binding thread, even though thread-bound? returns true.

thread-bound? should return false if there is a binding, and that binding was not established by the current thread.

[CLJ-703] Improve writeClassFile performance

Fri, 5 Oct 2012 19:08:06 -0500

This Discussion about timing issues when writing class files led to the the current implementation of synchronous writes to disk. This leads to bad performance/system-load when compiling Clojure code.

This Discussion questioned the current implentation.

Synchronous writes are not necessary and also do not ensure (crash while calling write) valid classfiles.

These Patches (0001 is just a code cleanup for creating the directory structure) ensures atomic creation of classfiles by using File.renameTo()

[CLJ-992] `iterate` reducer

Fri, 12 Oct 2012 08:10:52 -0500

Added a reducer implementation mirroring clojure.core/iterate.

[CLJ-1087] clojure.data/diff uses set union on key seqs

Mon, 15 Oct 2012 14:52:42 -0500

clojure.data/diff, on line 118, defines:

java.util.Map
(diff-similar [a b]
  (diff-associative a b (set/union (keys a) (keys b))))

Since keys returns a key seq, this seems like an error. clojure.set/union has strange and inconsistent behavior with regard to non-sets, and in this case the two key seqs are concatenated. Based on a cursory benchmark, it seems that this bug is a slight performance gain when the maps have no common keys, and a significant performance loss when the maps have the same keys. The results are still correct because of the merging reduce in diff-associative.

The patch is easy (just call set on each key seq).

[CLJ-862] pmap level of parallelism inconsistent for chunked vs non-chunked input

Sat, 20 Oct 2012 00:45:48 -0500

The code of pmap creates futures for the set of map operations it needs to perform with (map #(future %) coll), then acts on that sequence in a fashion obviously intended to keep only #CPUs+2 unfinished futures realized at any given time. It works exactly this way for non-chunked input seqs, but when pmap is passed a chunked seq, the seq of futures also becomes chunked. This causes an arbitrary number of futures to be realized as the #CPUs+2 window and chunk-size window interact.

The doc-string for pmap doesn't promise any particular level of parallelism, but I think the inconsistent parallelism for chunked vs non-chunked input comprises a bug.

[CLJ-1088] repl/source could support protocol functions

Sun, 21 Oct 2012 12:41:12 -0500

user=> (source clojure.core.protocols/coll-reduce)
Source not found

But since the protocol fn's var's metadata points to the protocol var, and the protocol var knows the file and line where it was defined, it would be trivial to improve 'source' to look like this:

user=> (source clojure.core.protocols/coll-reduce)
(defprotocol CollReduce
  "Protocol for collection types that can implement reduce faster than
  first/next recursion. Called by clojure.core/reduce. Baseline
  implementation defined in terms of Iterable."
  (coll-reduce [coll f] [coll f val]))

[CLJ-1094] Zero-arity versions of every-pred and some-fn

Thu, 25 Oct 2012 07:12:35 -0500

This patch adds zero-arity versions of every-pred and some-fn with these semantics.

(every-pred) === (constantly true)
(some-fn)    === (constantly nil)

These variants are useful in situations like the following:

;; compute-preds-for may return zero or many predicate fns
(let [preds (compute-preds-for something)]
  (filter (apply every-pred preds) some-coll))

[CLJ-1097] node-seq for clojure.zip

Mon, 29 Oct 2012 21:37:18 -0500

Many times it's easier to get to a zipper node via (first (filter pred ...)) instead of manually walking the tree via next/up/down. Other times it's easier to process certain nodes via filter & map instead of again, walking the tree. This patch provides a single function called node-seq that uses zip/next to generate a lazy-seq of nodes. Tests provide two examples.

[CLJ-1095] Allow map-indexed to accept multiple collections (a la map)

Thu, 8 Nov 2012 14:58:13 -0600

Bring external interface of map-indexed in line with map. Existing usages of map-indexed unchanged both in implementation and interface.

examples

(map vector (range 10 20) (range 30 35)) ;=> ([10 30] [11 31] [12 32] [13 33] [14 34])
(map-indexed vector (range 10 20) (range 30 35)) ;=> ([0 10 30] [1 11 31] [2 12 32] [3 13 33] [4 14 34])

The attached patch is not necessarily the best implementation (I haven't benchmarked it or tried any alternatives yet) but hopefully enough to start a conversation about whether this is an addition that is warranted. I know I wished for this behavior a few weeks ago though I ended up finding another way.

(I haven't sent my CA yet, but I have it signed and ready to send in the next few days)

[CLJ-842] clojure.pprint uses the old-style metadata.

Wed, 14 Nov 2012 13:19:38 -0600

I was looking through the implementation of clojure.pprint.* and found that it still uses the old-style metadata to mark vars as private - ^{:private true} instead of ^:private.

I have migrated the metadata to the new style in the attached trivial patch, which can be used in case this is deemed to be an issue.

FWIW, there are some other namespaces which use the old-style metadata as well; I am willing to fix those as well.

[CLJ-849] Add a pseudo-variable containing the current line number

Wed, 14 Nov 2012 15:44:21 -0600

Add a pseudo-variable (similar to *file*), containing the current line number. This should be available during AOT compilation. The name "*line-number*" might be good.

It will be useful for diagnostic log messages. Using exception stack traces has poor runtime performance and the line number should be available to the compiler at minimal cost.

[CLJ-666] Add support for Big* numeric types to Reflector

Thu, 15 Nov 2012 20:07:39 -0600

This should work as expected, for example:

(Integer. 1N)

Probably for BigInt, BigInteger, and BigDecimal.

Method to look at is c.l.Reflector.paramArgTypeMatch, per Rich in irc.

[CLJ-107] GC Issue 103: bit-count function

Thu, 15 Nov 2012 20:40:50 -0600

Reported by a...@thened.net, Apr 08, 2009

I posted this small patch to the mailing list last week but received no
feedback, so I'm attaching it here to make sure it doesn't get lost.  I
have submitted a CA.

http://groups.google.com/group/clojure/browse_thread/thread/4345f76a12bac6fe/

The new function bit-count returns the count of 1-bits in a number, like
C's popcount or Common Lisp's logcount.

[CLJ-99] GC Issue 95: max-key and min-key evaluate k multiple times for arguments

Thu, 15 Nov 2012 21:36:54 -0600

Reported by H.Duerer, Mar 13, 2009

max-key or min-key will evaluate (k value) multiple times for arguments if
more than 2 arguments are passed.

This is undesirable if k is expensive to calculate.

Something like the code below would avoid these double calculations (at the
price of generating more ephemeral garbage)

(defn max-key
  "Returns the x for which (k x), a number, is greatest."
  ([k x] x)
  ([k x y] (if (> (k x) (k y)) x y))
  ([k x y & more]
     (second (reduce (fn [x y] (if (> (first x) (first y)) x y))
                     (map #(vector (k %) %) (cons x (cons y more)))))))

[CLJ-1112] Var loading-verbosely should initialize from a JVM system property

Thu, 22 Nov 2012 02:14:32 -0600

I often find myself adding :verbose to a (require) or (use) clause of my (ns) in order to debug problems (especially macros, or bad namespace declarations). It would be very nice if I could define a JVM system property (say -Dclojure.load-verbosely=true) to default loading-verbosely to true for a REPL session, or as part of a build.

Sometimes I just like to see that namespaces load as a measure of progress, when starting an application, or when running a set of tests.

[CLJ-1113] `reductions` reducer

Fri, 23 Nov 2012 08:22:37 -0600

It would be nice to have a reducers implementation of the core `reductions` function.

Initial implementation attempt included. This version requires an initial reduction value parameter (vs using (f)) and includes that initial value in the final reduction. I'm not certain either of these decisions is optimal, but results in a function which most closely mimics `clojure.core/reductions`.

[CLJ-978] bean unable to handle non-public classes

Thu, 29 Nov 2012 10:01:49 -0600

Take the following Java as an example:

public interface IFoo {
  String getBar();
}

class FooImpl {
  String getBar() { return "bar"; }
}

As presently implemented, (bean my-foo) tries to invoke the following:

(. #<Method public java.lang.String FooImpl.getBar> (invoke my-foo nil))

However, as FooImpl is not public, this fails:

java.lang.IllegalAccessException: Class clojure.core$bean$fn__1827$fn__1828 can not access a member of class FooImpl with modifiers "public"
 at sun.reflect.Reflection.ensureMemberAccess (Reflection.java:65)
    java.lang.reflect.Method.invoke (Method.java:588)
    clojure.core$bean$fn__1827$fn__1828.invoke (core_proxy.clj:382)
    clojure.core$bean$v__1832.invoke (core_proxy.clj:388)
    clojure.core$bean$fn__1838$thisfn__1839$fn__1840.invoke (core_proxy.clj:406)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:60)
    clojure.lang.RT.seq (RT.java:473)

However, the same thing succeeds if we call #<Method public java.lang.String Foo.getBar> rather than #<Method public java.lang.String FooImpl.getBar>.

[CLJ-1090] Indirect function calls through Var instances fail to clear locals

Fri, 30 Nov 2012 14:26:45 -0600

If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error

I can provide a patch if it would be useful. The fix should be trivial, something along the lines of wrapping each argN in clojure/lang/Var.java inside a Util.ret1(argN, argN = null) as is done in RestFn.java.

[CLJ-1102] Better handling of exceptions with empty stack traces

Fri, 30 Nov 2012 14:57:46 -0600

I don't know what I did to cause some exceptions to be thrown while running Clojure tests that return a length 0 array from (.getStackTrace throwable), but according to the Java docs this is legal. I searched all places in the Clojure code that call .getStackTrace and found two that don't handle this correctly, one of which causes an ArrayOutOfBoundsException (that is the one I found during my testing).

[CLJ-200] Extend cond to support inline let, much like for

Sun, 2 Dec 2012 19:35:37 -0600

I find it occasionally very useful to do a few tests in a cond, then introduce some new symbols (for both clarity and efficiency) that can be referenced in later tests (or matching expressions). This parallels similar functionality inside the for macro, where the :let keyword is matched against a vector of symbol bindings and forms an implicit let around the remainder of the comprehension.

I'll be adding a patch for this shortly.

[CLJ-457] lazy recursive definition giving incorrect results

Mon, 3 Dec 2012 11:21:29 -0600

If you define a global data var in terms of a lazy sequence referring to that same var, you can get different results depending on the chunkiness of laziness of the functions being used to build the collection.

Clojure's lazy sequences don't promise to support this, but they shouldn't return wrong answers. In the example given in http://groups.google.com/group/clojure/browse_thread/thread/1c342fad8461602d (and repeated below), Clojure should not return bad data. An error message would be good, and even an infinite loop would be more reasonable than the current behavior.

(Similar issue reported here: https://groups.google.com/d/topic/clojure/yD941fIxhyE/discussion)

(def nums (drop 2 (range)))
(def primes (cons (first nums)
             (lazy-seq (->>
               (rest nums)
               (remove
                 (fn [x]
                   (let [dividors (take-while #(<= (* % %) x)
primes)]
                     (println (str "primes = " primes))
                     (some #(= 0 (rem x %)) dividors))))))))
(take 5 primes)

It prints out:
(primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
primes = (2)
2 3 5 7 9)

[CLJ-1115] multi arity into

Sun, 9 Dec 2012 07:39:55 -0600

Any reason why into isn't multi arity?

(into to & froms) => (reduce into to froms)

(into #{} [3 3 4] [2 1] ["a"]) looks better than (reduce into #{} [[3 3 4] [2 1] ["a"]])

[CLJ-1103] Make conj assoc dissoc and transient versions handle args similarly

Sun, 9 Dec 2012 17:30:00 -0600

A discussion came up in the Clojure Google group about conj giving an error when taking only a coll as an argument, as opposed to disj which works for this case:

https://groups.google.com/forum/?fromgroups=#!topic/clojure/Z9mFxsTYTqQ

I looked through the rest of the code for similar cases, and found that conj! assoc assoc! and disj! also had the same property, and there were some other differences between them in how different numbers of arguments were handled, such as:

conj handles an arbitrary number of arguments, but conj! does not.
assoc checks for a final key with no value specified (~~CLJ-1052~~), but assoc! did not.

[CLJ-1059] PersistentQueue doesn't implement java.util.List, causing nontransitive equality

Tue, 11 Dec 2012 13:58:01 -0600

PersistentQueue implements Sequential but doesn't implement java.util.List. Lists form an equality partition, as do Sequentials. This means that you can end up with nontransitive equality:

(def q (conj clojure.lang.PersistentQueue/EMPTY 1 2 3))
;=> #user/q
(def al (doto (java.util.ArrayList.) (.add 1) (.add 2) (.add 3)))
;=> #user/al
(def v [1 2 3])
;=> #user/v
(= al v)
;=> true
(= v q)
;=> true
(not= al q)
;=> true

This happens because PersistentQueue is a Sequential but not a List, ArrayList is a List but not a Sequential, and PersistentVector is both.

[CLJ-1128] Improve merge-with

Thu, 13 Dec 2012 17:41:39 -0600

Set a first map as an initial value for reduce to
avoid merge-entry (series of contains? calls and etc) call on the first map.

[CLJ-1044] Enable refering to ->type inside deftype

Sat, 15 Dec 2012 07:35:38 -0600

Inside defrecord is possible to refer to ->type-ctor, that is not possible inside deftype.
This patch adds an implicit declare, as done in defrecord

[CLJ-939] Exceptions thrown in the top level ns form are reported without file or line number

Sat, 15 Dec 2012 07:50:03 -0600

If there is an error in the `ns` form, an exception is thrown, which is not caught in `load`.

For example, with an invalid :only clause;

(ns clj14.myns
  (:use
   [clojure.core :only seq]))

This generates a Don't know how to create ISeq from: clojure.lang.Symbol exception, with source file or line number.

[CLJ-835] defmulti doc string doesn't mention needing to be passed a var for the value of :hierarchy

Wed, 19 Dec 2012 07:59:17 -0600

The :hierarchy option for defmulti should be passed a var, but this is not mentioned in the doc string.

The error message from passing the hierarchy directly is rather cryptic:

Evaluation aborted on java.lang.ClassCastException: clojure.lang.PersistentArrayMap cannot be cast to clojure.lang.IRef (SOURCE_FORM_44:19).

[CLJ-980] Documentation for extend-type falsely implies that & is allowed in protocol fn signatures

Wed, 19 Dec 2012 08:00:13 -0600

The documentation for extend-type contains the following example:

(extend-type MyType 
    Countable
      (cnt [c] ...)
    Foo
      (bar [x y] ...)
      (baz ([x] ...) ([x y & zs] ...)))

However, [x y & zs] is not a valid parameter list for a protocol fn. The documentation should be appropriately amended.

[CLJ-1137] Metadata on a def gets evaluated twice

Fri, 21 Dec 2012 11:32:28 -0600

Metadata on the symbol of a def special form is evaluated twice.

(def ^{:foo (println "HA")} a [])

prints out HA HA. Offending line is in Compiler$DefExpr, fixed.

[CLJ-1120] Introduce ex-message and ex-cause to abstract away the platform in dealing with ExceptionInfo

Fri, 21 Dec 2012 23:03:36 -0600

As described in the title. See ~~CLJS-429~~.

[CLJ-1104] Concurrent with-redefs do not unwind properly, leaving a var permanently changed

Fri, 21 Dec 2012 23:05:10 -0600

On 1.4 and latest master:

user> (defn ten [] 10)
#'user/ten
user> (doall (pmap #(with-redefs [ten (fn [] %)] (ten)) (range 20 100)))
(20 21 22 23 24 25 34 27 28 29 30 31 32 33 34 35 36 37 38 39 39 35 42 43 44 45 48 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 79 87 88 89 90 91 92 93 94 95 97 92 98 99)
user> (ten)
79

Not sure if this is a bug per se, but the doc doesn't mention lack of concurrency safety and my expectation was that the original value would always be restored after any (arbitrarily interleaved) sequence of with-redefs calls.

[CLJ-1134] star-directive in clojure.pprint/cl-format with an at-prefix ("~n@*") do not obey its specifications

Wed, 26 Dec 2012 19:57:30 -0600

The star-directive in clojure.pprint/cl-format with an at-prefix (~n@*) does not obey its specifications according to Common Lisp the Language, 2nd Edition. There are two bugs within ~n@* as of right now:

When ~n@* is supposed to jump forward over more than one argument, it jumps one step backward as if it had seen ~:*. For instance, (cl-format nil "~D ~3@*~D" 0 1 2 3) will return "0 0" and not "0 3" as expected.
When ~@* is seen, the formatter is supposed to jump to the first argument (as n defaults to 0, see specification linked above). However, whenever a ~@*-directive is seen, the formatter jumps to the second argument instead.

What (small set of) steps will reproduce the problem?

Inside a clean Clojure repl, perform these steps:

user=> (require '[clojure.pprint :refer [cl-format]])
nil
user=> (cl-format nil "~D ~3@*~D" 0 1 2 3)
"0 0"                                           ;; Expected: "0 3"
user=> (cl-format nil "~D~D~D~D ~@*~D" 0 1 2 3)
"0123 1"                                        ;; Expected: "0123 0"

What is the expected output? What do you see instead?

The expected output is "0 3" and "0123 0", but is "0 0" and "0123 1" as shown above.

What version are you using?

Tested on both 1.4.0 and 1.5.0-beta2, both have the defect described.

Please provide any additional information below.

The format strings which reproduces the problem has been compared with the format function from the Common Lisp implementations SBCL, CLisp and Clozure. All of them print the expected output.

[CLJ-1108] Allow to specify an Executor instance to be used with future-call

Thu, 27 Dec 2012 02:25:43 -0600

This adds an arity to future-call that expects a java.util.concurrent/ExecutorService instance to be used instead of clojure.lang.Agent/soloExecutor.

[CLJ-1030] Misleading ClassCastException when coercing a String to int

Sun, 6 Jan 2013 20:52:40 -0600

Observed behaviour

(int "0") =>
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Character

Expected behaviour
(int "0") =>
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer
or
IllegalArgumentException

[CLJ-1086] Support arity-1 for ->>

Sat, 12 Jan 2013 18:48:06 -0600

Currently (as of Clojure 1.4) ->> doesn't support arity-1, though -> does. Arity-1 for ->> would be useful to let somebody comment out forms as follows:

(->> foo
  #_(bar baz)
  #_quux)

Discussion: https://groups.google.com/forum/?fromgroups=#!topic/clojure/_IVl0Tb7Au4

My name on contributor list page http://clojure.org/contributing is: Shantanu Kumar

[CLJ-1148] adds docstring support to defonce, and stops it from stomping existing metadata

Thu, 17 Jan 2013 17:16:40 -0600

Two issues here:

1) defonce doesn't support docstrings, although def does, so it would be nice to bring the two in line with each other

2) defonce can stomp metadata, like this:

(defonce ^:private foo 5)
#'user/foo
(meta #'foo) --> the private is there
(defone foo 10)
foo is still 5
(meta #'foo) --> the private has been lost

[CLJ-1060] 'list*' returns not a list

Fri, 18 Jan 2013 09:41:40 -0600

Function 'list?' returns sequence, but not a list.
It is a bit confusing.

user=> (list? (list* 1 '(2 3)))
false

[CLJ-840] Add a way to access the current test var in :each fixtures for clojure.test

Fri, 18 Jan 2013 09:47:53 -0600

When looking at (log) output from tests written with clojure.test, I would like to be able to identify the output associated with each test. A mechanism to expose the current test var within an :each fixture would enable this.

One mechanism might be to bind a test-var var with the current test var before calling the each-fixture-fn in clojure.test/test-all-vars.

[CLJ-1151] Minor Code Cleanup in core.reducers: use required walk, drop this for coll

Mon, 21 Jan 2013 16:43:30 -0600

First, core.reducers requires clojure.walk :as walk, but does not use the alias.
Second, the two arity implementation of coll-reduce in function reducer uses 'this', whereas similar implementations in that file use 'coll'. AFAICT it makes no difference to use 'coll' (all tests pass, no change in performance) and it is more in line with the rest of the code.

The two things seem small enough to be put into one cleanup case.

[CLJ-1045] Generalize/refactor implementation of PersistentVector/coll-fold

Fri, 25 Jan 2013 14:29:31 -0600

Vector currently contains a specialized implementation of the folding algorithm "split the collection in half until the pieces are small enough". The attached commit lifts out the general strategy so that it can be reused by other collection types amenable to splitting.

CLJ-993 depends on this patch, as it uses the new fold-by-halves function.

[CLJ-1082] Subvecs of primitive vectors cannot be reduced

Sat, 26 Jan 2013 07:29:55 -0600

I could reproduce on current 10/05 git.

Reduce doesn't work on subvecs of primitive vectors.
Root cause seems to be that RT/subvec and APersistentVector$SubVector make assumptions about implementation

If reduce on a subvec doesn't work then neither will nice ops like fold.

How to cause:

(let [prim-vec (into (vector-of :long) (range 10000))]
(reduce + (subvec prim-vec 1 500)))

->> ClassCastException clojure.core.Vec cannot be cast to clojure.lang.PersistentVector clojure.lang.APersistentVector$SubVector.iterator (APersistentVector.java:523)

[CLJ-1096] Make destrucring emit direct keyword lookups

Sat, 26 Jan 2013 08:15:49 -0600

Currently associative destructuring emits calls to get. The attached patch modify desctruture to emit direct keyword lookups when possible.

Approved here https://groups.google.com/d/msg/clojure-dev/MaYcHQck8VA/nauMus4mzPgJ

[CLJ-1080] Eliminate many uses of reflection in Clojure code

Thu, 7 Feb 2013 08:54:50 -0600

There are dozens of uses of reflection in Clojure code that can be eliminated by adding suitable type hints. This patch adds the necessary type hints for most of those.

[CLJ-1079] Don't squash explicit :line and :column metadata in the MetaReader

Thu, 7 Feb 2013 12:35:21 -0600

I have been experimenting with using cljx to produce Clojure and ClojureScript source from a single file. This has gone well so far, with the exception that, due to the way the source transformation works, all of the linebreaks and other formatting is gone from the output. There is an option to include the original :line metadata in the output though, like so:

;;This file autogenerated from 
;;
;;  src/cljx/com/foo/hosty.cljx
;;
^{:line 1} (ns com.foo.hosty)
^{:line 3} (defn ^{:clj true} system-hash [x] ^{:line 5} (System/identityHashCode x))

(Hopefully, such hackery won't be necessary in the future with sjacket or something like it...)

Unfortunately, when read in using a LineNumberingPushbackReader, code like this has its :line metadata squashed by the line numbers coming from that. A REPL-friendly example would be:

=> (meta (read (clojure.lang.LineNumberingPushbackReader.
                 (java.io.StringReader. "^{:line 66} ()"))))
{:line 1}
=> (meta (read (java.io.PushbackReader.
                 (java.io.StringReader. "^{:line 66} ()"))))
{:line 66}

The latter seems more correct to me (and is equivalent to read-string).

[CLJ-706] make use of deprecated namespaces/vars easier to spot

Wed, 13 Feb 2013 00:38:18 -0600

From the mailing list http://groups.google.com/group/clojure/msg/c41d909bd58e4534. It is easy to use deprecated namespaces without knowing you are doing so. The documentation warnings are small, and there is no compiler warning.

Some possibilities include:

much more visible deprecation styling in the documentation
stderr warnings when referencing a deprecated thing.

I don't love the idea of stderr warnings on all the time. Rich: is there an approach to this that you would like to see a patch for?

[CLJ-771] Move unchecked-prim casts to clojure.unchecked

Wed, 13 Feb 2013 00:51:11 -0600

Per Rich's comment in ~~CLJ-767~~:

Moving unchecked coercions into unchecked ns is ok

[CLJ-949] let undeclared exceptions continue unchecked

Fri, 1 Mar 2013 09:37:59 -0600

The recent modifications regarding checked exceptions have eliminated the need for several try/catch blocks. This commit removes the blocks that no longer serve a purpose.

[CLJ-1160] reducers/mapcat ignores Reduced

Fri, 1 Mar 2013 09:40:09 -0600

The following code throws an exception:

(->> (concat (range 100) (lazy-seq (throw (Exception. "Too eager"))))
(r/mapcat (juxt inc str))
(r/take 5)
(into []))

This is because r/mapcat introduces an intermediate reduce which swallows the reduced value coming from r/take.

[CLJ-394] Add marker Interfaces for defrecords and deftypes plus boolean test fns

Fri, 1 Mar 2013 09:41:09 -0600

Sometimes one would like to know if an object is an instance of a deftype or a defrecord.
Add (possibly empty) marker Interfaces that allow an efficient test, plus test fns
'record?' and 'deftype?'.

[CLJ-858] Improve speed of STM by removing System.currentTimeMillis

Fri, 1 Mar 2013 09:41:33 -0600

Original post: https://groups.google.com/d/topic/clojure/kc99LcUK8Tk/discussion

This patch removes the milliseconds from inner class TVal in LockingTransaction.java and Ref.java. Using a little test suite[1] a increase of performance by up to 25% could be measured.

If necessary I can recreate the patch against current MASTER.

References:
[1] https://github.com/ska2342/clj-stm-perf-test/

[CLJ-970] extend/implement parameterized types (generics)

Fri, 1 Mar 2013 09:46:59 -0600

When extending parameterized types, class files can track the original signatures of the superclass and super interfaces so that the original types can be obtained at run time. This runtime reflection is used in some Java frameworks, and implementing it in Clojure can enable interop. See http://groups.google.com/group/clojure/browse_thread/thread/5efd692804df3f47/1336e591c2eedfa1 for examples of this request.

This proposal checks the :parameters keyword in type meta information. If a parameter is found, it is added to the class signature.

[CLJ-1073] Make print-sequential interruptible

Fri, 1 Mar 2013 10:18:01 -0600

This allows print-sequential to be interrupted by Thread.interrupt(), rather than requiring clients to resort to Thread.stop(). This is especially helpful when printing very large sequences.

See also clojure-dev discussion at https://groups.google.com/d/topic/clojure-dev/vs0RNUQXiYE/discussion

[CLJ-1076] pprint tests fail on Windows, expecting \n

Sat, 2 Mar 2013 14:51:46 -0600

New pprint tests were committed recently, but they fail on Windows because the tests check for \n, while pprint seems to output \r\n. A log with the test failures is attached.

The first failing commit is https://github.com/clojure/clojure/commit/4ca0f7ea17888ba7ed56d2fde0bc2d6397e8e1c0

[CLJ-991] partition-by reducer

Mon, 4 Mar 2013 14:49:42 -0600

partition-by reducer

[CLJ-1180] defprotocol doesn't resolve tag classnames

Sun, 10 Mar 2013 17:56:10 -0500

defprotocol doesn't resolve tag classnames, this results in exceptions being thrown when the declared protocol uses as a tag an imported class that is not imported in the namespace that uses it.

user=> (import 'clojure.lang.ISeq)
clojure.lang.ISeq
user=> (defprotocol p (^ISeq f [_]))
p
user=> (ns x)
nil
x=> (defn x [y] (let [z (user/f y)] (inc z)))
CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: ISeq, compiling:(NO_SOURCE_PATH:4:33)

[CLJ-1177] clojure.java.io/resource and non-ASCII characters

Sun, 10 Mar 2013 17:56:52 -0500

clojure.java.io/resource corrupts path containing UTF-8 characters without issuing warning.

user=> (System/getProperty "java.runtime.version")
"1.8.0-ea-b79"
user=> (clojure-version)
"1.5.0"
user=> (System/getProperty "user.dir")
"/dir/déf"
user=> (clojure.java.io/resource "myfile.txt")
#<URL file:/dir/d%c3%a9f/resources/myfile.txt>
user=> (slurp (clojure.java.io/resource "myfile.txt") :encoding "UTF-8")
FileNotFoundException /dir/dÃ©f/resources/myfile.txt (No such file or directory)  java.io.FileInputStream.open (FileInputStream.java:-2)

[CLJ-1169] Add filename and line number to defn parameter declaration error

Sun, 10 Mar 2013 17:58:32 -0500

When mistyping parameter list in defn declaration, e.g.

(defn test
(some-error))

error message shows name of parameter (without quotes), but not function name, filename or line number:

Exception in thread "main" java.lang.IllegalArgumentException: Parameter declaration some-error should be a vector
at clojure.core$assert_valid_fdecl.invoke(core.clj:6567)
at clojure.core$sigs.invoke(core.clj:220)
at clojure.core$defn.doInvoke(core.clj:294)
at clojure.lang.RestFn.invoke(RestFn.java:467)
at clojure.lang.Var.invoke(Var.java:427)
at clojure.lang.AFn.applyToHelper(AFn.java:172)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.lang.Compiler.macroexpand1(Compiler.java:6366)
at clojure.lang.Compiler.macroexpand(Compiler.java:6427)
at clojure.lang.Compiler.eval(Compiler.java:6495)
at clojure.lang.Compiler.load(Compiler.java:6952)
at clojure.lang.Compiler.loadFile(Compiler.java:6912)
at clojure.main$load_script.invoke(main.clj:283)
at clojure.main$init_opt.invoke(main.clj:288)
at clojure.main$initialize.invoke(main.clj:316)
at clojure.main$null_opt.invoke(main.clj:349)
at clojure.main$main.doInvoke(main.clj:427)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)

[CLJ-713] Upgrade ASM to a more current version

Mon, 11 Mar 2013 15:43:51 -0500

Move to the latest ASM (currently 3.3.1)
No longer subset-ed
Still re-rooted it under clojure.asm

[CLJ-1093] Empty record literals gets incorrectly evaluated to array-maps

Wed, 13 Mar 2013 14:04:29 -0500

before patch:
user=> (defrecord a [b])
user.a
user=> #user.a{}
{}

after patch:
user=> (defrecord a [b])
user.a
user=> #user.a{}
#user.a{:b nil}

[CLJ-1004] ArrayChunk implements Seqable

Mon, 25 Mar 2013 23:10:02 -0500

I've found it helpful to be able to iterate over ArrayChunks. Implementing Seqable means they can be used by most collections functions.

[CLJ-1026] Mixed end-of-line endings in the source code

Sat, 30 Mar 2013 08:44:15 -0500

While examining some of the Clojure source code, I discovered that some files had mixed line endings, or CRLF line endings on a non-Windows box. Using .gitattributes, we can correct that so that files have the right endings for the platform that it's on.

[CLJ-1165] Forbid varargs defprotocol/definterface method declarations because those cannot be defined anyway

Thu, 4 Apr 2013 19:24:27 -0500

Protocol, interface method declarations don't allow for varags. Currently, for example

  (defprotocol FooBar
    (foo [this & more]))

compiles just fine, and & is interpreted as a usual argument that happens to be
named & without special meaning. But clearly, the user wanted to specify a
varags parameter here. The same applies to definterface.

Similarly, providing method implementations via defrecord, deftype, and reify
don't allow for varags (but dynamic extensions via extend do).

So this patch makes defprotocol and definterface throw an
IllegalArgumentException if a user tries to use varargs in method signatures.

Similarly, defrecord, deftype, and reify throw an IllegalArgumentException if
any method implementation arglist contains a varargs argument.

This patch is a cut-down variant of my patch to http://dev.clojure.org/jira/browse/CLJ-1024
which has been reverted shortly before Clojure 1.5 was released. The ~~CLJ-1024~~ patch
was the same as this one, but it has also forbidden destructuring in defprotocol and
definterface. This was a bit too much, because although destructuring has no
semantic meaning with method declarations, it still can serve a documentation purpose.

This has been discussed on the list: https://groups.google.com/d/topic/clojure-dev/qjkW-cv8nog/discussion

[CLJ-1191] Improve apropos to show some indication of namespace of symbols found

Fri, 5 Apr 2013 13:34:10 -0500

apropos does find all symbols in all namespaces that match the argument, but the return value gives no clue as to which namespace the found symbols are in. It can even return multiple occurrences of the same symbol, which only gives a clue that the symbol exists in more than one namespace, but not which ones. For example:

user=> (apropos "replace")
(postwalk-replace prewalk-replace replace re-quote-replacement replace replace-first)

It would be nice if the returned symbols could indicate the namespace, either always, or if the found symbol is not in the current namespace.

[CLJ-976] Implement reader literal and print support for PersistentQueue data structure

Sat, 6 Apr 2013 08:07:20 -0500

Clojure's PersistentQueue structure has been in the language for quite some time now and has found its way into a fair share of codebases. However, the creation of queues is a two step operation often of the form:

(conj clojure.lang.PersistentQueue/EMPTY :a :b :c)

;=> #<PersistentQueue clojure.lang.PersistentQueue@78d5f6bc>

A better experience might be the following:

#queue [:a :b :c]

;=> #queue [:a :b :c]

(pop #queue [:a :b :c])

;=> #queue [:b :c]

This syntax is proposed and discussed in the Clojure-dev group at https://groups.google.com/forum/?fromgroups#!topic/clojure-dev/GQqus5Wycno

Open question: Should the queue literal's arguments eval? The implications of this are illustrated below:

;; non-eval case
#queue [1 2 (+ 1 2)]

;=> #queue [1 2 (+ 1 2)]


;; eval case
#queue [1 2 (+ 1 2)]

;=> #queue [1 2 3]

The answer to this open question will determine the implementation.

[CLJ-937] cl-format prints ratio arguments with bad format for E, F, G directives

Mon, 8 Apr 2013 12:02:21 -0500

user=> (use 'clojure.pprint)
nil
user=> (cl-format false "~e" 4/5)
"4./5E+2"
user=> (cl-format false "~f" 4/5)
"4/5.0"
user=> (cl-format false "~g" 4/5)
"4/5. "

Patch changes cl-format so that when E, F, or G directive is used, the corresponding arg is coerced from a clojure.lang.Ratio to a double before formatting.

[CLJ-1197] Allow fold to parallelize over lazy sequences

Wed, 10 Apr 2013 10:34:09 -0500

This patch implements foldable-seq, which allows fold to parallelize over a lazy sequence. See this conversation on the Clojure mailing list:

https://groups.google.com/forum/#!msg/clojure/8RKCjF00ukQ/b5mmmOB5Uh4J

The patch is code only, sadly. No tests because I've not been able to find any existing tests for fold:

https://groups.google.com/d/msg/clojure-dev/plQ16L1_FC0/CIyMVIgSZkkJ

However, I have tested it in a separate project successfully.

[CLJ-1185] `reductions should respect `reduced

Wed, 10 Apr 2013 17:08:45 -0500

This returns 16:

(reduce (fn [acc x]
          (let [x' (* x x)]
            (if (> x' 10)
              (reduced x')
              x')))
        (range))

But replacing `reduce with `reductions will never terminate:

(reductions (fn [acc x]
              (let [x' (* x x)]
                (if (> x' 10)
                  (reduced x')
                  x')))
            (range))

This is because `reductions ignores 'clojure.lang.Reduced, it never tests for `reduced?

I know that I only just discovered the `reduced, function, but `reductions is a big part of my debugging process, so it's unfortunate that they don't work together.

[CLJ-935] clojure.string/trim uses different defn of whitespace as triml, trimr

Thu, 11 Apr 2013 17:36:32 -0500

clojure.string/triml and trimr use Character/isWhitespace to determine whether a character is whitespace, but trim uses some other definition of white space character. For example:

user=> (use 'clojure.string)
nil
user=> (def s " \u2002 foo")
#'user/s
user=> (trim s)
"? foo"
user=> (triml s)
"foo"

The attached patch changes trim to use Character/isWhitespace. I suppose other possibilities are to change triml and trimr to use trim's notion of whitespace, whatever that is, or to just leave these functions inconsistent with each other. It does seem that it would be a nice property that (trim s) is equal to (triml (trimr s)) for all strings.

The patch also changes triml to only call .length on s once.

[CLJ-1005] Use transient map in zipmap

Thu, 11 Apr 2013 18:19:28 -0500

The attached patch changes zipmap to use a transient map internally. The definition is also moved so that it resides below that of #'transient. The original definition is commented out (like that of #'into).

[CLJ-1036] Util/hasheq should be hashing a BigInteger to the same values as Long, and BigInt

Fri, 12 Apr 2013 08:48:23 -0500

The doc string for hash states that it defines a hash code function that is consistent with =, but for java.math.BigInteger hash is not consistent with =.

user=> (apply = [(Long. -1) -1N (biginteger -1)])
true
user=> (map hash [(Long. -1) -1N (biginteger -1)])
(0 0 -1)
user=>

It is possible to have a PHM with two key/value pairs where the keys are equal, and the hash codes are different:

user=> (assoc clojure.lang.PersistentHashMap/EMPTY (biginteger -1) :oops! -1N :one)
{-1N :one, -1 :oops!}

The expected behavior is the same as PersistentArrayMap, which does not have this issue, because it does not hash its keys:

user=> (assoc clojure.lang.PersistentArrayMap/EMPTY (biginteger -1) :oops! -1N :one)
{-1 :one}

This same misbehavior also occurs for Doubles and Floats:

thalia.core=> (apply = [(Float. 1e9) (Double. 1e9)])
true
thalia.core=> (map hash [(Float. 1e9) (Double. 1e9)])
(1315859240 1104006501)

That leads to the same difference in array-map and hash-map behavior as above for BigInteger and BigInt.

[CLJ-1157] Classes generated by gen-class aren't loadable from remote codebase for mis-implementation of static-initializer

Fri, 12 Apr 2013 09:19:05 -0500

When a client program uses a remote service which uses RMI, and the service returns a object which created with gen-class with clojure as the return value, the return value is not loadable at client side.

At client side, a following exeption will be thrown.

Exception in thread "main" java.lang.ExceptionInInitializerError
        at java.io.ObjectStreamClass.hasStaticInitializer(Native Method)
        at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1723)
        at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:69)
        at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:247)
        at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:245)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:244)
        at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:600)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
        at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:324)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:173)
        at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194)
        at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148)
        at $Proxy0.getResult(Unknown Source)
        at client.SampleClient$_main.doInvoke(SampleClient.clj:12)
        at clojure.lang.RestFn.invoke(RestFn.java:397)
        at clojure.lang.AFn.applyToHelper(AFn.java:159)
        at clojure.lang.RestFn.applyTo(RestFn.java:132)
        at client.SampleClient.main(Unknown Source)
 Caused by: java.io.FileNotFoundException: Could not locate remoteserver/SampleInterfaceImpl__init.class or remoteserver/SampleInterfaceImpl.clj on classpath: 
        at clojure.lang.RT.load(RT.java:434)
        at clojure.lang.RT.load(RT.java:402)
        at clojure.core$load$fn__5039.invoke(core.clj:5520)
        at clojure.core$load.doInvoke(core.clj:5519)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.lang.Var.invoke(Var.java:415)
        at remoteserver.SampleInterfaceImpl.<clinit>(Unknown Source)
        ... 23 more

HOW TO REPRODUCT THIS ISSUE

If you want to see this issue at your computer, clone my example project from my github.

git clone git://github.com/tyano/clojure_genclass_fix.git

and build them (You must have installed Leiningen 2):

cd clojure_genclass_fix
sh build.sh

start rmiregistry:

rmiregistry &

start remoteserver:

cd remoteserver
sh start.sh

You will see a message "Server ready. " or "Server ready. (rebind)".

At last, start client program:

cd ../client
sh start.sh

Without my patch, you will see a same Exception described above. But with clojure with my patch, you will see a right response message: "response = this is sample."

THE REASON

The reason of this problem is in bytecodes generated by gen-class. A gen-classed class (in this case, SampleInterfaceImpl.class) uses a static-initializer for loading SampleInterfaceImpl__init.class (which load other classes which implements functions in the class). The static-initializer is like bellow: (the following code is decompiled with JD - http://java.decompiler.free.fr/?q=jdgui )

static
  {
    RT.var("clojure.core", "load").invoke("/remoteserver/SampleInterfaceImpl");
  }

Very simple code. it seems non-problematic. But RT.load changes the classloader for loading __init.class in the processing! RT.load in default uses a context-classloader for loading classes. But all classes depending on a gen-classed class must be loaded a same classloader with a main gen-classed class. In this case, RT.load must use a remote URLClassLoader which load a main class.

So, gen-class must be create bytecodes that is same with the following java code.

static
  {
    Var.pushThreadBindings(RT.map(new Object[] { Compiler.LOADER, SampleInterfaceImpl.class.getClassLoader() }));
    try {
        RT.var("clojure.core", "load").invoke("/remoteserver/SampleInterfaceImpl");
    }
    finally 
    {
        Var.popThreadBindings();
    }
  }

With this code, RT.load will uses a same classloader which load SampleInterfaceImpl.class.
My patch for gen-class will create bytecodes equal to the above example.

You can use an attached patch '20130204_fix_classloader.diff', or pull 'fix_classloader' branch from my github repositry ( git@github.com:tyano/clojure.git ).

[CLJ-196] file returns "NO_SOURCE_PATH", but the doc says it should be nil

Fri, 12 Apr 2013 10:05:40 -0500

According to http://clojure.org/api, *file* should return nil in the repl, but it returns "NO_SOURCE_PATH".

This has been true for a long time. Latest patch changes only the docstring of *file* to accurately reflect current behavior.

[CLJ-1182] Regexp are never equal

Sat, 13 Apr 2013 21:13:52 -0500

The following (= #"asd" #"asd") will return false in CLJ, even if they are the same.

Consequently, everything with a regexp in it (lists, vectors, maps...) will never be equal.

It seems to have been fixed for CLJS, but not for CLJ.
https://github.com/clojure/clojurescript/commit/e35c3a57472fa62ae41591418a73794dc8ac6dde

[CLJ-1204] hash is inconsistent with = for many BigInteger values

Thu, 25 Apr 2013 18:42:40 -0500

The function hash is documented to be consistent with =. For many BigInteger values, hash is inconsistent with =. This leads to incorrect behavior for data structures like hash maps that use hash.

user> (apply = [-1 -1N (biginteger -1)])
true
user> (map hash [-1 -1N (biginteger -1)])
(0 0 -1)

;; Incorrect return value with multiple keys = to each other
user> (assoc (hash-map -1N :should-be-replaced) (biginteger -1) :new-val)
{-1N :should-be-replaced, -1 :new-val}

;; array-map gives correct value, since it uses =, not hash
user> (assoc (array-map -1N :should-be-replaced) (biginteger -1) :new-val)
{-1N :new-val}

[CLJ-1118] inconsistent numeric comparison semantics between BigDecimal and other Numerics

Thu, 25 Apr 2013 19:58:50 -0500

user> clojure-version
{:major 1, :minor 5, :incremental 0, :qualifier "beta1"}
user> (== 2.0 2.0M)
true
user> (== 2 2.0M)
false <-- this one is not like the others
user> (== 2 2.0)
true
user> (== 2N 2.0)
true
user> (== 2 (double 2.0M))
true

It's not clear if this is a bug or an enhancement request, Should BigDecimal's be special in comparason to their smaller equivalents?

[CLJ-428] subseq, rsubseq enhancements to support priority maps?

Wed, 1 May 2013 02:44:36 -0500

See dev thread at http://groups.google.com/group/clojure-dev/browse_thread/thread/fdb000cae4f66a95.

Note: subseq currently returns () instead of nil in some situations. If the rest of this idea dies we might still want to fix that.

[CLJ-766] Implicit casting behaviour of into-array differs from -array

Fri, 3 May 2013 20:46:29 -0500

Current patch: byte-short-array-ctors.diff

The behavior of byte-array and short-array is inconsistent with the other <type>-array functions and with the into-array function when invoked with types other than byte or short. All of the other cases upcast to Number, then extract the primitive value, allowing this operation to succeed (assuming the value is in range). byte-array and short-array throw a ClassCastException.

Example:

base64v3a=> (into-array Byte/TYPE [1 2 3 4])  ;; int to byte ok
#<byte[] [B@5ee04fd>
base64v3a=> (byte-array [1 2 3 4])  ;; int to byte NOT ok!! 
ClassCastException java.lang.Long cannot be cast to java.lang.Byte
  clojure.lang.Numbers.byte_array (Numbers.java:1418)
base64v3a=> (long-array [1 2 3 4])  ;; int to long ok
#<long[] [J@3f9f4d1d>

into-array (via RT.seqToTypedArray) and the other <type>-array functions all upcast to Number (via Numbers.<type>_array}}), then obtain the proper primitive value. Numbers.byte-array and Numbers.short-array do casts directly to Byte and Short (yielding the ClassCastException).

The attached patch makes the Byte and Short cases match the other types and the into-array behavior. Tests are included. The submitter is a contributor.

[CLJ-866] Provide a clojure.test function to run a single test case with fixtures

Sat, 4 May 2013 09:36:20 -0500

At present, clojure.test test cases are functions and can be invoked directly. However, in the case that the test relies on fixtures, this does not work. Please provide a function that can run a single test case with all fixtures applied.

[CLJ-1202] protocol fns with dashes may get compiled into property access when used higher order

Fri, 10 May 2013 15:19:29 -0500

user=> (defprotocol Foo (-foo [x]))
Foo
user=> (deftype Bar [] Foo (-foo [_] :baz))
user.Bar
user=> (map -foo [(Bar.)])
IllegalArgumentException No matching field found: foo for class user.Bar  
clojure.lang.Reflector.getInstanceField (Reflector.java:271)

I would have expected to see (:baz). The full stack is:

IllegalArgumentException No matching field found: foo for class user.Bar
	clojure.lang.Reflector.getInstanceField (Reflector.java:271)
	clojure.lang.Reflector.invokeNoArgInstanceMember (Reflector.java:300)
	user/eval79/fn--80/G--71--82 (NO_SOURCE_FILE:11)
	user/eval79/fn--80/G--70--85 (NO_SOURCE_FILE:11)
	clojure.core/map/fn--4207 (core.clj:2485)
	clojure.lang.LazySeq.sval (LazySeq.java:42)
	clojure.lang.LazySeq.seq (LazySeq.java:60)
	clojure.lang.RT.seq (RT.java:484)
	clojure.core/seq (core.clj:133)
	clojure.core/print-sequential (core_print.clj:46)
	clojure.core/fn--5406 (core_print.clj:143)
	clojure.lang.MultiFn.invoke (MultiFn.java:231)
nil

I suspect this is somehow related to the property access changes to make Clojure/ClojureScript compatible. I was in fact prepping core.logic for a unified code base and was adopting the ClojureScript protocol naming convention when I encountered this issue.

~~CLJ-872~~ added dash property support to Clojure.

[CLJ-1209] Teach clojure.test reporting about ex-info/ex-data

Sat, 11 May 2013 04:12:35 -0500

When clojure.test/deftest does error reports on unexpected exceptions it currently ignores ExceptionInfo and the valuable ex-data it carries. So this patch simple prints this data, it might be helpful to pprint or format it in another way but this was good enough for me.

See example from my tests: https://gist.github.com/thheller/5559391

[CLJ-1117] Partition does not follow docs

Fri, 17 May 2013 10:14:56 -0500

The doc for partition states "In case there are not enough padding elements, return a partition with less than n items."

However, the behavior of this function is as follows:

user=> (partition 3 (range 10))
((0 1 2) (3 4 5) (6 7 8))
user=> (partition 4 (range 10))
((0 1 2 3) (4 5 6 7))
user=> (partition 5 (range 10))
((0 1 2 3 4) (5 6 7 8 9))

Either the doc should be updated to make it clear that not providing a pad will mean that items are dropped, or the functionality of partition should be fixed to the following:

user=> (partition 3 (range 10))
((0 1 2) (3 4 5) (6 7 8) (9))

[CLJ-1122] Add contributing.md file to github repository (shows clear message on issues/pull request create form)

Fri, 24 May 2013 08:24:38 -0500

This adds a clear message when someone wants to create a pull request/issue and invites the user to read the contribution guidelines: see https://github.com/blog/1184-contributing-guidelines.

The same thing could be done for all the clojure/* repositories.

The content of the file is just a markdown version of http://clojure.org/contributing

Preview here: https://github.com/mpenet/clojure/blob/aef170ca5eca1b71a2eb1ef320223d1277df0e5e/CONTRIBUTING.md

[CLJ-1121] -> and ->> have unexpected behavior when combined with unusual macros

Fri, 24 May 2013 08:48:42 -0500

My intuitive understanding of the classic threading macros is that the meaning of forms like (-> a b c) can be understood syntactically independent of the meaning of the symbols involved or the fact that the two threading macros are defined recursively. However the recursive definition breaks that expectation. After

(macroexpand-1 (macroexpand-1 '(-> a b c)))

=> (c (-> a b))

c is now in control if it is a macro, and is now seeing the argument (-> a b) rather than (b a) as would be the case if we had written (c (b a)) originally.

Admittedly I do not know of a realistic example where this is an important distinction (I noticed this when playing with a rather perverse use of ->> with macros from korma), but at the very least it means that the behavior of the threading macros isn't quite as easy to accurately explain as I thought it was.

[CLJ-944] compiler makes different concrete maps then the reader

Fri, 24 May 2013 09:03:46 -0500

I (Stu) agree with Nicola's assessment in the comments: the single real problem here is that the compiler's MapExpr parser emits maps differently from other map-makers, e.g. RT's factory functions.

Patch clj944-plus-tests does three things:

has the compiler emit map types consistent with the reader (like the previous "0002" patch)
adds tests
removes broken tests

Original report follows:

Hi guys, I am getting the following exception:

(.containsKey {:one 1} :one)
;=> ClassCastException clojure.lang.PersistentArrayMap cannot be cast to clojure.lang.PersistentHashMap

The map is a clojure.lang.PersistentArrayMap, which obviously has a containsKey method (https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentArrayMap.java#L95).

Casting it works fine though:

(.containsKey ^clojure.lang.PersistentArrayMap {:one 1} :one)
;=> true

The mailing list suggest that the compiler injects an incorrect cast to clojure.lang.PersistentHashMap. In this case it should probably be cast to a clojure.lang.Associative, the highest common interface having the .containsKey method.

The problem is not present in Clojure 1.2.1.

[CLJ-1187] Clojure loses quoted metadata on empty literals

Fri, 24 May 2013 09:14:44 -0500

user=> (meta '^:foo [])
nil

while
user=> (meta '^:foo [1])
{:foo true}

This bug propagates to ^:const vars:
user=> (def ^:const foo ^:foo [])
#'user/foo
user=> (meta foo)
nil
user=> (meta @#'foo)
{:foo true}

[CLJ-1176] clojure.repl/source fails when read-eval bound to :unknown

Fri, 24 May 2013 09:42:41 -0500

clojure.repl/source is broken in Clojure 1.5.0 when *read-eval* is bound to :unknown, since source-fn reads without binding.

Reproduce:

Either set

:jvm-opts ["-Dclojure.read.eval=unknown"]

in Leiningen or eval at REPL:

(alter-var-root #'*read-eval* (constantly :unknown))

(use 'clojure.repl)
(source drop-last)

Expected:

Source of drop-last.

Actual:

RuntimeException Reading disallowed - *read-eval* bound to :unknown

[CLJ-1175] NPE in clojure.lang.Delay/deref

Fri, 24 May 2013 11:02:21 -0500

Summary: Delays get into a corrupt state if an exception thrown, allowing deref to behave differently on back-to-back calls. Patch causes Delay to rethrow the original exception, includes tests.

If a Delay wraps a function which throws an exception, then forcing that delay multiple times causes behavior which is, to me at least, surprising and unexpected: the first time it is forced, the expected exception is thrown; but after that it will behave as if all locals the expression refers to are nil. This can manifest in multiple ways, depending on the expression being delayed:

;; calling a local as a function causes an NPE inside clojure.lang.Delay
(let [f #(/ 1 0) d (delay (f))]
  [(try (force d) 
        (catch Exception e e))
   (try (force d)
        (catch Exception e e))])
[#<ArithmeticException java.lang.ArithmeticException: Divide by zero> 
 #<NullPointerException java.lang.NullPointerException>]

;; if nil is a valid value, this can cause subsequent forces to "succeed"
;; even though the first fails as it should
(let [x (java.util.Date.) d (delay (seq x))]
  [(try (force d) 
        (catch Exception e e))
   (try (force d)
        (catch Exception e e))])
[#<IllegalArgumentException java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.util.Date> 
 nil]

The reason for this is that clojure.core/delay creates a ^:once function, promising to call it only once, so that the compiler can do locals-clearing on its lexical closure. However, when an exception is thrown the Delay object is left thinking it has never called the function, so when it is forced again it calls the function again, breaking its promise to the compiler and causing the function to run after its locals have been cleared.

In fact, once we realize that locals-clearing is involved, we can make the delay behave differently the first N times it is forced, instead of only the first time, by constructing an expression which throws an exception before using all of its locals:

(let [x (java.util.Date.) 
            y (Long. 1)
            d (delay (let [y (seq y)]
                       (cons (seq x) y)))]
  [(try (force d) 
        (catch Exception e e))
   (try (force d)
        (catch Exception e e))
   (try (force d)
        (catch Exception e e))])
[#<IllegalArgumentException java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long> 
 #<IllegalArgumentException java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.util.Date>
 (nil)]

I'm not sure what the right fix for this issue is: perhaps the best choice is even to leave it alone, and just document that delay's behavior is undefined if the expression throws an exception. However, I propose making the Delay remember the first exception that was thrown, and rethrow it on subsequent force attempts. This makes sense, in a way: the "result" of the expression was this exception e being thrown, and so this should happen every time. It might be preferable to have the Delay retry the expression until it succeeds, but I don't believe this is possible without abandoning locals-clearing, which would cause perfectly-valid delays to now run out of memory, holding onto no-longer-needed locals just in case the expression needs to retry at some later date.

[CLJ-1161] sources jar has bad versions.properties resource

Fri, 24 May 2013 11:13:59 -0500

Summary: patch leaves version.properties file out of sources JAR, where it causes problems for tools.

The "sources" jar (at least since Clojure 1.4 and including 1.5 RC) has a bad version.properties file in it. The resource clojure/version.properties is literally:

version=${version}

The regular Clojure jar has the correct version string in that resource.

I came across a problem when I was experimenting with the sources jar (as used by IDEs). I naively added the sources jar to my classpath, and Clojure died on start up. The bad clojure/versions.properties file was found first, which led to a parse error as the clojure version was being set.

[CLJ-1171] Compiler macro for clojure.core/instance? disregards lexical shadows on class names

Fri, 24 May 2013 11:29:14 -0500

Summary

The compiler tries to emit jvm native instanceof expressions for direct clojure.core/instance? calls.
For that, it tries to resolve its first argument as a class name. However, it disregards lexical bindings when doing that.
This is incongruent to the default implementation in core.clj

Patches

[Stu] All three patches should be applied IMO.

0002 makes instance? respect lexical bindings
0003 makes instance?'s compiled form check arity, consistent with higher-order behavior
0001 has a minimal test for both 0002 and 0003.

Data

Test case

user=> (let [Long String] (instance? Long "abc"))
false
;; expected true as in
user=> (let [Long String] (apply instance? [Long "abc"]))
true

Culprit method

https://github.com/clojure/clojure/blob/4ccb10edbe66eae81336a4c0972050d9759b0bf7/src/jvm/clojure/lang/Compiler.java#L3578

List Discussion

https://groups.google.com/d/topic/clojure/mf25OlFRpa8/discussion

Tangent

This was discovered because the same compiler macro also omits the arity check implicit in the default definition. This could also conveniently be fixed when touching that method:

user=> (instance? String)
false
;; expected
user=> (apply instance? [String])
ArityException Wrong number of args (1) passed to: core$instance-QMARK-  clojure.lang.AFn.throwArity (AFn.java:437)

EDIT elaborated on ticket title and description; added tangent

[CLJ-1193] bigint, biginteger throw on double values outside of long range

Fri, 24 May 2013 11:46:12 -0500

This works fine:

user> (bigint (* 0.99 Long/MAX_VALUE))
9131138316486227968N

but this and any other Float or Double values outside the range of a
long throw an exception:

user> (bigint (* 1.01 Long/MAX_VALUE))
IllegalArgumentException Value out of range for long: 9.315605757223324E18 clojure.lang.RT.longCast (RT.java:1178)

Similarly for biginteger

[CLJ-850] Hinting the arg vector of a primitive-taking fn with a non-primitive type results in AbstractMethodError when invoked

Fri, 24 May 2013 12:12:20 -0500

Summary: The CLJ-850-conform-to-invokePrim.diff patch is constructed per Rich's feedback, and appears good to me [Stu].

See the following examples:

user=> (defn f1 ^String [^String s] s)
#'user/f1
user=> (f1 "foo")
"foo"
user=> (defn f2 ^long [^String s ^long i] i)
#'user/f2
user=> (f2 "foo" 1)
1
user=> (defn f3 ^String [^String s ^long i] s)                                       
#'user/f3
user=> (f3 "foo" 1)
AbstractMethodError user$f3.invokePrim(Ljava/lang/Object;J)Ljava/lang/Object;  user/eval8 (NO_SOURCE_FILE:6)

[CLJ-1107] 'get' should throw exception on non-Associative argument

Fri, 24 May 2013 12:31:42 -0500

The implementation of clojure.core/get returns null if its argument is not a valid associative collection. However, calling 'get' on something which is neither nil nor an Associative collection is almost certainly a bug, and should be indicated by an exception.

This behavior can obscure common programmer errors such as:

(def a (atom {:a 1 :b 2})

(:foo a)   ; forgot to deref a
;;=> nil

~~CLJ-932~~ was accepted as a similar enhancement to 'clojure.core/contains?'

Attached patch 0001 throws an IllegalArgumentException as the fall-through case of RT.getFrom.

[CLJ-669] clojure.java.io/do-copy: use java.nio for Files

Fri, 24 May 2013 12:45:25 -0500

NIO Channels reduce CPU/Disk load when copying Files (by using
syscalls like sendfile internally on Linux/Solaris).

CPU-Load goes from 100% to 0% on my system when copying large files, because no userspace-copying is involved:

Patch: http://github.com/juergenhoetzel/clojure/commit/2b5ab103cbcfe6c49236ac6966c032d3c922233d

[CLJ-322] Enhance AOT compilation process to emit classfiles only for explicitly-specified namespaces

Fri, 24 May 2013 13:25:25 -0500

Summary: still needs decision on implementation approach.

This was originally/erroneously reported by Howard Lewis Ship in the clojure-contrib assembla:

My build file specifies the namespaces to AOT compile but if I include another namespace
(even from a JAR dependency) that is not AOT compiled, the other namespace will be compiled as well.

In my case, I was using clojure-contrib's clojure.contrib.str-utils2 namespace, and I got a bunch of
clojure/contrib/str_utils2 classes in my output directory.

I think that the AOT compiler should NOT precompile any namespaces that are transitively reached,
only namespaces in the set specified by the command line are appropriate.

As currently coded, you will frequently find unwanted third-party dependencies in your output JARs;
further, if multiple parties depend on the same JARs, this could cause bloating and duplication in the
eventual runtime classpath.

Having the option of shipping either all AOT-compiled classfiles or mixed source/AOT depending upon one's distribution requirements would make that phase of work with a clojure codebase significantly easier and less error-prone. The only question in my mind is what the default should be. We're all used to the current behaviour, but I'd guess that any nontrivial project where the form of the distributable matters (i.e. the source/AOT mix), providing as much control as possible by default makes the most sense. Given the tooling that most people are using, it's trivial (and common practice, IIUC) to provide a comprehensive list of namespaces one wishes to compile, so making that the default shouldn't be a hurdle to anyone. If an escape hatch is desired, a --transitive switch to clojure.lang.Compile could be added.

[CLJ-1074] Read/print round-trip for +/-Infinity and NaN

Sat, 25 May 2013 12:39:35 -0500

A few float-related forms (namely, Double/POSITIVE_INFINITY, Double/NEGATIVE_INFINITY, Double/NaN) are not eval-able after a round-trip via

(read-string (binding [*print-dup* true] (pr-str f))

The two options I see are to provide print-method implementations for these and their Float cousins, or to make Infinity, -Infinity, +Infinity, and NaN readable values. Since it sounds like edn may want to provide a spec for these values (see https://groups.google.com/d/topic/clojure-dev/LeJpOhHxESs/discussion and https://github.com/edn-format/edn/issues/2), I think making these values directly readable as already printed is preferable. Something like Double/POSITIVE_INFINITY seems too low-level from edn's perspective, as it would refer to a Java class and constant.

I'm attaching a patch implementing reader support for Infinity, -Infinity, +Infinity, and NaN.

[CLJ-873] Allow the function / to be referred to in namespaces other than clojure.core

Sat, 25 May 2013 14:05:26 -0500

The attached patch gives the programmer the option of referring to the division function in namespaces other than just clojure.core. For example,

(ns foo
(:require [cljs.core :as core]))
(apply core// '(1 2 3))

The above lines do not compile without this patch.

[CLJ-787] transient blows up when passed a vector created by subvec

Tue, 28 May 2013 11:50:25 -0500

Subvectors created with subvec from a PersistentVector cannot be made transient:

user=> (transient (subvec [1 2 3 4] 2))
ClassCastException clojure.lang.APersistentVector$SubVector cannot be cast to clojure.lang.IEditableCollection  clojure.core/transient (core.clj:2864)

[CLJ-1189] Map-destructuring :or fumble needs compiler warning

Tue, 28 May 2013 11:52:12 -0500

Here is a map-destructuring blunder that I wish the compiler warned about:

(defn server
[{servlet ::servlet
type ::type
:or {::type :jetty}
:as service-map}]

It would be splendid to get a warning that :or keys that are not symbols being bound have no effect.

The incomplete code snippet above comes from Pedestal.service 0.1.0.

Here is a complete one-line example with the coding error:

user> (defn picnic [{botulism :botulism :or {:botulism 6}}] botulism)
#'user/picnic
user> (picnic {})
nil
user> ;; I intended 6.

[CLJ-1162] Error Message when calling deref on a non-IDeref is unhelpful

Tue, 28 May 2013 11:54:55 -0500

If you just type "@1" is the repl, on previous versions you'll get an error that Long cannot be cast to IDeref. In 1.5, the error message is that it cannot be cast to java.util.concurrent.Future. This is because it assumes that anything that isn't an IDeref is automatically a Future. The deref method should generate a custom error stating that the type you've passed in is neither an IDeref nor a Future.

[CLJ-1078] Added queue, queue* and queue? to clojure.core

Fri, 31 May 2013 10:05:34 -0500

Original patch added functions for PersistentQueue. queue, queue? and queue* match the list functions of the same naming conventions. Patches include updates to tests.

In the updated patch, "queue" accepts a single collection argument as per Rich's suggestion, and does not provide a "queue*".

[CLJ-1083] Incorrect ArityException message for function names containing ->

Fri, 7 Jun 2013 14:01:34 -0500

user=> (defn a->b [])
#'user/a->b
user=> (a->b 1)
ArityException Wrong number of args (1) passed to: user$a clojure.lang.AFn.throwArity (AFn.java:437)

Note that the reported function name in the stack trace is "user$a", where it should be "user$a->b" (or some mangled variant thereof?)

See discussion here: https://groups.google.com/d/msg/clojure/PVNoLclhhB0/_NWqyE0cPAUJ

[CLJ-1200] RestFn & ArraySeq performance

Fri, 7 Jun 2013 19:35:06 -0500

I was profiling one of my projects and noticed a hotspot inside functions using rest arguments.

Most overloads of RestFn.invoke will construct an ArraySeq object. The ArraySeq constructor calls java.lang.Class.getComponentType, which seems to be pretty slow. The array's component type is cached in a field on the ArraySeq object for the sole purpose of passing it to Reflector.prepRet. I'm not entirely sure of the total utility of prepRet, but it's clearly a no-op when passed Object.class, such as the case with the non-specialized base class of ArraySeq: the component type of object[] is Object.class.

The attached patch eliminates the component type field from ArraySeq and passes Object.class to prepRet directly. It may be possible to eliminate calls to prepRet all together, but I'll assume that's a different ticket. With this patch, ArraySeq initialization no longer shows up as a hotspot when profiling.

Before the patch:

user=> (dotimes [n 10] (time (dotimes [i 5000000] ((fn [& args]) 1 2 3 4))))
"Elapsed time: 874.742 msecs"
"Elapsed time: 900.277 msecs"
"Elapsed time: 911.164 msecs"
"Elapsed time: 872.132 msecs"
"Elapsed time: 885.495 msecs"
"Elapsed time: 897.537 msecs"
"Elapsed time: 879.691 msecs"
"Elapsed time: 888.52 msecs"
"Elapsed time: 871.556 msecs"
"Elapsed time: 1088.682 msecs"

After the patch:

user=> (dotimes [n 10] (time (dotimes [i 5000000] ((fn [& args]) 1 2 3 4))))
"Elapsed time: 628.144 msecs"
"Elapsed time: 634.163 msecs"
"Elapsed time: 617.397 msecs"
"Elapsed time: 622.714 msecs"
"Elapsed time: 646.743 msecs"
"Elapsed time: 648.708 msecs"
"Elapsed time: 629.223 msecs"
"Elapsed time: 638.058 msecs"
"Elapsed time: 725.473 msecs"
"Elapsed time: 636.909 msecs"

That's about a 30% reduction in execution time.

Granted it only represents a change of 52 nanoseconds per RestFn invoke (181 ns to 129 ns), but this actually was a pretty decent win for a library that makes makes almost exclusively variadic function calls in a tight loop.

[CLJ-1216] Evaling ((fn [do] do) 1) returns nil while ((fn [do] do do) 1) returns 1

Sun, 9 Jun 2013 16:31:22 -0500

user=> ((fn [do] do) 1)
nil

user=> ((fn [do] (do do)) 1)
1

user=> ((fn [] do))
nil

user=> ((fn [] do do))
CompilerException java.lang.RuntimeException: Unable to resolve symbol: do in this context, compiling:(NO_SOURCE_PATH:0:0)

[CLJ-1184] Evaling #{do ...} or [do ...] is treated as the do special form

Sun, 9 Jun 2013 16:31:55 -0500

Evaluating a persistent collection for which the function 'first' returns the symbol 'do' leads to that collection being treated as the do special form, even though it may be a vector or even a set. IMHO, the expected result would be to report that 'do' cannot be resolved. For the cause, see the if condition on line 6604 of Compiler.java in clojure-1.5.1.

E.g.:
[do 1 2]
;=> 2

#{"hello" "goodbye" do}
;=> "hello"
; Wat?

[CLJ-827] unsigned-bit-shift-right

Mon, 10 Jun 2013 01:57:52 -0500

Add a clojure equivalent of >>>.

A simple version of this is implemented here (https://github.com/joegallo/clojure/tree/unsigned-bit-shift-right), and just follows the example set by shift-right.

The downside of this implementation is that it treats all integer types as longs, and shifts them accordingly, which yields different results than you would get in java. A previous version of this did not have the same problem, when BitOps was its own thing. I'm not sure if this limitation is acceptable and appropriate, or needs to be worked around (my inclination is the latter).

[CLJ-1125] Clojure can leak memory when used in a servlet container

Fri, 14 Jun 2013 10:56:13 -0500

When used within a servlet container
(Jetty/Tomcat/JBossAS/Immutant/etc), the thread locals Var.dvals (used
to store dynamic bindings) and LockingTransaction.transaction (used to
store the currently active transaction(s)) prevent all of the classes
loaded by an application's clojure runtime from being garbage collected,
resulting in a memory leak.

The issue comes from threads living beyond the lifetime of a
deployment - servlet containers use thread pools that are shared
across all applications within the container. Currently, the dvals and
transaction thread locals are not discarded when they are no longer
needed, causing their contents to retain a hard reference to their
classloaders, which, in turn, causes all of the classes loaded under
the application's classloader to be retained until the thread exits
(which is generally at JVM shutdown).

I've attached a patch that does the following:

Var.dvals is now removed when the thread bindings are popped
Var.dvals no longer has an initialValue, so checking to see if it is
set will no longer set it to an empty Frame
The outer transaction in LockingTransaction.transaction now removes
the thread local when it is finished

There is still the opportunity for memory leaks if agents or futures
are used, and the executors used for them are not shutdown when the
app is undeployed. That's a solvable problem, but should probably be
solved by the containers themselves (and/or the war generation tools)
instead of in clojure itself.

This patch has a small performance impact: its use of a try/finally
around running transactions to remove the outer transaction adds
4-6 microseconds to each transaction call on my hardware.

Providing an automated test for this patch is difficult - I've tested
it locally with repeated deployments to a container while monitoring
GC and permgen. All of clojure's tests pass with it applied.

The above is a condensation of:
https://groups.google.com/d/topic/clojure-dev/3CXDe8_9G58/discussion

I'm happy to provide whatever feedback/work is needed to get this
applied.

[CLJ-1217] for consumes sequence argument more eagerly than necessary

Fri, 14 Jun 2013 14:36:08 -0500

In a call like (do (for [x (do (println "realized") nil)] x) nil), no elements of the for comprehension are ever requested, and so it is not actually necessary to evaluate the inner do-block. However, this expression causes "realized" to be printed, because the first sequence-expression in for is evaluated even if no items are ever requested from the output lazy-seq.

It's not documented whether this is intended or unintentional, but I was surprised by this behavior, and a brief unscientific survey on #clojure suggests that other users, even "old hands" who've been using clojure for years, don't expect this either.

I've attached a patch that wraps the problematic expression in a lazy-seq call. This is not quite ideal, because it means that the first iteration is "lazied" twice, as in ((fn step [s] (lazy-seq ...)) (lazy-seq xs)), but a change to make this not happen would be much broader in scope, and this seemed the least dangerous.