Clojure JIRA

[CLJ-1203] Fallback to hash-based comparison for non-Comparables in Util.compare()

Mon, 29 Apr 2013 02:30:08 -0500

I oftentimes use sorted collections, and my comparator functions usually look like that:

(fn [a b]
  (let [x (compare-according-to-my-use-case a b)]
    (if (zero? x)
       (compare a b)
       x)))

That is, I define the sorting order depending on the requirements of my use-case, and if that doesn't suffice to make a distinction, I simply fall back to the standard `compare` function in order to get a stable ordering anyhow. I think that's a widely used idiom, e.g., also used in "Clojure Programming" (for example page 109).

The problem with this approach is that several data structures don't implement Comparable, and thus aren't applicable to `compare` (you'll get a ClassCastException). Examples are maps, records, deftypes, and sequences.

The patch I'm going to attach extends `Util.compare()` with a fallback clause that performs a `hasheq()`-based comparison. This results in a meaningless but at least stable sorting order which suffices for use-cases like the one shown above.

[CLJ-1200] RestFn & ArraySeq performance

Thu, 18 Apr 2013 16:03:25 -0500

I was profiling one of my projects and noticed a hotspot inside functions using rest arguments.

Most overloads of RestFn.invoke will construct an ArraySeq object. The ArraySeq constructor calls java.lang.Class.getComponentType, which seems to be pretty slow. The array's component type is cached in a field on the ArraySeq object for the sole purpose of passing it to Reflector.prepRet. I'm not entirely sure of the total utility of prepRet, but it's clearly a no-op when passed Object.class, such as the case with the non-specialized base class of ArraySeq: the component type of object[] is Object.class.

The attached patch eliminates the component type field from ArraySeq and passes Object.class to prepRet directly. It may be possible to eliminate calls to prepRet all together, but I'll assume that's a different ticket. With this patch, ArraySeq initialization no longer shows up as a hotspot when profiling.

Before the patch:

user=> (dotimes [n 10] (time (dotimes [i 5000000] ((fn [& args]) 1 2 3 4))))
"Elapsed time: 874.742 msecs"
"Elapsed time: 900.277 msecs"
"Elapsed time: 911.164 msecs"
"Elapsed time: 872.132 msecs"
"Elapsed time: 885.495 msecs"
"Elapsed time: 897.537 msecs"
"Elapsed time: 879.691 msecs"
"Elapsed time: 888.52 msecs"
"Elapsed time: 871.556 msecs"
"Elapsed time: 1088.682 msecs"

After the patch:

user=> (dotimes [n 10] (time (dotimes [i 5000000] ((fn [& args]) 1 2 3 4))))
"Elapsed time: 628.144 msecs"
"Elapsed time: 634.163 msecs"
"Elapsed time: 617.397 msecs"
"Elapsed time: 622.714 msecs"
"Elapsed time: 646.743 msecs"
"Elapsed time: 648.708 msecs"
"Elapsed time: 629.223 msecs"
"Elapsed time: 638.058 msecs"
"Elapsed time: 725.473 msecs"
"Elapsed time: 636.909 msecs"

That's about a 30% reduction in execution time.

Granted it only represents a change of 52 nanoseconds per RestFn invoke (181 ns to 129 ns), but this actually was a pretty decent win for a library that makes makes almost exclusively variadic function calls in a tight loop.

[CLJ-1165] Forbid varargs defprotocol/definterface method declarations because those cannot be defined anyway

Thu, 4 Apr 2013 19:24:27 -0500

Protocol, interface method declarations don't allow for varags. Currently, for example

  (defprotocol FooBar
    (foo [this & more]))

compiles just fine, and & is interpreted as a usual argument that happens to be
named & without special meaning. But clearly, the user wanted to specify a
varags parameter here. The same applies to definterface.

Similarly, providing method implementations via defrecord, deftype, and reify
don't allow for varags (but dynamic extensions via extend do).

So this patch makes defprotocol and definterface throw an
IllegalArgumentException if a user tries to use varargs in method signatures.

Similarly, defrecord, deftype, and reify throw an IllegalArgumentException if
any method implementation arglist contains a varargs argument.

This patch is a cut-down variant of my patch to http://dev.clojure.org/jira/browse/CLJ-1024
which has been reverted shortly before Clojure 1.5 was released. The ~~CLJ-1024~~ patch
was the same as this one, but it has also forbidden destructuring in defprotocol and
definterface. This was a bit too much, because although destructuring has no
semantic meaning with method declarations, it still can serve a documentation purpose.

This has been discussed on the list: https://groups.google.com/d/topic/clojure-dev/qjkW-cv8nog/discussion

[CLJ-1161] sources jar has bad versions.properties resource

Fri, 12 Apr 2013 09:36:58 -0500

The "sources" jar (at least since Clojure 1.4 and including 1.5 RC) has a bad version.properties file in it. The resource clojure/version.properties is literally:

version=${version}

The regular Clojure jar has the correct version string in that resource.

I came across a problem when I was experimenting with the sources jar (as used by IDEs). I naively added the sources jar to my classpath, and Clojure died on start up. The bad clojure/versions.properties file was found first, which led to a parse error as the clojure version was being set.

[CLJ-1112] Var loading-verbosely should initialize from a JVM system property

Thu, 22 Nov 2012 02:14:32 -0600

I often find myself adding :verbose to a (require) or (use) clause of my (ns) in order to debug problems (especially macros, or bad namespace declarations). It would be very nice if I could define a JVM system property (say -Dclojure.load-verbosely=true) to default loading-verbosely to true for a REPL session, or as part of a build.

Sometimes I just like to see that namespaces load as a measure of progress, when starting an application, or when running a set of tests.

[CLJ-1094] Zero-arity versions of every-pred and some-fn

Thu, 25 Oct 2012 07:12:35 -0500

This patch adds zero-arity versions of every-pred and some-fn with these semantics.

(every-pred) === (constantly true)
(some-fn)    === (constantly nil)

These variants are useful in situations like the following:

;; compute-preds-for may return zero or many predicate fns
(let [preds (compute-preds-for something)]
  (filter (apply every-pred preds) some-coll))

[CLJ-1074] Read/print round-trip for +/-Infinity and NaN

Mon, 3 Dec 2012 13:18:40 -0600

A few float-related forms (namely, Double/POSITIVE_INFINITY, Double/NEGATIVE_INFINITY, Double/NaN) are not eval-able after a round-trip via

(read-string (binding [*print-dup* true] (pr-str f))

The two options I see are to provide print-method implementations for these and their Float cousins, or to make Infinity, -Infinity, +Infinity, and NaN readable values. Since it sounds like edn may want to provide a spec for these values (see https://groups.google.com/d/topic/clojure-dev/LeJpOhHxESs/discussion and https://github.com/edn-format/edn/issues/2), I think making these values directly readable as already printed is preferable. Something like Double/POSITIVE_INFINITY seems too low-level from edn's perspective, as it would refer to a Java class and constant.

I'm attaching a patch implementing reader support for Infinity, -Infinity, +Infinity, and NaN.

[CLJ-1073] Make print-sequential interruptible

Fri, 1 Mar 2013 10:18:01 -0600

This allows print-sequential to be interrupted by Thread.interrupt(), rather than requiring clients to resort to Thread.stop(). This is especially helpful when printing very large sequences.

See also clojure-dev discussion at https://groups.google.com/d/topic/clojure-dev/vs0RNUQXiYE/discussion

[CLJ-1066] No dependency on jsr166y

Sat, 20 Oct 2012 09:53:42 -0500

The Clojure 1.5.0-alpha4 jars that have been deployed to maven.org seem to have been compiled against JDK6 which causes
an exception if one tries to use reducers/fold.

— snip —
Setup:

user=> (require '[clojure.core.reducers :as r])
nil
user=> (def v (vec (range 10000)))
#'user/v

;; JDK7 + clojure 1.5.0-alpha4 from maven.org
;; → :dependencies [[org.clojure/clojure "1.5.0-alpha4"]] in project.clj

user=> (r/fold + (r/map inc v))
ClassNotFoundException jsr166y.ForkJoinTask java.net.URLClassLoader$1.run (URLClassLoader.java:366)

;; JDK7 + clojure 1.5.0-alpha4 from maven.org
;; → :dependencies [[org.clojure/clojure "1.5.0-alpha4"]
;; [org.codehaus.jsr166-mirror/jsr166y "1.7.0"]]

user=> (r/fold + (r/map inc v))
50005000

;; JDK7 + clojure 1.5.0-alpha4 locally compiled (mvn install) against OpenJDK7
;; → :dependencies [[org.clojure/clojure "1.5.0-alpha4"]] in project.clj

user=> (r/fold + (r/map inc v))
5000050000
— snip —

It would be wonderful if this issue could be fixed before the release of 1.5.0.

Have a nice day

Wolodja

[CLJ-1061] when-first double evaluation

Sat, 20 Oct 2012 09:53:42 -0500

The when-first macro will evaluate the xs expression twice. Admittedly, it does exactly what the doc string says, but that seems undesirable to me. Even without side effects, there's a potential performance issue if xs is some expensive operation.

Patch attached. The main diff is:

- `(when (seq ~xs)
- (let [~x (first ~xs)]
- ~@body))))
+ `(when-let xs# (seq ~xs)
+ (let ~x (first xs#)
+ ~@body))))

[CLJ-1050] Remove a lock in the common case of deref of delay

Tue, 18 Sep 2012 06:53:01 -0500

Currently, when deref is called in Delay.java, a lock on the Delay is always acquired.
This is wasteful as most of the time you just want to read the val.

The attached patch changes this behaviour to the following:

val is initialised to a known secret value. (During its constructor so it is visible from any thread).
When deref is called, val is checked to the known secret value.

If it is not the secret value, then it has to be the value computed by the function and we return it.
If it is the secret value, then we lock this object and revert to the current behaviour.

This is faster than what is done currently and can be very much faster if there is contention on the delay.

[CLJ-1042] [PATCH] Allow negative substring indices for (subs)

Wed, 19 Sep 2012 13:34:50 -0500

This adds Python-style negative string indices for (subs), e.g.:

(subs "foo bar" -3) ;; -> "bar"

[CLJ-1024] Varargs protococol impls can be defined but not called

Thu, 14 Feb 2013 10:03:07 -0600

The compiler accepts this:

(deftype foo []
clojure.lang.IFn
(invoke [this & xs]))

However calling ((foo.) :bar) will throw an AbstractMethodError. Wouldn't some checking be desirable?

[CLJ-1018] range's behavior is inconsistent

Fri, 12 Apr 2013 09:59:12 -0500

Problem statement: The current behavior of range is inconsistent. (range 0 9 0) has always produced (). (range 0 9 -1) has always produced (). (range 9 0 1) has always produced (). However, (range 9 0 0) produces (9 9 9 9 ...), and (range 0 0 0) produces '(0 0 0 0 ...)

Proposal: Make the behavior of range consistent when using a step of 0 to make it produce an empty list.

Please see attached code and patch.

[CLJ-1011] clojure.data/diff should cope with null and false values in maps

Sat, 18 Aug 2012 07:12:52 -0500

Current behaviour of clojure.data/diff:

=> (diff {:a false} {:a true})
(nil {:a true} nil)
=> (diff {:a false} {:a nil})
(nil nil nil)

Proposed behaviour:

=> (diff {:a false} {:a true})
({:a false} {:a true} nil)
=> (diff {:a false} {:a nil})
({:a false} {:a nil} nil)

[CLJ-1010] A left-to-right-variant of `comp`

Thu, 14 Jun 2012 18:27:23 -0500

The function composition function `comp` is quite inefficient in cases like `(apply comp large-seq-of-fns)`, because its arity-greater-than-3 version reverses the seq.

I would be great if there was an alternative `comp*` (or whatever) function which is just like `comp` but composes left-to-right.

[CLJ-991] partition-by reducer

Mon, 4 Mar 2013 14:49:42 -0600

partition-by reducer

[CLJ-990] Implement clojure.core.reducers/mapcat and some initial reducers tests

Fri, 1 Mar 2013 09:49:21 -0600

The first patch implements a reducers equivalent of clojure.core/mapcat, and the second patch adds a new reducers.clj to the clojure tests that checks that map, mapcat, filter, and reduce are equivalent in clojure.core and clojure.core.reducers.

[CLJ-987] pprint doesn't flush the underlying stream

Fri, 1 Mar 2013 09:49:20 -0600

Unlike prn, pprint doesn't flush the underlying stream.

pretty_writer currently overrides .flush with behaviour that pushes its own data, but does not flush the underlying stream.

pprint should behave similarly to prn with respect to flushing the underlying stream.

[CLJ-982] Implement an interface? predicate to balance class?

Sat, 9 Jun 2012 09:08:46 -0500

clojure.core implements a class? predicate to detect if a thing is an instance of java.lang.Class. The attached patch implements interface? to further distinguish classes that are also interfaces. This gives us Clojure api for e.g., enumerating all interfaces a thing's base class implements; as in, (filter #(interface? %) (supers (class my-thing))).

[CLJ-960] Capture :column metadata (needed for ClojureScript source maps)

Sat, 20 Oct 2012 09:53:42 -0500

I've begun working on implementing SourceMaps for ClojureScript. For an overview of SourceMaps, see: http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/ For discussion of the feature in ClojureScript, see: https://groups.google.com/d/topic/clojure-dev/zgmXO2iM1JQ/discussion

In order to produce accurate source maps, I need column information in addition to line information from the Clojure reader.

I've made the necessary enhancement to LispReader, etc. but have some cleanup and testing left to do. I'd also like a sanity check from the core team before attaching a formal patch. You can find my work in progress here: https://github.com/brandonbloom/clojure/compare/columns

Clojure JIRA

[CLJ-1203] Fallback to hash-based comparison for non-Comparables in Util.compare()

[CLJ-1200] RestFn & ArraySeq performance

[CLJ-1165] Forbid varargs defprotocol/definterface method declarations because those cannot be defined anyway

[CLJ-1161] sources jar has bad versions.properties resource

[CLJ-1112] Var *loading-verbosely* should initialize from a JVM system property

[CLJ-1094] Zero-arity versions of every-pred and some-fn

[CLJ-1074] Read/print round-trip for +/-Infinity and NaN

[CLJ-1073] Make print-sequential interruptible

[CLJ-1066] No dependency on jsr166y

[CLJ-1061] when-first double evaluation

[CLJ-1050] Remove a lock in the common case of deref of delay

[CLJ-1042] [PATCH] Allow negative substring indices for (subs)

[CLJ-1024] Varargs protococol impls can be defined but not called

[CLJ-1018] range's behavior is inconsistent

[CLJ-1011] clojure.data/diff should cope with null and false values in maps

[CLJ-1010] A left-to-right-variant of `comp`

[CLJ-991] partition-by reducer

[CLJ-990] Implement clojure.core.reducers/mapcat and some initial reducers tests

[CLJ-987] pprint doesn't flush the underlying stream

[CLJ-982] Implement an interface? predicate to balance class?

[CLJ-960] Capture :column metadata (needed for ClojureScript source maps)

[CLJ-1112] Var loading-verbosely should initialize from a JVM system property