Clojure JIRA

[UNIFY-2] Remove reflection warnings

Thu, 5 Jan 2012 07:37:14 -0600

Reflection warning, clojure/core/unify.clj:30 - reference to field getClass can't be resolved.
Reflection warning, clojure/core/unify.clj:30 - reference to field isArray can't be resolved.

These reflective calls occur frequently enough that they should be resolved.

[TBENCH-16] Add performance testing for STM

Thu, 23 Feb 2012 14:38:00 -0600

Write a test-suite to detect regressions in STM performance.

[LOGIC-50] Rel relation PersistentHashSet becomes LazySeq after issuing a retraction

Wed, 5 Sep 2012 11:32:52 -0500

The first retraction of facts from a relation transforms the internal PersistentHashSet -set var+atom, which guarantees uniqueness, into a LazySeq which allows duplicate facts to be added. Once the first retraction occurs, a LazySeq persists for the rest of the life of the relation. Only the value of the primary -set var+atom is affected, not the -index var+atom values.

This issue is not an external correctness issue but does affect performance of subsequent duplicate fact additions which grow the size relation.

Preparation:

user=> (defrel foo x y)
#<user$eval4287$fn__4288 user$eval4287$fn__4288@1a9d489b>
user=> (facts foo [[:joe :man][:jane :woman][:sue :boy]])
nil
user=> foo_2-set
#<Atom@52aaf223: #{[:joe :man] [:jane :woman] [:sue :boy]}>
user=> (class @foo_2-set)
clojure.lang.PersistentHashSet
user=> (retraction foo :jane :woman)
nil

Without patch applied:

user=> foo_2-set
#<Atom@52aaf223: ([:joe :man] [:sue :boy])>
user=> (class @foo_2-set)
clojure.lang.LazySeq
user=> (facts foo [[:joe :man][:jane :woman][:sue :boy]])
nil
user=> foo_2-set
#<Atom@52aaf223: ([:sue :boy] [:jane :woman] [:joe :man] [:joe :man] [:sue :boy])>
user=> (class @foo_2-set)
clojure.lang.LazySeq

With patch applied:

user=> foo_2-set
#<Atom@31eb9b15: #{[:joe :man] [:sue :boy]}>
user=> (class @foo_2-set)
clojure.lang.PersistentHashSet
user=> (facts foo [[:joe :man][:jane :woman][:sue :boy]])
nil
user=> foo_2-set
#<Atom@31eb9b15: #{[:joe :man] [:jane :woman] [:sue :boy]}>
user=> (class @foo_2-set)
clojure.lang.PersistentHashSet

I've filed this as a Minor issue as it does not affect core.logic correctness and degraded performance can be avoided by not re-asserting duplicate facts.

I will also issue a GitHub pull request which can be used or ignored at your convenience.

[JDBC-31] distinct? throws clojure.lang.ArityException, when applied with no arguments

Sun, 10 Jun 2012 17:24:42 -0500

HSQLDB returns an empty ResultSet when using (.getGeneratedKeys stmt)
and no keys are generated. So this Exception is thrown for each record
without generated keys.

While this Exception is caught in do-prepared-return-keys, this can lead to a huge overhead caused by the JVM exception handling. I did a performance test.

Before Patch:

clojure.java.test-jdbc> (time (sql/with-connection hsqldb-db (count (apply sql/insert-records :dummy  (map #(hash-map :name (str %) :id %) (range 10000))))))
"Elapsed time: 3429.346743 msecs"
10000

After Patch:

 clojure.java.test-jdbc> (time (sql/with-connection hsqldb-db (count (apply sql/insert-records :dummy  (map #(hash-map :name (str %) :id %) (range 10000))))))
"Elapsed time: 1397.444753 msecs"

[JDBC-29] Performance improvement (Remove intermediate lazy sequence in resultset-seq)

Thu, 10 May 2012 12:09:00 -0500

This improves performance on large result sets by up to 30%.

[CLJ-1090] Indirect function calls through Var instances fail to clear locals

Fri, 30 Nov 2012 14:26:45 -0600

If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error

I can provide a patch if it would be useful. The fix should be trivial, something along the lines of wrapping each argN in clojure/lang/Var.java inside a Util.ret1(argN, argN = null) as is done in RestFn.java.

[CLJ-1087] clojure.data/diff uses set union on key seqs

Mon, 15 Oct 2012 14:52:42 -0500

clojure.data/diff, on line 118, defines:

java.util.Map
(diff-similar [a b]
  (diff-associative a b (set/union (keys a) (keys b))))

Since keys returns a key seq, this seems like an error. clojure.set/union has strange and inconsistent behavior with regard to non-sets, and in this case the two key seqs are concatenated. Based on a cursory benchmark, it seems that this bug is a slight performance gain when the maps have no common keys, and a significant performance loss when the maps have the same keys. The results are still correct because of the merging reduce in diff-associative.

The patch is easy (just call set on each key seq).

[CLJ-1050] Remove a lock in the common case of deref of delay

Tue, 18 Sep 2012 06:53:01 -0500

Currently, when deref is called in Delay.java, a lock on the Delay is always acquired.
This is wasteful as most of the time you just want to read the val.

The attached patch changes this behaviour to the following:

val is initialised to a known secret value. (During its constructor so it is visible from any thread).
When deref is called, val is checked to the known secret value.

If it is not the secret value, then it has to be the value computed by the function and we return it.
If it is the secret value, then we lock this object and revert to the current behaviour.

This is faster than what is done currently and can be very much faster if there is contention on the delay.

[CLJ-1000] Performance drop in PersistentHashMap.valAt(...) in v.1.4 -- Util.hasheq(...) ?

Fri, 1 Mar 2013 09:49:21 -0600

It seems there is a 30-40% performance degradation of PersistentHashMap.valAt(...) in Clojure 1.4.
Possibly due to references to new CPU-hungry implementation of Util.hasheq(...).

I have created a demo project with more details and some profiling information here:
https://github.com/oshyshko/clj-perf

[CLJ-988] the locking in MultiFn.java (synchronized methods) can cause lots of contention in multithreaded programs

Fri, 21 Sep 2012 14:21:33 -0500

if you call a single multimethod a lot in multithreaded code you get lots of contention for the lock on the multimethod. this contention slows things down a lot.

this is due to getMethod being synchronized. it would be great if there was some fast non-locking path through the multimethod.

[CLJ-910] [Patch] Allow for type-hinting the method receiver in memfn

Sat, 1 Sep 2012 08:37:48 -0500

The attached patch copies metadata given to the method name symbol of memfn to the method receiver in the expansion. That way, memfn is able to be used even for type-hinted calls resulting in a big performance win.

user> (time (dotimes [i 1000000] ((memfn intValue) 1)))
Reflection warning, NO_SOURCE_FILE:1 - call to intValue can't be resolved.
"Elapsed time: 2579.229115 msecs"
nil
user> (time (dotimes [i 1000000] ((memfn ^Number intValue) 1)))
"Elapsed time: 12.015235 msecs"
nil

[CCACHE-27] Missing LU and LRU is linear complexity - performance

Wed, 12 Sep 2012 09:27:56 -0500

Profiling some code with YourKit showed a hotspot in cache statistics on (miss) for LU and LRU caches.

Basically two issues: (count (keys {....})) is a count of a seq, not efficient. Replaced with a count of the map.

Secondly, (apply f anything) is slow. Profiler showed that the (apply min-key) was really slow. This is mitigated by using a c.d.priority-map instead. On a priority-map, (ffirst {}) or (first (peek {}).

Also inverted the logic for threshold comparison. Since there is a (build-leastness-queue) that populates statistics, should caches should favor codepaths for the state of being full all the time?

[ALGOM-4] algo.monad state-m fetch-val bug and efficiency issue

Sat, 8 Sep 2012 20:47:32 -0500

;; the bug

(defn fetch-val
"Return a state-monad function that assumes the state to be a map and
returns the value corresponding to the given key. The state is not modified."
[key]
(domonad state-m
[s (fetch-state)]
(key s))) ;; does not work for integer or string keys

;; I propose replacing it with (get s key)

;; the efficiency issue :
;;
;; domonad with monad parameter binds all the monad functions,
;; looking these up in the state-m map on each call
;;
;; solution :

(defn fetch-val
"Return a state-monad function that assumes the state to be a map and
returns the value corresponding to the given key. The state is not modified."
[key]
(fn [s]
[(get s key) s]))

;; - we avoid the monad map lookups
;; - coding style brought up to par with the rest of state-m functions