Clojure JIRA

[MTOWER-3] BigDecimal: unnecessary reflection used in math functions for coercion (bigint)

Mon, 13 May 2013 14:48:19 -0500

This improves performance, because no reflection has to be used (bigint):

Patched:
clojure.math.numeric-tower> (time (let [b (BigDecimal. "2.2")] (dotimes [x 10000000] (round b))))
"Elapsed time: 1640.177041 msecs"
Orig:
clojure.math.numeric-tower> (time (let [b (BigDecimal. "2.2")] (dotimes [x 10000000] (round b))))
"Elapsed time: 3498.339271 msecs"

[LOGIC-64] Support inequalities in finite domain sugar

Sat, 27 Oct 2012 11:33:23 -0500

The attached patch enables the follow code:

(run* [x]
  (infd x (interval 0 9))
  (eqfd (!= 6 (* 2 x))))

You can also substitute != with <, >, >=, or <=

[JDBC-31] distinct? throws clojure.lang.ArityException, when applied with no arguments

Sun, 10 Jun 2012 17:24:42 -0500

HSQLDB returns an empty ResultSet when using (.getGeneratedKeys stmt)
and no keys are generated. So this Exception is thrown for each record
without generated keys.

While this Exception is caught in do-prepared-return-keys, this can lead to a huge overhead caused by the JVM exception handling. I did a performance test.

Before Patch:

clojure.java.test-jdbc> (time (sql/with-connection hsqldb-db (count (apply sql/insert-records :dummy  (map #(hash-map :name (str %) :id %) (range 10000))))))
"Elapsed time: 3429.346743 msecs"
10000

After Patch:

 clojure.java.test-jdbc> (time (sql/with-connection hsqldb-db (count (apply sql/insert-records :dummy  (map #(hash-map :name (str %) :id %) (range 10000))))))
"Elapsed time: 1397.444753 msecs"

[JDBC-29] Performance improvement (Remove intermediate lazy sequence in resultset-seq)

Thu, 10 May 2012 12:09:00 -0500

This improves performance on large result sets by up to 30%.

[DXML-1] Stack overflow when parsing huge XML file

Tue, 20 Mar 2012 08:05:15 -0500

This is using Ryan Senior's new 0.0.3-SNAPSHOT.

While trying to parse a huge XML file (7.5 GB compressed, a dump of Wikipedia), got a stack overflow error. Some digging turned up this bug:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6440214

Modifying clojure.data.xml/source-seq to disable the IS_COALESCING property got rid of the error.

The old lazy-xml contrib code worked (although used up tons more memory).

Attached is a patch that adds keyword options to source-seq, parse, and parse-str, allowing the consumer to disable coalescing and sidestep the upstream bug.

[CLJS-441] Require implicit 'do nodes for all blocks in AST

Fri, 21 Dec 2012 17:36:42 -0600

Discussion here: https://groups.google.com/d/topic/clojure-dev/6CVTsW1EQn4/discussion

[CLJS-440] Replace :is-loop flag in AST with distinct :loop op

Fri, 21 Dec 2012 17:12:07 -0600

Discussion here: https://groups.google.com/d/topic/clojure-dev/6CVTsW1EQn4/discussion

[CLJS-437] Missing "Too few arguments to if" check

Fri, 7 Dec 2012 00:03:55 -0600

(if) and (if 1) both return nil. JVM clojure throws "Too few arguments to if".

[CLJS-432] Include line and file information in error messages

Fri, 30 Nov 2012 11:25:39 -0600

Just as the ana/warning function did, now errors and assertions include line and file source information. I needed this sorely.

[CLJS-431] namespace and name fail on '/

Fri, 30 Nov 2012 11:56:54 -0600

Both the 'namespace and the 'name functions use .lastIndexOf to search for "/". This fails on the division symbol if you don't provide a starting index to search from.

[CLJS-424] Internal representation of keywords can not be differentiated by printable characters

Sat, 22 Dec 2012 15:45:34 -0600

Currently, Symbols are represented by (str \uFDD1 \' name) and keywords are represented by (str \uFDD0 \' name)

The \uFDDX character is not printable, so it appears as a little box when printed in a javascript repl such as the browser's console. The following character is always a \' regardless of whether the object is a symbol or a keyword. This has bitten me during debugging on several occasions. The attached patch changes keywords to be represented internally by (str \uFFD1 \: name)

[CLJS-416] emit-block doesn't use context argument

Fri, 21 Dec 2012 17:30:54 -0600

emit-block currently has arguments [context statements ret] but the context argument is never used and all call sites always draw statements and ret from the same block AST node. The attach patch simplifies emit-block and all call sites to use arguments [{:keys [statement ret]}]

[CLJS-409] Small code cleanup for emitting goog.provide statements

Sat, 27 Oct 2012 16:52:59 -0500

Just abstracts out a duplicated piece of code.

[CLJS-408] Include :form on fn :methods in AST

Sat, 27 Oct 2012 16:52:42 -0500

All core CLJS AST nodes include a :form key. Several quasi-nodes similarly include their originating code forms. Although :fn nodes include their :form, it varies between single and multiple arity functions. I've found it easier to assume multiple arities and always analyze :methods directly. Unfortunately, the methods lack :form keys. This simple patch exposes the form of each method, simplifying function transformations.

[CLJS-387] Add docstring from def and ns definitions to @namespaces metadata map, and make reflect functions make use of that

Wed, 17 Oct 2012 10:57:56 -0500

The docstrings were parsed from the definitions-forms for def and ns, but not added to the @namespaces metadata map.
There is no :doc entry used in the ns' metadata in the @namespaces.
No ns's :doc info is communicated to the browser in the reflect functions with reflect/doc.
Patch-file is attached with code-changes that add the :doc info for ns and def to the @namespaces, and enhances the reflect functions to communicate that info to the browser in the reflect/doc call.

[CLJS-370] Incorrect behavior of integer? for integral floating point expressions

Wed, 5 Sep 2012 11:40:48 -0500

This includes infinity, exponential expressions, etc.

See: http://stackoverflow.com/questions/3885817/how-to-check-if-a-number-is-float-or-integer

[CLJS-369] gensyms break re-analyze; prevents variable shadowing analysis

Wed, 17 Oct 2012 10:53:50 -0500

From my email discussion with dnolon before making this patch:

The current analyzer does some trickery to prepare for emitting to JavaScript. Among this trickery is gensyms for locals (including "this"), the new $ prefix on some namespaces, uniqify on parameters, and more. This must be mildly annoying for people writing alternate compiler backends, but for the most part non-blocking because fewer symbol collisions should never be an additional problem for a target language with different symbol resolution rules.

[snip lots more text]

Consider what an ANF transform would look like for the :let form:

(defmethod anf :let
  [{:keys [env bindings statements ret form] :as ast}]
  (let [bindings (mapcat (fn [{:keys [sym init] :as binding}]
                           [name (anf init)])
                         bindings)
        body-env (-> bindings last :env)]
    (ana/analyze env `(~(first form) [~@(map :form bindings)]
                          ~@(anf-block (assoc ast :env body-env))))))

Simple enough, right? This walks each binding, ANF transforms the init expression, gets the environment in which to analyze the body, and then analyzes a new let (or loop) statement with the modified bindings.

Unfortunately, this doesn't work. When the ana/analyze call happens, body-env contains gensymed locals. The result is that body-env is invalidated by the outer analyze call, which is re-generating the symbols for the local variables. If you take the gensyms out of analyze-let, then analyze becomes pure (modulo def forms) and this code magically becomes correct. I've run into this same problem anywhere gensyms are used in analyze.

Commit message on the patch:

AST Changes
    
    * Anywhere a binding was introduced for a local used to be a symbol,
      now it is a map with a :name key and potentially a :shadow key.
    
    * Bindings vectors are no longer alternating symbols, then init maps.
      Instead, the are a vector of maps of the shape described for locals
      plus an :init key.
    
    * The :gthis key for functions has been replaced with :type, which
      is the symbol describing the type name of the enclosing deftype form.
    
    * recur frames now expose :params as binding maps, instead of :names
    
    Benefits:
    
    * Shadowed variables are now visible to downstream AST transforms.
    
    * :tag, :mutable, and other metadata are now uniform across ops
    
    * Eliminates usages of gensym inside the analyzer, which was a source
      of state that made the analyzer impossible to use for some
      transformations of let, letfn, etc which require re-analyzing forms.
    
    * Removes JavaScript shadowing semantics from the analyze phase.

[CLJS-360] Analyzer incorrectly merges metadata when redefining vars

Fri, 31 Aug 2012 11:29:48 -0500

I'm experimenting with using the analyzer for some more sophisticated macros, including a CPS transform and control constructs. During interactive development, I discovered that the analyzer is incorrectly merging metadata on vars when redefining them. This patch changes redef's to replace, rather than merge, existing var metadata.

The patch does not include a test, since the tests don't currently muck with the analyzer directly. Here's some code you can play with in your repl:

(require '[cljs.analyzer :as ana])
(require '[cljs.compiler :as comp])

(def env (ana/empty-env))

(defn show-foo [form]
  (ana/analyze env form)
  (-> @ana/namespaces (get 'cljs.user) :defs (get 'x) :foo))

(show-foo '(def ^{:foo 1} x 1))

(show-foo '(def ^{:foo 2} x 1))

(show-foo '(def x 1))  ; before patch, this returns 2. After patch, nil.

[CLJS-326] hash-set and PersistentHashSet/fromArray == faster set construction

Mon, 25 Jun 2012 06:33:47 -0500

As discussed with mmarczyk in irc, the attached patch implements Clojure's hash-set function and modifies the compiler to create sets using a new fromArray static method. To test these, I created three new benchmarks. Here's before and after benchmark results:

$ time ./script/benchmark | grep '#{'
[], #{}, 100000 runs, 77 msecs
[], #{1 2 3}, 100000 runs, 582 msecs
[coll #{1 2 3}], (conj coll 4), 100000 runs, 104 msecs
[], #{}, 100000 runs, 740 msecs
[], #{1 2 3}, 100000 runs, 1809 msecs
[coll #{1 2 3}], (conj coll 4), 100000 runs, 221 msecs
[], #{}, 100000 runs, 281 msecs
[], #{1 2 3}, 100000 runs, 1310 msecs
[coll #{1 2 3}], (conj coll 4), 100000 runs, 288 msecs
./script/benchmark 109.41s user 3.82s system 127% cpu 1:28.96 total

$ time ./script/benchmark | grep '#{'
[], #{}, 100000 runs, 1 msecs
[], #{1 2 3}, 100000 runs, 333 msecs
[coll #{1 2 3}], (conj coll 4), 100000 runs, 100 msecs
[], #{}, 100000 runs, 1 msecs
[], #{1 2 3}, 100000 runs, 1670 msecs
[coll #{1 2 3}], (conj coll 4), 100000 runs, 325 msecs
[], #{}, 100000 runs, 1 msecs
[], #{1 2 3}, 100000 runs, 689 msecs
[coll #{1 2 3}], (conj coll 4), 100000 runs, 321 msecs
./script/benchmark 103.94s user 4.89s system 108% cpu 1:39.88 total
grep '#{' 0.00s user 0.00s system 0% cpu 1:39.85 total

[CLJS-321] Support with-out-str

Mon, 22 Oct 2012 18:06:13 -0500

Attached patch implements with-out-str macro in terms of goog.string.StringBuffer and print-fn via .append

The attached patch also fixes ~~CLJS-319~~

[CLJS-320] memfn not supported

Mon, 18 Jun 2012 21:01:52 -0500

The docs say "it almost always preferable to do this directly now", but this was one more thing that slowed down the porting of a small piece of JVM Clojure code for me. I'm of the opinion that parity with Clojure proper is desirable where it doesn't hurt platform interop or performance.

[CLJS-319] missing spaces when printing the same thing more than once

Thu, 5 Jul 2012 15:06:57 -0500

E.g. (println 1 1) prints "11\n" instead of "1 1\n".

Here's the culprit:
https://github.com/clojure/clojurescript/blob/master/src/cljs/cljs/core.cljs#L6136-6137

[CLJS-304] Comparing a js/Date to nil throws

Mon, 18 Jun 2012 20:59:50 -0500

(= (js/Date.) nil)

[CLJS-301] Inline instance?

Sat, 22 Dec 2012 16:33:35 -0600

See discussion on http://dev.clojure.org/jira/browse/CLJS-300

This is a low priority patch for inlining instanceof checks. I don't know if it has any significant performance improvement, but the patch was sort of a side effect of my work on ~~CLJS-300~~, so I figured I'd bundle it up and share it here in case someone wants to test it's perf impact and apply if it is a win.

[CLJS-300] RegExp objects print incorrectly

Mon, 4 Jun 2012 23:52:42 -0500

#"x" is currently printing as #</x/>

js/RegExp needs an IPrintable implementation: (-pr-seq [re _] (list "#\"" (.-source re) "\""))

Unfortunately, this is blocked by the same GClosure issue as http://dev.clojure.org/jira/browse/CLJS-68

[CLJS-297] Eliminate :meta, :vector, :set, and :map ops

Fri, 31 Aug 2012 09:24:51 -0500

The attached patch eliminates the :meta, :set, :vector, and :map ops.

These four operations can be defined more simply in terms of
calls to with-meta, set, vector, and hash-map respectively.

The compiler was optimizing construction of vectors and maps. Now,
those optimizations are implemented as macros. Additionally, sets
are optimized in much the same way.

3 files changed, 52 insertions, 99 deletions

Also worth mentioning: as macros instead of ops & emit methods, these optimizations can apply to any backend. The macros create ClojureScript forms, rather than manually generating JavaScript.

[CLJS-294] AST contains munged symbols

Sun, 3 Jun 2012 17:12:28 -0500

All of the symbols in the abstract syntax tree are munged according for use in JavaScript. This is a problem because it means the AST is lossy and new compiler backends may have differing munging rules.

The attached patch removes all munging from the analysis phase and moves it into the emit phase. The emit phase is backend specific, so it's an appropriate place for munging.

[CLJS-289] Represent ast :children as a vector of keys

Wed, 29 Aug 2012 00:09:12 -0500

See discussion here: https://groups.google.com/d/topic/clojure-dev/vZLVKmKX0oc/discussion

Summary: Duplication of ast nodes in various keys and in :children is problematic for some users. In particular, it doesn't play nicely with printing. A solution is needed which preserves "The AST is data. Period."

The attached patch makes no changes to how the sub nodes are currently stored outside of the :children key. Instead, it replaces :children values with a vector of keys into the main ast node. This preserves child ordering and allows a children function to be defined as:

(defn children [ast]
(mapcat #(if (coll? %) % [%]) ((apply juxt (:children ast)) ast)))

The attached patch has two caveats: 1) Many (but not all?) blocks will now be present in the children hierarchy as 'do forms. 2) Interleaved forms are intentionally bugged with respect to ordering. The :children vector for those forms match the actual behavior, not the expected behavior. This can be fixed along with http://dev.clojure.org/jira/browse/CLJS-288

[CLJS-260] Add clojure.core/shuffle implementation

Sun, 20 May 2012 22:28:09 -0500

I added a simple implementation of clojure.core/shuffle, which uses goog.array's Fisher-Yates for its implementation. Included in the patch is a test to make sure it works.

[CLJS-215] js-obj function broken

Sun, 29 Apr 2012 17:08:26 -0500

The js-obj macro is fine, but the js-obj function was incorrectly using "{foo: bar}" expecting "foo" to be evaluated as a variable, instead of as a string key.

Since we have the js-obj macro for speed, I just deleted the broken overloads, so now all the non-zero arity cases simply call gobject

[CLJS-211] apply does not work on IFn objects in CLJS

Sat, 28 Apr 2012 12:50:06 -0500

http://dev.clojure.org/jira/browse/CLJS-135 fixed this for multimethods, but the same issue applies to anyone who implements IFn.

The attached patch fixes this by modifying extend-type to emit an apply method on the extended object's prototype.

[CLJS-210] Implement Var form, var-get, and var? in CLJS

Sun, 28 Oct 2012 21:03:58 -0500

See discussion of Vars in CLJS here: http://dev.clojure.org/display/design/Dynamic+Binding
My goal is to eventually implement dynamic scope capture, via bound-fn and friends. The attached patch is a step in that direction.

This patch adds support for parsing the (var foo) form. The #' reader macro is provided by JVM Clojure.

#'foo emits code to construct a Var object. In this patch, each invocation of 'var will create a unique Var object. This means they are '= by fully qualified symbol, but not 'identical?. Simple memoization would fix that, but I'm not going to bother until I get to Dynamic Var objects.

The main advantage of this level of Var support is for the interactive development convenience of being able to defer dereferencing top-levels. For example, (def fog (comp f #'g)) will pick up redefinitions of 'g, but not of 'f.

[CLJS-177] js->clj should convert JavaScript null to Clojure nil

Tue, 10 Apr 2012 19:38:02 -0500

Prior to commit ebf45f5 [A], js->clj converted JavaScript null to Clojure nil without any errors.

As of commit ebf45f5 (related to ~~CLJS-175~~), js->clj throws a TypeError when it encounters JavaScript null:

TypeError: Cannot read property 'constructor' of null
b===c}function q(b,c){return gb.call(e,b,c)}function pb(b){return b.construct
^
TypeError: Cannot read property 'constructor' of null

The attached patch [B] adds a test to demonstrate the issue and provides a suggested fix.

Note: ~~CLJS-175~~ intentionally limited the types of objects it would attempt to convert to Clojure code. For example, ~~CLJS-175~~ notes that conversion "doesn't make sense for something like Date or RegExp." I agree for those types of objects, but "null" seems straightforward enough that it should be converted.

[A] https://github.com/clojure/clojurescript/commit/ebf45f5
[B] Also viewable here: https://github.com/jasonrudolph/clojurescript/compare/master...js-null-to-clj-nil

[CLJS-176] Track line & column in generated JS (needed for SourceMaps)

Tue, 17 Apr 2012 12:57:22 -0500

The attached patch modifies the ClojureScript compiler by replacing all calls to print, println, emits, etc, such that all code generation funnels through the new emitx and emitln. The emitx and emitln functions track position as a 2-tuple vector of [line, column], zero-based. The line and column information is a prerequisite for generating correct source maps. NOTE: This is the position in the generated JavaScript source, not the :line and :column metadata from the Clojure reader.

In the emitln function, there are a few commented out lines which constitute a "test". Basically, it causes the compiler to append a line number // comment on each line of the generated source. The comments are horizontally aligned at 120 characters, so you can quickly visually inspect for alignment bugs. Unfortunately, I'm not really sure how better to test this, or to ensure that this keeps working until the rest of SourceMaps gets implemented In theory, as long as no one prints anything directly, always using emitx and emitln, nothing should break.

[CLJS-169] Forward slashes should not be escaped in javascript strings

Fri, 30 Mar 2012 15:21:25 -0500

cljs.compiler/escape-char incorrectly escapes forward slashes.

Generally, this isn't a problem because most Javascript engines treat unknown escape sequences as non-escape sequences. However, I ran into a JSON parser that barfs on it while working on source-maps. I tried to use escape-string for producing the JSON-based source-map by reusing the existing compiler functions for emitting valid Javascript.

[CLJS-164] Add missing core printing functions

Fri, 30 Mar 2012 15:20:55 -0500

I've attached a patch which adds a few missing printing functions to ClojureScript core.

New functions:

prn-str
print-str
println-str

[CLJ-1084] (object-array [1]) is ~3x slower than (object-array (rseq [1]))

Sat, 20 Oct 2012 09:53:42 -0500

{{user=> (time (dotimes [_ 10000000] (object-array [1])))
"Elapsed time: 1178.116257 msecs"
nil
user=> (time (dotimes [_ 10000000] (object-array (rseq [1]))))
"Elapsed time: 422.42248 msecs"
nil}}

This appears to be because PersistentVector$ChunkedSeq does not implement Counted, so RT.count is iterating the ChunkedSeq to get its count.

[CLJ-1020] clojure.inspector/inspect-table gives up when first element of coll is nil

Thu, 13 Sep 2012 14:30:24 -0500

clojure.inspector/inspect-table gives up when first element of coll is nil. The patch provided is rather trivial...instead of blindly choosing the first element (which might be nil), it would be more convenient to choose the first element that is NOT nil and use its keys for columns...a similar issue exists with clojure.pprint/print-table where the keys of the first element are used (if not provided explicitly). The same is not true for 'inspect-table' though. As a result, one cannot 'inspect' a collection of maps where the first element is nil. My (trivial) patch looks for the first element which is NOT nil and uses its keys instead. Maps have to have the same length anyway so no problems there...

[CLJ-991] partition-by reducer

Mon, 4 Mar 2013 14:49:42 -0600

partition-by reducer

[CLJ-963] Support pretty printing namespace declarations under code-dispatch

Sat, 1 Sep 2012 08:37:48 -0500

Currently when you print code with an (ns ) declaration in it, the ns declaration comes out kind of messy. This patch makes that beautiful.

[CLJ-953] drop-while doc string wrong, nil instead of logical false

Fri, 30 Mar 2012 09:16:31 -0500

"Returns a lazy sequence of the items in coll starting from the first
-  item for which (pred item) returns nil."
+  item for which (pred item) returns logical false."

[CLJ-886] java.io/do-copy can garble multibyte characters

Fri, 23 Mar 2012 08:40:35 -0500

See comments in fix:

(defmethod do-copy [InputStream Writer] [#^InputStream input #^Writer output opts]
;; WRONG! if the buffer boundry falls in the middle of a multibyte character, we will get garbled results.
#_
(let [#^"[B" buffer (make-array Byte/TYPE (buffer-size opts))]
(loop []
(let [size (.read input buffer)]
(when (pos? size)
(let [chars (.toCharArray (String. buffer 0 size (encoding opts)))]
(do (.write output chars)
(recur)))))))
;; here we decode the characters before stuffing them into the buffer
(let [#^"[C" buffer (make-array Character/TYPE (buffer-size opts))
in (InputStreamReader. input (encoding opts))]
(loop []
(let [size (.read in buffer 0 (alength buffer))]
(if (pos? size)
(do (.write output buffer 0 size)
(recur)))))))

[CLJ-879] Allow :require to support a :refer clause

Fri, 17 Feb 2012 14:18:28 -0600

There's been discussion previously about the complexity of the ns
macro. In particular the fact that :use causes all vars to be referred
by default is widely seen as unfortunate, and the distinction between
use and require is a source of conceptual overhead for people new to
the language.

We can't change the fact that :use refers everything by default
without breaking lots of existing code. But it would be possible to
enhance :require to support referring specified vars, leaving us free
to deprecate or otherwise discourage the use of :use.

Clojure-dev thread discussing this: http://groups.google.com/group/clojure-dev/browse_thread/thread/91b708ddb909affd

[CCACHE-22] Change limit parameter name in TTLCache

Fri, 15 Jun 2012 13:49:12 -0500

"limit" is used in other caches to represent quantity, for TTL it represents time. Using the same name is confusing