<< Back to previous view

[CLJ-1218] mapcat is too eager Created: 16/Jun/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Gary Fredericks Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: lazy


 Description   

The following expression prints 1234 and returns 1:

(first (mapcat #(do (print %) [%]) '(1 2 3 4 5 6 7)))

The reason is that (apply concat args) is not maximally lazy in its arguments, and indeed will realize the first four before returning the first item. This in turn is essentially unavoidable for a variadic concat.

This could either be fixed just in mapcat, or by adding a new function (to clojure.core?) that is a non-variadic equivalent to concat, and reimplementing mapcat with it:

(defn join
  "Lazily concatenates a sequence-of-sequences into a flat sequence."
  [s]
  (lazy-seq (when-let [[x & xs] (seq s)] (concat x (join xs)))))


 Comments   
Comment by Gary Fredericks [ 17/Jun/13 7:54 AM ]

I realized that concat could actually be made lazier without changing its semantics, if it had a single [& args] clause that was then implemented similarly to join above.

Comment by John Jacobsen [ 27/Jul/13 8:08 AM ]

I lost several hours understanding this issue last month [1, 2] before seeing this ticket in Jira today... +1.

[1] http://eigenhombre.com/2013/07/13/updating-the-genome-decoder-resulting-consequences/

[2] http://clojurian.blogspot.com/2012/11/beware-of-mapcat.html

Comment by Gary Fredericks [ 05/Feb/14 1:35 PM ]

Updated join code to be actually valid.





[CLJ-1217] for consumes sequence argument more eagerly than necessary Created: 14/Jun/13  Updated: 05/Feb/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alan Malloy Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: lazy

Attachments: Text File 0001-Don-t-realize-seq-exprs-in-for-unless-necessary.patch    
Patch: Code

 Description   

In a call like (do (for [x (do (println "realized") nil)] x) nil), no elements of the for comprehension are ever requested, and so it is not actually necessary to evaluate the inner do-block. However, this expression causes "realized" to be printed, because the first sequence-expression in for is evaluated even if no items are ever requested from the output lazy-seq.

It's not documented whether this is intended or unintentional, but I was surprised by this behavior, and a brief unscientific survey on #clojure suggests that other users, even "old hands" who've been using clojure for years, don't expect this either.

I've attached a patch that wraps the problematic expression in a lazy-seq call. This is not quite ideal, because it means that the first iteration is "lazied" twice, as in ((fn step [s] (lazy-seq ...)) (lazy-seq xs)), but a change to make this not happen would be much broader in scope, and this seemed the least dangerous.






Generated at Wed Nov 26 09:01:38 CST 2014 using JIRA 4.4#649-r158309.