<< Back to previous view

[CLJ-1197] Allow fold to parallelize over lazy sequences Created: 10/Apr/13  Updated: 27/Sep/13  Resolved: 27/Sep/13

Status: Resolved
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Paul Butcher Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: reducers

Attachments: File foldable-seq.diff    
Patch: Code

 Description   

This patch implements foldable-seq, which allows fold to parallelize over a lazy sequence. See this conversation on the Clojure mailing list:

https://groups.google.com/forum/#!msg/clojure/8RKCjF00ukQ/b5mmmOB5Uh4J

The patch is code only, sadly. No tests because I've not been able to find any existing tests for fold:

https://groups.google.com/d/msg/clojure-dev/plQ16L1_FC0/CIyMVIgSZkkJ

However, I have tested it in a separate project successfully.



 Comments   
Comment by Stuart Halloway [ 27/Sep/13 3:12 PM ]

Hi Paul,

Seqs are fundamentally not foldable. That said, what you seem to be trying to do (partition a seq into foldable subjobs) is straightforward in Clojure, see

https://github.com/stuarthalloway/exploring-clojure/blob/master/examples/exploring/reducing_apple_pie.clj?#L62-73

That said, if the input is truly arriving sequentially, pmap or some variant may do as well or better, e.g.

https://github.com/stuarthalloway/exploring-clojure/blob/master/examples/exploring/reducing_apple_pie.clj?#L77-83

Generated at Sun Sep 21 09:25:01 CDT 2014 using JIRA 4.4#649-r158309.