Clojure

Allow fold to parallelize over lazy sequences

Details

  • Type: Enhancement Enhancement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Declined
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
  • Patch:
    Code

Description

This patch implements foldable-seq, which allows fold to parallelize over a lazy sequence. See this conversation on the Clojure mailing list:

https://groups.google.com/forum/#!msg/clojure/8RKCjF00ukQ/b5mmmOB5Uh4J

The patch is code only, sadly. No tests because I've not been able to find any existing tests for fold:

https://groups.google.com/d/msg/clojure-dev/plQ16L1_FC0/CIyMVIgSZkkJ

However, I have tested it in a separate project successfully.

Activity

Hide
Stuart Halloway added a comment -

Hi Paul,

Seqs are fundamentally not foldable. That said, what you seem to be trying to do (partition a seq into foldable subjobs) is straightforward in Clojure, see

https://github.com/stuarthalloway/exploring-clojure/blob/master/examples/exploring/reducing_apple_pie.clj?#L62-73

That said, if the input is truly arriving sequentially, pmap or some variant may do as well or better, e.g.

https://github.com/stuarthalloway/exploring-clojure/blob/master/examples/exploring/reducing_apple_pie.clj?#L77-83

Show
Stuart Halloway added a comment - Hi Paul, Seqs are fundamentally not foldable. That said, what you seem to be trying to do (partition a seq into foldable subjobs) is straightforward in Clojure, see https://github.com/stuarthalloway/exploring-clojure/blob/master/examples/exploring/reducing_apple_pie.clj?#L62-73 That said, if the input is truly arriving sequentially, pmap or some variant may do as well or better, e.g. https://github.com/stuarthalloway/exploring-clojure/blob/master/examples/exploring/reducing_apple_pie.clj?#L77-83

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: