Clojure

• Type: Defect
• Status: Closed
• Priority: Major
• Resolution: Declined
• Affects Version/s: Release 1.3
• Fix Version/s: None
• Component/s: None
• Labels:

  None
• Environment:

  java 6 jdk on linux, jre 6 on windows 7

I've a head holding problem that I believe is a bug in clojure 1.3. I
wrote the following function to split a a lazy seq of strings across
files of x size:

(defn split-file
 ([path strs size]
    (trampoline split-file path (seq strs) size 0))
 ([path strs size part]
    (with-open [f (clojure.java.io/writer (str path "." part))]
      (loop [written 0, ss strs]
        (when ss
          (if (>= written size)
            #(split-file path ss size (inc part))
            (let [s (first ss)]
              (.write f s)
              (recur (+ written (.length s)) (next ss)))))))))

If I call the 3 arg version of the function:
(split-file "foo" (repeat 100000000 "blah blah blah") 100000000)

I see memory usage increases as I'm writing each file with the usual
gc slow down, then memory usage goes back down again as I get to a new
split file.

Memory usage is fine if I call the 4 arg version (which only writes
one part of the split file):
(split-file "foo" (repeat 100000000 "blah blah blah") 100000000 0)

I can also avoid the head holding problem by removing trampoline and
recursively calling split-file directly, but then those recursive
calls use up stack and don't close files until all calls complete

• Assignee:

  Unassigned

  Reporter:

  Gerrard McNulty

• Created:

  27/Nov/11 5:53 AM

  Updated:

  01/Mar/13 12:46 PM

  Resolved:

  27/Nov/12 2:29 PM