[CLJ-668] Improve slurp performance by using native Java StringWriter and jio/copy Created: 01/Nov/10 Updated: 12/Oct/15 Resolved: 12/Oct/15
|Affects Version/s:||Release 1.3|
|Fix Version/s:||Release 1.8|
|Reporter:||Jürgen Hötzel||Assignee:||Timothy Baldridge|
|Labels:||ft, io, performance|
Instead of copying each character from InputReader to StringBuffer.
Approach: Use StringWriter and jio/copy vs character by character copy. Results from the current patch see a 4-5x perf boost after the jit warms up, with purely in-memory streams (ByteArrayInputStream over a 6MB string).
|Comment by Alex Miller [ 21/Apr/14 3:28 PM ]|
This is double-better with the changes in Clojure 1.6 to improve jio/copy performance by using the NIO impl. Rough timing difference on a 25M file: old= 2316.021 msecs, new= 93.319 msecs.
Filer did not supply a patch and is not a contributor. If someone wants to make a patch (and better timing info demonstrating performance improvements), that would be great.
|Comment by Timothy Baldridge [ 10/Sep/14 10:29 PM ]|
Fixed the ticket formatting a bit, and added a patch I coded up tonight. Should be pretty close to the old patch, as we both use StringWriter, but I didn't really look at the old patch beyond noticing that it was using StringWriter.
|Comment by Alex Miller [ 11/Sep/14 7:01 AM ]|
Can you update the perf comparison on latest code and do both a small and big file?
|Comment by Michael Blume [ 03/Sep/15 7:44 PM ]|
Is this blocked on a new perf comparison? Should I post one?
|Comment by Michael Blume [ 03/Sep/15 8:19 PM ]|
|Comment by Michael Blume [ 03/Sep/15 8:28 PM ]|
|Comment by Michael Blume [ 03/Sep/15 8:30 PM ]|
This is with an SSD, no idea what the numbers would look like with a spinny disk.
|Comment by Alex Miller [ 09/Oct/15 8:23 AM ]|
Updated patch to contain ticket id in commit description and provide a few more lines of diff context. No actual changes in the new patch, attribution retained.