<< Back to previous view

[CLJ-886] java.io/do-copy can garble multibyte characters Created: 28/Nov/11  Updated: 23/Mar/12  Resolved: 23/Mar/12

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.3
Fix Version/s: Release 1.4

Type: Defect Priority: Major
Reporter: Jeff Palmucci Assignee: Unassigned
Resolution: Completed Votes: 2
Labels: patch,
Environment:

all


Attachments: File clj-886.diff     Text File CLJ-886-fix2.patch    
Patch: Code and Test
Approval: Ok
Waiting On: Rich Hickey

 Description   

See comments in fix:

(defmethod do-copy [InputStream Writer] [#^InputStream input #^Writer output opts]
;; WRONG! if the buffer boundry falls in the middle of a multibyte character, we will get garbled results.
#_
(let [#^"[B" buffer (make-array Byte/TYPE (buffer-size opts))]
(loop []
(let [size (.read input buffer)]
(when (pos? size)
(let [chars (.toCharArray (String. buffer 0 size (encoding opts)))]
(do (.write output chars)
(recur)))))))
;; here we decode the characters before stuffing them into the buffer
(let [#^"[C" buffer (make-array Character/TYPE (buffer-size opts))
in (InputStreamReader. input (encoding opts))]
(loop []
(let [size (.read in buffer 0 (alength buffer))]
(if (pos? size)
(do (.write output buffer 0 size)
(recur)))))))



 Comments   
Comment by Jeff Palmucci [ 05/Dec/11 11:23 AM ]

Make patch file per http://clojure.org/patches. Also sent in my CA.

Comment by Jeff Palmucci [ 19/Dec/11 8:53 AM ]

Any reason why this hasn't been applied yet? It's a pretty serious error. Just wondering if I've submitted anything incorrectly. Thanks.

Comment by Andy Fingerhut [ 09/Feb/12 8:13 PM ]

CLJ-886-fix2.patch includes Jeff Palmucci's fix, plus another one found while enhancing clojure.java.io's unit tests to fail with the original code. Tested on Mac OS X 10.6.8 with java 1.6.0_29 and Ubuntu Linux 11.04 with JVM 1.7.0_02.

Comment by Jeff Palmucci [ 13/Feb/12 7:29 AM ]

Andy's fix is good. Please use the fix2 patch instead of the original.

Comment by Stuart Sierra [ 17/Feb/12 2:55 PM ]

Screened. Patch applies successfully and the tests seem to be complete.

Generated at Thu Nov 27 23:54:01 CST 2014 using JIRA 4.4#649-r158309.