Clojure

clojure.java.shell/sh in-enc doesn't default to UTF-8

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: Release 1.2
  • Component/s: None
  • Labels:
    None
  • Approval:
    Ok

Description

clojure.java.shell/sh was recently updated (changed) to say that it defaults to UTF-8 for both in-enc and out-enc, but it does not do so for in-enc.

On line 122 of the current build:

(with-open [osw (OutputStreamWriter. (.getOutputStream proc) ^String in-enc)]

in-enc is nil by default, but OutputStreamWriter needs to be passed "UTF-8" instead if in-enc is nil. I would add a patch but I still need to submit the CLA.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Assembla Importer added a comment -

Converted from http://www.assembla.com/spaces/clojure/tickets/413
Attachments:
413-parse-args-defaults-in-enc-and-out-enc-to-UTF-8.patch - https://www.assembla.com/spaces/clojure/documents/a1Pm9AL2ar351-eJe5cbCb/download/a1Pm9AL2ar351-eJe5cbCb

Show
Assembla Importer added a comment - Converted from http://www.assembla.com/spaces/clojure/tickets/413 Attachments: 413-parse-args-defaults-in-enc-and-out-enc-to-UTF-8.patch - https://www.assembla.com/spaces/clojure/documents/a1Pm9AL2ar351-eJe5cbCb/download/a1Pm9AL2ar351-eJe5cbCb
Hide
Permalink
Assembla Importer added a comment -

bpsm said: It was tough verifying that out-enc is even honored: If out-enc is not specified, it's nil. This gets passed to the two-argument stream-to-enc. The two-argument stream-to-enc calls (stream-to-string stream nil) since nil is not equal to :bytes. This in turn calls java.io/copy with :encoding nil, which gets passed on up a chain of do-copy calls until we finally try to extract the encoding from the opts map. The accessor function (java.io/encoding opts) does the actual defaulting to UTF-8.

Maybe the implementation in shell should be more explicit?

Side note: the stream-to-enc, stream-to-bytes, stream-to-string smells like it's reinventing parts of java.io, but I've probably overlooked something. It's late where I am.

Show
Assembla Importer added a comment - bpsm said: It was tough verifying that out-enc is even honored: If out-enc is not specified, it's nil. This gets passed to the two-argument stream-to-enc. The two-argument stream-to-enc calls (stream-to-string stream nil) since nil is not equal to :bytes. This in turn calls java.io/copy with :encoding nil, which gets passed on up a chain of do-copy calls until we finally try to extract the encoding from the opts map. The accessor function (java.io/encoding opts) does the actual defaulting to UTF-8. Maybe the implementation in shell should be more explicit? Side note: the stream-to-enc, stream-to-bytes, stream-to-string smells like it's reinventing parts of java.io, but I've probably overlooked something. It's late where I am.
Hide
Permalink
Assembla Importer added a comment -

bpsm said: [file:a1Pm9AL2ar351-eJe5cbCb]

Show
Assembla Importer added a comment - bpsm said: [file:a1Pm9AL2ar351-eJe5cbCb]
Hide
Permalink
Assembla Importer added a comment -

bpsm said: My analysis from last night was wrong. The defaulting of :in-enc and :out-enc actually occurs in parse-args. parse-args was still defaulting to platform default encoding despite the fact that sh clearly wants to default to UTF-8.

This likely wasn't noticed because the unit test for parse-args was not updated when it was decided to use UTF-8 for clojure.java.shell in place of the previous decision to use platform default encoding.

Show
Assembla Importer added a comment - bpsm said: My analysis from last night was wrong. The defaulting of :in-enc and :out-enc actually occurs in parse-args. parse-args was still defaulting to platform default encoding despite the fact that sh clearly wants to default to UTF-8. This likely wasn't noticed because the unit test for parse-args was not updated when it was decided to use UTF-8 for clojure.java.shell in place of the previous decision to use platform default encoding.
Hide
Permalink
Assembla Importer added a comment -

stu said: Thanks Marcus and Ben!

Show
Assembla Importer added a comment - stu said: Thanks Marcus and Ben!
Hide
Permalink
Assembla Importer added a comment -

stu said: Updating tickets (#340, #399, #402, #403, #413)

Show
Assembla Importer added a comment - stu said: Updating tickets (#340, #399, #402, #403, #413)

People

  • Assignee:
    Unassigned
    Reporter:
    Anonymous
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4#649-r158309) | Report a problem
Powered by a free Atlassian JIRA open source license for Clojure. Try JIRA - bug tracking software for your team.