http://dev.clojure.org/jira This file is an XML representation of an issue en-us 4.4 649 25-07-2011 http://dev.clojure.org/jira/browse/CLJ-1177 Clojure <p>clojure.java.io/resource corrupts path containing UTF-8 characters without issuing warning.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">user=&gt; (<span class="code-object">System</span>/getProperty <span class="code-quote">"java.runtime.version"</span>) <span class="code-quote">"1.8.0-ea-b79"</span> user=&gt; (clojure-version) <span class="code-quote">"1.5.0"</span> user=&gt; (<span class="code-object">System</span>/getProperty <span class="code-quote">"user.dir"</span>) <span class="code-quote">"/dir/déf"</span> user=&gt; (clojure.java.io/resource <span class="code-quote">"myfile.txt"</span>) #&lt;URL file:/dir/d%c3%a9f/resources/myfile.txt&gt; user=&gt; (slurp (clojure.java.io/resource <span class="code-quote">"myfile.txt"</span>) :encoding <span class="code-quote">"UTF-8"</span>) FileNotFoundException /dir/déf/resources/myfile.txt (No such file or directory) java.io.FileInputStream.open (FileInputStream.java:-2)</pre> </div></div> CLJ-1177

clojure.java.io/resource and non-ASCII characters

Enhancement Minor Open Unresolved Unassigned Trevor Wennblom bug enhancement Thu, 7 Mar 2013 16:54:43 -0600 Sun, 10 Mar 2013 17:56:52 -0500 Release 1.5 1 0 <p>Below is a workaround, at least. I don't know, but perhaps the as-file method for URLs in io.clj of Clojure, the part that converts %hh sequences to a character with code point in the range 0 through 255, is at least partly at fault here. I don't know right now if it is possible to modify that code to handle the general case of whatever character encoding munging is going on here to when .getResource creates the URL object.</p> <p>clojure.java.io/resource is documented to return a Java object of type java.net.URL, which seems like it does %hh escaping of many characters. Reference <span class="error">&#91;1&#93;</span> to a Java bug from 2001 where a Java user was surprised by the then-recent change in behavior of the getResource method <span class="error">&#91;2&#93;</span>.</p> <p>Doing a little searching I found this StackOverflow question <span class="error">&#91;3&#93;</span>, which has what might be a workaround. I tried it on my Mac OS X 10.6 system running JDK 1.6 and it seemed to work:</p> <p>(slurp (.getContent (clojure.java.io/resource "abcíd/foo.txt")))</p> <p>That getContent is a method for class java.net.URL <span class="error">&#91;4&#93;</span></p> <p><span class="error">&#91;1&#93;</span> <a href="http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4466485">http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4466485</a><br/> <span class="error">&#91;2&#93;</span> <a href="http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Class.html#getResource%28java.lang.String%29">http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Class.html#getResource%28java.lang.String%29</a><br/> <span class="error">&#91;3&#93;</span> <a href="http://stackoverflow.com/questions/13013629/best-international-alternative-to-javas-getclass-getresource">http://stackoverflow.com/questions/13013629/best-international-alternative-to-javas-getclass-getresource</a><br/> <span class="error">&#91;4&#93;</span> <a href="http://docs.oracle.com/javase/1.5.0/docs/api/java/net/URL.html#getContent%28%29">http://docs.oracle.com/javase/1.5.0/docs/api/java/net/URL.html#getContent%28%29</a></p> <p>Hi Andy,</p> <p>Thanks for the background and suggestions, that's very helpful.</p> <p>I'm gradually learning Clojure with no Java experience. In this case I was searching for the preferred Clojure way to access items in directories declared under :resource-paths in a Leiningen project.clj file. Perhaps clojure.java.io/resource isn't the best way to do this as it's possibly too tied to the expectation for a URI instead of a more general IRI.</p> <p>You're suggested workaround did work for my use case:</p> <p>(slurp (.getContent (clojure.java.io/resource "abcíd/foo.txt")))</p> <p>but hopefully there would be more native/direct Clojure way to accomplish the same eventually.</p> <p>I don't know if java.net.IDN would be useful internally as a fix in clojure.java.io/resource — I'm assuming not since it wasn't added until Java 6.<span class="error">&#91;1&#93;</span></p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">user=&gt; (<span class="code-keyword">import</span> 'java.net.IDN) java.net.IDN user=&gt; (java.net.IDN/toASCII <span class="code-quote">"/dir/déf"</span>) <span class="code-quote">"xn--/dir/df-gya"</span> user=&gt; (java.net.IDN/toUnicode <span class="code-quote">"xn--/dir/df-gya"</span>) <span class="code-quote">"/dir/déf"</span></pre> </div></div> <p><span class="error">&#91;1&#93;</span>: <a href="http://docs.oracle.com/javase/6/docs/api/java/net/IDN.html">http://docs.oracle.com/javase/6/docs/api/java/net/IDN.html</a></p> <p>Patch clj-1177-patch-v1.txt dated Mar 8 2013 is an attempt to solve this issue, in what I think may be a correct way. As specified in RFC 3986, when taking a Unicode string and making a URL of it, it should be encoded in UTF-8 and then each individual byte is subject to the %HH hex encoding. This patch reverses that to turn URLs into file names.</p> <p>Tested on Mac OS X 10.6 with a command line like this (it doesn't work without the -Dfile.encoding=UTF-8 option on my Mac, probably because the default encoding is MacRoman):</p> <p>% java -cp clojure.jar:path/to/resource -Dfile.encoding=UTF-8 clojure.main<br/> user=&gt; (require '<span class="error">&#91;clojure.java.io :as io&#93;</span>)<br/> nil<br/> user=&gt; (io/resource "abcíd/foo.txt")<br/> #&lt;URL <a href="file:/Users/jafinger/clj/clj-ns-browser/resource/abc%c3%add/foo.txt">file:/Users/jafinger/clj/clj-ns-browser/resource/abc%c3%add/foo.txt</a>&gt;<br/> user=&gt; (slurp (io/resource "abcíd/foo.txt"))<br/> "The quick brown fox jumped over the lázy dög!\n"</p> Global Rank Patch Code