http://dev.clojure.org/jira This file is an XML representation of an issue en-us 4.4 649 25-07-2011 http://dev.clojure.org/jira/browse/CLJ-870 Clojure <p>clojure.string/replace uses javas replace function to do it's work, the replace function has the tricky habit of 'double evaluating' the replacement string (third argument). This means that every \ in the string (so i.e. "<br clear="all" />") is evaluated by the java code behind replace.</p> <p>Since this behavior isn't documented it can lead to confusing errors, for example (made up semi real world example to explain the issue):</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>(clojure.string/replace "c:/windows/" #"/" "\\")</pre> </div></div> <p>This should replace all unix folder separators with windows separators, this crashes with a index out of bound exemption since java tries to evaluate "<br clear="all" />" as a semi regexp.</p> <p>or</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>(println (str "\"" (clojure.string/replace "my string with \\ and \" in it" #"[\"\\]" (fn [x] (str "\\" x))) "\""))</pre> </div></div> <p>The expected result would be:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>"my string with \\ and \" in it"</pre> </div></div> <p>the actual result is:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>"my string with \ and " in it"</pre> </div></div> <p>This should return an 'escaped' string, it does not since the 'double evaluation' of the return string results in \\\\ being reduced to <br clear="all" /> again.</p> HW/OS/SW indipendant - issue is part of java interface CLJ-870

clojure.string/replace behaves unexpectedly when \ or $ are part of the result string

Defect Minor Closed Completed Stuart Sierra Heinz N. Gies Fri, 4 Nov 2011 05:28:31 -0500 Sat, 18 Aug 2012 07:50:41 -0500 Sat, 18 Aug 2012 07:50:41 -0500 Release 1.2 Release 1.3 Release 1.5 2 3 <p>A working code for the replace example is: </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>(println (str "\"" (clojure.string/replace "my string with \\ and \" in it" #"[\"\\]" (fn [x] (str "\\\\" (if (= x "\\") "\\") x))) "\""))</pre> </div></div> <p>which is horribly ugly since you need to hand escape the backslash or it crashes.</p> <p>Vetted on 1.4.0 master. Needs patch.</p> <p>No hand-escaping is necessary, just use #(java.util.regex.Matcher/quoteReplacement %).</p> <p>Edit: I see, we're talking about when you use a function as a replacement, not a replacement <b>string</b> like "<br clear="all" />x" - the function's output should not be interpreted as regex replacement, I agree. Looks like it just needs a small patch in clojure.string/replace-by.</p> <p>Patch and test for this issue.</p> <p><a href="http://dev.clojure.org/jira/browse/CLJ-870" title="clojure.string/replace behaves unexpectedly when \ or $ are part of the result string"><del>CLJ-870</del></a>-also-fix-replace-first.patch combines Alan Malloy's two patches, and adds the same change for replace-first. I like this change, too, and think replace and replace-first should be consistent with each other in this behavior.</p> <p>If there is any interest in a combined patch for this issue, <a href="http://dev.clojure.org/jira/browse/CLJ-753" title="clojure.string/replace-first returns nil with replacement fn when regex doesn&#39;t match"><del>CLJ-753</del></a> and <a href="http://dev.clojure.org/jira/browse/CLJ-905" title="clojure.string/replace-first treats \ and $ specially when match is a string, unlike clojure.string/replace"><del>CLJ-905</del></a> all in one, I'd be happy to merge them.</p> <p>Proposed combined patch for <a href="http://dev.clojure.org/jira/browse/CLJ-753" title="clojure.string/replace-first returns nil with replacement fn when regex doesn&#39;t match"><del>CLJ-753</del></a>, <a href="http://dev.clojure.org/jira/browse/CLJ-870" title="clojure.string/replace behaves unexpectedly when \ or $ are part of the result string"><del>CLJ-870</del></a>, and <a href="http://dev.clojure.org/jira/browse/CLJ-905" title="clojure.string/replace-first treats \ and $ specially when match is a string, unlike clojure.string/replace"><del>CLJ-905</del></a>, since they all affect the behavior of clojure.string/replace and replace-first.</p> <p>The 1/12/2012 combined patch is not in correct format. Please recreate, see <a href="http://clojure.org/patches">http://clojure.org/patches</a> for correct format.</p> <p><a href="http://dev.clojure.org/jira/browse/CLJ-753" title="clojure.string/replace-first returns nil with replacement fn when regex doesn&#39;t match"><del>CLJ-753</del></a>-870-905-combined-fix2.patch should have proper patch format.</p> <p>Attaching slightly improved patch <a href="http://dev.clojure.org/jira/browse/CLJ-753" title="clojure.string/replace-first returns nil with replacement fn when regex doesn&#39;t match"><del>CLJ-753</del></a>-870-905-combined-fix3.patch, along with a README to explain in gory detail the behavior and performance changes to clojure.string/replace and clojure.string/replace-first, in case it would help anyone review the changes. Deleting my older patches.</p> <p>This is a small nit pick, but re-qr is a pretty non-descriptive name. Perhaps we could change it to re-quote-replacement?</p> <p>Patches clj-753-870-905-combined-fix4.patch dated Aug 15 2012 and the patch that Aaron reviewed, <a href="http://dev.clojure.org/jira/browse/CLJ-753" title="clojure.string/replace-first returns nil with replacement fn when regex doesn&#39;t match"><del>CLJ-753</del></a>-870-905-combined-fix3.patch dated Feb 28 2012, are almost identical.</p> <p>fix3 uses the name re-qr for a function, where fix4 uses the name re-quote-replacement. I have no strong preference for either of them.</p> <p>Added patch addressing what I think is Aaron's reason for changing state to Incomplete, so as a result I am changing state back to its former Vetted state. If there is something else a non-screener should be doing to bring Incomplete tickets to the attention of screeners, please let me know, and I will document it here: <a href="http://dev.clojure.org/display/design/JIRA+workflow">http://dev.clojure.org/display/design/JIRA+workflow</a></p> <p>I will be looking at this.</p> <p>Screened. This is an adequate fix for this particular problem, given the current API.</p> <p>However, the long and complicated docstring proves that <tt>replace</tt> and <tt>replace-first</tt> should each be four separate functions:</p> <p><tt>code</tt><br/> replace-char <span class="error">&#91;input char char&#93;</span><br/> replace-string <span class="error">&#91;input string string&#93;</span><br/> replace-re <span class="error">&#91;input pattern string&#93;</span><br/> replace-re-by <span class="error">&#91;input pattern fn&#93;</span><br/> <tt>code</tt></p> <p>This is how these functions were originally written back in the days of clojure.contrib.str-utils. This would be a minor breaking change.</p> Approval Ok Global Rank Patch Code and Test Waiting On richhickey