http://dev.clojure.org/jira This file is an XML representation of an issue en-us 4.4 649 25-07-2011 http://dev.clojure.org/jira/browse/CLJ-855 Clojure <p>Expressions passed to try that trigger the use of clojure.lang.Reflector result in their thrown checked exceptions being wrapped in RuntimeException. As a result, the subsequent set of catches won't switch on the expected checked exception.</p> <p>Attached: patch for regression test that exposes the problem</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-none">(defn- get-exception [expression] (try (eval expression) nil (catch java.lang.Throwable t t))) (deftest catch-receives-checked-exception (are [expression expected-exception] (= expected-exception (type (get-exception expression))) "Eh, I'm pretty safe" nil '(java.io.FileReader. "CAFEBABEx0/idonotexist") java.io.FileNotFoundException)) ; fails</pre> </div></div> <p>Clojure ML Thread: <a href="https://groups.google.com/forum/#!topic/clojure/I5l1YHVMgkI/discussion">https://groups.google.com/forum/#!topic/clojure/I5l1YHVMgkI/discussion</a><br/> Introduced: <a href="https://github.com/clojure/clojure/commit/8fda34e4c77cac079b711da59d5fe49b74605553">https://github.com/clojure/clojure/commit/8fda34e4c77cac079b711da59d5fe49b74605553</a></p> CLJ-855

catch receives a RuntimeException rather than the expected checked exception

Defect Major Closed Completed Ben Smith-Mannschott Paul Michael Bauer Tue, 11 Oct 2011 20:44:25 -0500 Fri, 1 Mar 2013 12:47:00 -0600 Wed, 29 Feb 2012 15:07:14 -0600 Release 1.3 Release 1.4 2 5 <p>(marked up code snippet in description to display as code.)</p> <p>(This incorporates a slightly modified version of Paul Michael Bauer's<br/> checked-exception-regression-test patch.)</p> <p>ClojureException is an RTE. RTEs that originate from Clojure all are<br/> of this type. This isn't strictly necessary for this proof of concept,<br/> but it seemed prudent since it seems that the lack of specificity resulting<br/> from using plain RTE everywhere got us into this mess in the first place.</p> <p>WrappedException is a ClojureException. It is used when Clojure is<br/> wrapping the underlying cause to work around Java's obsession with<br/> checked exceptions. It's <b>always</b> the underlying cause of<br/> WrappedException that is relevant to catchers.</p> <p>Util.runtimeException(s) now returns a ClojureException<br/> This is not strictly necessary for this patch to work</p> <p>Util.runtimeException(s,e) now returns a WrappedException.</p> <p>Util.runtimeException(e) now retuns a WrappedException when e is not<br/> already a RuntimeException.</p> <p>clojure.core/treye is a macro built on top of the compiler-provided<br/> special form try. (A <b>real</b> solution to this problem would involve<br/> altering the try special form itself, but this ist just a trial balloon<br/> to figure out if this is the right way to go.)</p> <p>Paul Michael Bauer's try_catch.clj uses treye in place of try and passes.</p> <p>&gt; <a href="http://james-iry.blogspot.com/2010/08/on-removing-java-checked-exceptions-by.html">http://james-iry.blogspot.com/2010/08/on-removing-java-checked-exceptions-by.html</a></p> <p>seems like refactoring Reflector.java to rethrow using this approach<br/> is preferable to some wrapping/unwrapping scheme</p> <p>(edit: missed Kevin's post)<br/> <a href="http://james-iry.blogspot.com/2010/08/on-removing-java-checked-exceptions-by.html">http://james-iry.blogspot.com/2010/08/on-removing-java-checked-exceptions-by.html</a></p> <p>I'd rather see Reflector.java modified to use this "chucking" technique than have a special try form that specially unwraps an exception and re-throws.</p> <p>Maybe there's a need for WrappedException, but since it's not immediately necessary if we fix Reflector, then I'd rather see it in a separate commit.</p> <p>(I'm swamped at work today and don't have time to fix Reflector myself right now)</p> <p>I added Util.throwUnchecked and Util.throwCause (also Util.declareThrows and Util.invokeThrowing) as justified in <a href="https://groups.google.com/d/msg/clojure-dev/4QynV81W0Qk/rIN9fYUK-AkJ">https://groups.google.com/d/msg/clojure-dev/4QynV81W0Qk/rIN9fYUK-AkJ</a><br/> I've used this to fix the invoke* methods in Reflector. Please review.</p> <p>There are a lot of other instances in the code, where checked exceptions are runtime-ified, but that's a separate issue.</p> <p>Applying the patch <a href="http://dev.clojure.org/jira/browse/CLJ-855" title="catch receives a RuntimeException rather than the expected checked exception"><del>CLJ-855</del></a>_throw_undeclared_checked.patch produces java that can not be compiled in four places:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"><span class="code-keyword">catch</span> (Exception e) { Util.throwCause(e); }</pre> </div></div> <p>This leaves the compiler complaining about a missing <tt>return</tt>.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"><span class="code-keyword">catch</span> (Exception e) { <span class="code-keyword">return</span> Util.throwCause(e); }</pre> </div></div> <p>compiles.</p> <p>Also, the patch doesn't actually cause try_catch.clj to pass. I suspect the RuntimeException being caught by try_catch.clj is not, in fact, originating from one of the four points you have corrected. (Indeed, I discovered something similar when working on my patch, but it was late, and I ended up solving it in such a way that I didn't have to find the actual source of this particular wrapped exception.) I'm still investigating...</p> <p>@Herwig thanks.<br/> A few issues with the patch:<br/> 1. doesn't build. Use "return Util.throwCause(...)" instead of just "Util.throwCause(...)" Otherwise the compiler complains about a missing return statement.<br/> 2. could you observe the whitespace conventions in the rest of the Java source files (tabs for indentation)?<br/> 3. The semantics of throwCause are different than before. Previously, we threw the Throwable only if t wasnt 1) null 2) an instance of Exception or 3) instance of Error. Subtle, and maybe it's a non-issue?</p> <p>The wrapping RuntimeException that causes try_catch.clj to fail is actually coming out of <tt>Compiler.eval(Object,boolean)</tt> without involving Reflector at all. </p> <p><a href="http://dev.clojure.org/jira/browse/CLJ-855" title="catch receives a RuntimeException rather than the expected checked exception"><del>CLJ-855</del></a> appears to be about more than just oddness in Reflector.java. My guess is it requires a more general solution.</p> <p>Oh, it seems I was too fast with the second commit. That's kind of embarrassing, sorry about that.<br/> Please only consider the first commit of the patch, where I implemented the chucked exception helpers.</p> <p>I had considered the issue with only rethrowing causes that are instances of Exception or Error. I dismissed it, because I considered it an artifact of not having chucked exceptions before. The throwCause is for unwrapping InvokationTargetExceptions and such. The only case where it could unwrap differently than the original code, is when having causes other than Exception or Error. i.e. a third subclass of Throwable, which is mostly unheard of (but should probably be unwrapped too).</p> <p>Rich, do you remember, if there are any further implications to the catch idiom in Reflector?</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"><span class="code-keyword">catch</span>(Exception e) { <span class="code-keyword">if</span>(e.getCause() <span class="code-keyword">instanceof</span> Exception) <span class="code-keyword">throw</span> Util.runtimeException(e.getCause()); <span class="code-keyword">else</span> <span class="code-keyword">if</span>(e.getCause() <span class="code-keyword">instanceof</span> Error) <span class="code-keyword">throw</span> (Error) e.getCause(); <span class="code-keyword">throw</span> Util.runtimeException(e); }</pre> </div></div> <hr /> <p>Regarding the actual source of the wrapping (Compiler.eval): It showed up in my `rgrep catch(Exception`, but I considered it part of the "other issue" (of general refactoring). In fact, I think it won't show up, mostly, because people noticed this issue when having <b>reflective calls</b> in clojure code, <b>which don't use eval</b>. So the test case is probably testing another source of RuntimeException wrapping.</p> <p>This shows, IMO, that most occurrences of `throw Util.runtimeException` should be replaced by `return Util.throwUnchecked`, but probably not all. This will require some scrutiny and careful testing. We need to consider every `catch (Exception e)` in the clj code base. </p> <p>I'll take a stab at it, tonight (CEST), also testing properly this time (sorry again, had some troubles with my build system yesterday)</p> <p>I've added a unit test which provokes this issue in the Reflector, as opposed to in the Compiler by way of eval. You can browse it here:</p> <p><a href="https://github.com/bpsm/clojure/commits/CLJ-855">https://github.com/bpsm/clojure/commits/CLJ-855</a></p> <p>I've refined my patch using the "try unwraps" strategy. The refined patch renames Clojure's <tt>try</tt> special form to <tt>try*</tt> and provides <tt>try</tt> with transparent unwrapping as a macro on top of <tt>try*</tt>. The curious may look here: </p> <p><a href="https://github.com/bpsm/clojure/commits/CLJ-855-try-unwraps">https://github.com/bpsm/clojure/commits/CLJ-855-try-unwraps</a></p> <p>I'll post new patches after I've had some sleep.</p> <p>I intend to try my hand at the "sneaky throws" strategy that seems to be all the rage here. <img class="emoticon" src="http://dev.clojure.org/jira/images/icons/emoticons/tongue.gif" height="20" width="20" align="absmiddle" alt="" border="0"/> My experimentation with Herwig's patch showed that it will require investigating some failing unit tests. I believe it will prove a more invasive change than "try unwraps", but may be worth it. A nice challenge.</p> <p><a href="http://dev.clojure.org/jira/browse/CLJ-855" title="catch receives a RuntimeException rather than the expected checked exception"><del>CLJ-855</del></a>-try-unwraps.patch:</p> <ol> <li>Teaches the Java portion of Clojure core to use <tt>WrappedException</tt> (a <tt>RuntimeException</tt>) when wrapping checked exceptions for re-throwing.</li> <li>Changes <tt>try</tt> to transparently unwrap (only!) <tt>WrappedException</tt>, passing the cause on to be handled by one of its catch clauses. This appears to solve <a href="http://dev.clojure.org/jira/browse/CLJ-855" title="catch receives a RuntimeException rather than the expected checked exception"><del>CLJ-855</del></a>.</li> <li>The original behavior of the try special form (no transparent unwrapping) is still available under the name <tt>try*</tt>.</li> </ol> <p><a href="http://dev.clojure.org/jira/browse/CLJ-855" title="catch receives a RuntimeException rather than the expected checked exception"><del>CLJ-855</del></a>-sneaky-throw.patch</p> <p>Using "sneaky throw" allows us to effectively rethrow checked exceptions without needing to declare or catch them. In effect, we're back to where we were before wrapping them all up in RTEs (8fda34e4c77...), but without needing to pollute seemingly every method signature in the Java portion of Clojure core with "throws Exception"</p> <p>This patch was less work than I expected (thanks be to sed) but I'm not as confident in its correctness as I am in that of the try-unwrap variant. I can't claim to have really groked Clojure's approach to exceptions.</p> <p>I don't understand why Reflector gives preference to rethrowing the cause of the exception it catches (especially now that we're no longer wrapping all over the place.)</p> <p>I also don't understand why Var.dissoc returns the exception it catches rather than throwing it. (I've left a big fat TODO comment there, which should go from any final version of this patch, should it be accepted.)</p> <p>Another option, not to make things even more complicated, is to revert the original commit. Perhaps I'm missing the rationale, but...</p> <p>Checked exceptions are irrelevant to Clojure code, since they are a fabrication of the Java compiler.</p> <p>Using Clojure code from the Java side could be confusing if you call invoke on IFn and get a checked exception even though it is not declared with a throws clause. It is also confusing to get checked exceptions wrapped in one or more layers of RTEs. Was it not sufficient the way it was before having IFn declare "throws Exception"?</p> <p><a href="http://dev.clojure.org/jira/browse/CLJ-855" title="catch receives a RuntimeException rather than the expected checked exception"><del>CLJ-855</del></a>-sneaky-throw.patch applied. Thanks Ben!</p> Approval Ok Global Rank Patch Code and Test