<< Back to previous view

[CLJ-1485] clojure.test.junit/with-junit-output doesn't handle multiple expressions Created: 29/Jul/14  Updated: 04/May/15

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.6
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: clojure.test, ft

Attachments: Text File clj-1485.patch    
Patch: Code
Approval: Triaged

 Description   

From the docstring description, and the use of ~@body, the intent of with-junit-output was to support a body containing multiple forms (for side-effects). However, calling it with multiple expressions will yield an error about the bindings in the let form.

(defmacro with-junit-output
  "Execute body with modified test-is reporting functions that write
  JUnit-compatible XML output."
  {:added "1.1"}
  [& body]
  `(binding [t/report junit-report
             *var-context* (list)
             *depth* 1]
     (t/with-test-out
       (println "<?xml version=\"1.0\" encoding=\"UTF-8\"?>")
       (println "<testsuites>"))
     (let [result# ~@body]
       (t/with-test-out (println "</testsuites>"))
       result#)))

Cause: The ~@body in the macro is spliced into code expecting a single expression.

Approach: Wrap a (do ) around the ~@body.

Patch: clj-1485.patch

Screened by: Alex Miller



 Comments   
Comment by Howard Lewis Ship [ 29/Jul/14 4:59 PM ]

Patch for issue





[CLJ-1391] Allow logical operators on assert expressions Created: 26/Mar/14  Updated: 26/Mar/14

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.5
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Sanel Zukan Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: clojure.test


 Description   

With current code, it is not possible to express logical operators on some clojure.test assert expressions. For example, this will work:

(is (thrown? Exception <some-expression>))

however, here will fail:

(is (not (thrown? Exception <some-expression>)))

since '(thrown?)' is not an ordinary function, but looks like. This also adds confusion which is hard to explain to others unless '(is)' code was shown first.

Also, if the one would like to implement macro (e.g. 'is-not-thrown?') in form:

(defmacro is-not-thrown? [e expr]
  `(is (not ('thrown? ~e ~expr))))

which could be even more confusing for a person not knowing how 'thrown?' is implemented.






[CLJ-1209] clojure.test does not print ex-info in error reports Created: 11/May/13  Updated: 14/May/15

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Critical
Reporter: Thomas Heller Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: clojure.test

Attachments: Text File 0002-CLJ-1209-show-ex-data-in-clojure-test.patch     File clj-test-print-ex-data.diff     Text File output-with-0002-patch.txt    
Patch: Code
Approval: Triaged

 Description   

clojure.test does not print the data attached to ExceptionInfo in error reports.

(use 'clojure.test)
(deftest ex-test (throw (ex-info "err" {:some :data})))
(ex-test)

ERROR in (ex-test) (core.clj:4591)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: err
 at clojure.core$ex_info.invoke (core.clj:4591)
    user/fn (NO_SOURCE_FILE:2)
    clojure.test$test_var$fn__7666.invoke (test.clj:704)
    clojure.test$test_var.invoke (test.clj:704)
    ...

Approach: In clojure.stacktrace, which clojure.test uses for printing exceptions, add a check for ex-data and pr it.

After:

ERROR in (ex-test) (core.clj:4591)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: err
{:some :data}
 at clojure.core$ex_info.invoke (core.clj:4591)
    user/fn (NO_SOURCE_FILE:3)
    clojure.test$test_var$fn__7667.invoke (test.clj:704)
    clojure.test$test_var.invoke (test.clj:704)

Patch: 0002-CLJ-1209-show-ex-data-in-clojure-test.patch



 Comments   
Comment by Alex Miller [ 20/Dec/13 9:53 AM ]

Great idea, thx for the patch!

Comment by Alex Miller [ 20/Dec/13 9:54 AM ]

Would be great to see a before and after example of the output.

Comment by Ivan Kozik [ 12/Jul/14 10:35 PM ]

Attaching sample output

Comment by Stuart Sierra [ 05/Sep/14 3:24 PM ]

As pointed out on IRC, there's a possible risk of trying to print an infinite lazy sequence that happened to be included in ex-data.

To mitigate, consider binding *print-length* and *print-level* to small numbers around the call to pr.

Comment by Stephen C. Gilardi [ 13/May/15 2:39 PM ]

http://dev.clojure.org/jira/browse/CLJ-1716 may cover this well enough that this issue can be closed.

Comment by Alex Miller [ 14/May/15 8:35 AM ]

I don't think 1716 covers it at all as clojure.test/clojure.stacktrace don't use the new throwable printing. But they could! And that might be a better solution than the patch here.

For example, the existing patch does not consider what to do about nested exceptions, some of which might have ex-data. The new printer handles all that in a consistent way.





[CLJ-866] Provide a clojure.test function to run a single test case with fixtures Created: 27/Oct/11  Updated: 22/Nov/13  Resolved: 22/Nov/13

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Release 1.6

Type: Enhancement Priority: Major
Reporter: Hugo Duncan Assignee: Unassigned
Resolution: Completed Votes: 11
Labels: clojure.test

Attachments: Text File CLJ-866-test.patch     Text File clj-866-test-vars.patch    
Patch: Code and Test
Approval: Ok

 Description   

clojure.test test cases are functions and can be invoked directly. However, in the case that the test relies on fixtures, this does not work. It should be possible to run a single test case with all fixtures applied.

Approach: Add new test-vars that takes a collection of vars to run.

Patch: code: clj-866-test-vars.patch, test: CLJ-866-test.patch

Screened by: Alex Miller



 Comments   
Comment by Anthony Grimes [ 22/Oct/12 6:17 PM ]

I just added clj-866-test-vars.patch (22/Oct/12 6:09PM).

I had to implement this hackishly in Leiningen a few days ago, so I'm very excited to get this functionality in clojure.test itself.

This patch adds a test-vars function that solves this problem (and is more general). You can test as many vars as you want with it, with fixtures. It works by grouping vars passed by their namespace and then running them all with appropriate fixtures applied. Being able to run a single test isn't the problem here, being able to run only specific tests is. If we wrote a function to run one test with fixtures but we actually needed to run several, just not all tests, we'd end up having to run once-fixtures more than once which is wasteful. I think test-vars is a good solution that solves both this problem and the one I just mentioned.

Comment by Alex Miller [ 04/May/13 9:36 AM ]

This is highly useful. Could you add a test to the patch?

Comment by Gary Fredericks [ 03/Aug/13 3:35 PM ]

Attached patch with a test, which should apply cleanly on top of Anthony's.

Also made sure a mostly-equivalent version (that used test-var) was failing without the new function.

Comment by Alex Miller [ 08/Aug/13 2:20 AM ]

New version of the patch incorporating feedback has been attached, switching to Vetted so someone can take a look again.





Generated at Fri May 29 13:33:52 CDT 2015 using JIRA 4.4#649-r158309.