Clojure

clojure.test fixtures don't work with test-ns-hook

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: Release 1.3
  • Component/s: None
  • Labels:
    None
  • Approval:
    Ok

Description

I just noticed something about clojure.test that isn't documented, and might not be by design.

The docs say:

"Fixtures allow you to run code before and after tests, to set up
the context in which tests should be run."

and also

"By default, these functions will search for all tests defined in
a namespace and run them in an undefined order. However, if you
are composing tests, as in the "arithmetic" example above, you
probably do not want the "addition" and "subtraction" tests run
separately. In that case, you must define a special function
named "test-ns-hook" that runs your tests in the correct order:

(defn test-ns-hook []
     (arithmetic))
"

The actual behavior is that fixtures are run if you don't specify your own test-ns-hook</code>. You can figure this out if you read the source, or from the docstring of <code>test-ns if you know how fixtures are implemented, but it's certainly not clear.

Might I suggest that one of the following be done?

  • The docs are extended or clarified to warn that fixtures will not be run if you define your own test-ns-hook, or
  • test-ns</code> be extended (and <code>test-all-vars</code> refactored) to call fixtures prior to calling the <code>test-ns-hook, or
  • An exported function be introduced (and test-all-vars refactored to use it) to thread execution via the ns fixtures, so that hooks can use it?

I'm leaning towards the second or third option; the third allows the test-ns-hook</code> to optionally invoke the fixtures, which adds flexibility, but of course if you're defining a <code>test-ns-hook you're at liberty to simply not define the fixtures at all (which lends weight to the second solution).

Note that it might only make sense to run "once" fixtures around a custom test-ns-hook, unless tests themselves are put in charge of running "each" fixtures.

Activity

Hide
Assembla Importer added a comment -

stuart.sierra said: I am hoping to replace clojure.test with lazytest, currently under development. http://github.com/stuartsierra/lazytest

Lazytest will have better support for fixtures, renamed contexts. Therefore, I am not continuing work on clojure.test.

Show
Assembla Importer added a comment - stuart.sierra said: I am hoping to replace clojure.test with lazytest, currently under development. http://github.com/stuartsierra/lazytest Lazytest will have better support for fixtures, renamed contexts. Therefore, I am not continuing work on clojure.test.
Hide
Assembla Importer added a comment -

stuart.sierra said: [file:bgJ3zGZzOr37CfeJe5cbLA]: documentation patch only

Show
Assembla Importer added a comment - stuart.sierra said: [file:bgJ3zGZzOr37CfeJe5cbLA]: documentation patch only
Hide
Assembla Importer added a comment -

stuart.sierra said: This is too hard to fix correctly, and test-ns-hook is very rarely used.

So I have merely updated the documentation to make it clear that fixtures and test-ns-hook are mutually incompatible.

Show
Assembla Importer added a comment - stuart.sierra said: This is too hard to fix correctly, and test-ns-hook is very rarely used. So I have merely updated the documentation to make it clear that fixtures and test-ns-hook are mutually incompatible.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: