Clojure

clojure.test: `use-fixtures` should (or should be able to) run for for a `testing` context

Details

  • Type: Enhancement Enhancement
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:

Description

As implemented currently, the clojure.test/use-fixtures hooks will run on a per-deftest basis, but not on a per-testing (clojure.test/testing) basis.

So, for a given deftest with multiple testing clauses, use-fixtures will be only run once.

Coming from an RSpec (Ruby) background this is surprising - the default behavior is the opposite, and in my opinion more intuitive.

Personally, this resulted in me writing tests that gave false positives - i.e. it rendered a portion of my test suite useless until I noticed this (by mere luck).

Anecdotically, I commented the issue with another Clojure developer and he also suffered the issue once. I mean, I'm not alone in this one, and surely other developers have stumbled with this despite the lack of JIRA report until now.

In any case, the current working alternative (one deftest per test which depends on use-fixtures) can be overly verbose (in comparison with my original attempt), and also the need for doing that in the first place can be easy to forget.

I'd ask to:

  • Implement an option for use-fixtures called e.g. :run-nested?
  • Make this option mandatory to specify, i.e. programmers pass either :run-nested? true or :run-nested? false, unless a global default is set beforehand. People should be aware of this nuance and be forced to think about it at least once (else they risk false positives/negatives)

I hope this sounds like a reasonable, needed, non-breaking proposal.

Cheers - Victor

Activity

Hide
Alex Miller added a comment -

Hi Victor, it would be helpful if you provided a full code example in the description.

Fixtures to me seem pretty well defined in the docs (https://github.com/clojure/clojure/blob/master/src/clj/clojure/test.clj#L158-L193) and are strongly tied to the invocation of tests as functions in vars. Invoking them on a per-`testing` basis seems like significantly different to me.

Another option would be to create a new kind of fixture with a different scope that would get reinvoked in `testing` blocks in particular like a `:group` assertion. I guess that would also apply to nested `testing` blocks?

I don't actually have a good sense of what kind of test would need this, hence the request for an example.

Show
Alex Miller added a comment - Hi Victor, it would be helpful if you provided a full code example in the description. Fixtures to me seem pretty well defined in the docs (https://github.com/clojure/clojure/blob/master/src/clj/clojure/test.clj#L158-L193) and are strongly tied to the invocation of tests as functions in vars. Invoking them on a per-`testing` basis seems like significantly different to me. Another option would be to create a new kind of fixture with a different scope that would get reinvoked in `testing` blocks in particular like a `:group` assertion. I guess that would also apply to nested `testing` blocks? I don't actually have a good sense of what kind of test would need this, hence the request for an example.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated: