Clojure

add test.generative to Clojure's tests

Details

  • Type: Enhancement Enhancement
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: Release 1.5
  • Component/s: None
  • Labels:
    None
  • Patch:
    Code and Test
  • Approval:
    Ok

Description

The 0.1.5 release of test generative includes data generators and specs based on data generation, plus a runner that can run both c.t and c.t.g. tests under the same umbrella.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Stuart Halloway added a comment -

Note that this patch also removes the cyclic load tests. Those tests relied on leaving broken code on the test classpath, which is very hostile to tooling (in this case the test runner's ability to walk the code in the test directory.)

I believe such tests are an antipattern, and should be implemented differently or e.g. placed in an ancillary test suite.

Show
Stuart Halloway added a comment - Note that this patch also removes the cyclic load tests. Those tests relied on leaving broken code on the test classpath, which is very hostile to tooling (in this case the test runner's ability to walk the code in the test directory.) I believe such tests are an antipattern, and should be implemented differently or e.g. placed in an ancillary test suite.
Hide
Permalink
Fogus added a comment -

I agree with you position on the cyclic load tests. The patch looks sane to me and ran without hitch. A couple points of note:

  • This requires one to re-run antsetup – not a problem, but it might trip a few people
  • The final output of the test run is different than before. In the case of a successful run we now see only the generative success stats. Is the regular clojure.test stats included? It's not clear to me, but I don't think they are. In the fail case we see something like the following. I actually like the map output better, but one advantage to the stack trace output was that it sometimes helped identify code bugs. In any case, it would be nice to see an aggregate score at the end. This is a test.generative feature request I know.
     [java] {:clojure.test/vars (test-equality),
     [java]  :thread/name "main",
     [java]  :pid 8682,
     [java]  :thread 1,
     [java]  :type :assert/fail,
     [java]  :level :warn,
     [java]  :test/actual (not (not true)),
     [java]  :test/expected (not (= nil nil)),
     [java]  :line 28,
     [java]  :tstamp 1345649256308,
     [java]  :file "data_structures.clj"}

In all, I like this! Now we need to start thinking up more specs.

Show
Fogus added a comment - I agree with you position on the cyclic load tests. The patch looks sane to me and ran without hitch. A couple points of note:
  • This requires one to re-run antsetup – not a problem, but it might trip a few people
  • The final output of the test run is different than before. In the case of a successful run we now see only the generative success stats. Is the regular clojure.test stats included? It's not clear to me, but I don't think they are. In the fail case we see something like the following. I actually like the map output better, but one advantage to the stack trace output was that it sometimes helped identify code bugs. In any case, it would be nice to see an aggregate score at the end. This is a test.generative feature request I know.
     [java] {:clojure.test/vars (test-equality),
     [java]  :thread/name "main",
     [java]  :pid 8682,
     [java]  :thread 1,
     [java]  :type :assert/fail,
     [java]  :level :warn,
     [java]  :test/actual (not (not true)),
     [java]  :test/expected (not (= nil nil)),
     [java]  :line 28,
     [java]  :tstamp 1345649256308,
     [java]  :file "data_structures.clj"}
In all, I like this! Now we need to start thinking up more specs.
Hide
Permalink
Stuart Halloway added a comment -

The stats reports are aggregate, i.e. they rollup both the generative and clojure.test stats. It would be straightforward to separate these in the report. I will look at doing that in the next drop of c.t.g.

The output already includes stack traces for errors, but not for test failures. I thought this was consistent with clojure.test, but maybe I missed something.

Show
Stuart Halloway added a comment - The stats reports are aggregate, i.e. they rollup both the generative and clojure.test stats. It would be straightforward to separate these in the report. I will look at doing that in the next drop of c.t.g. The output already includes stack traces for errors, but not for test failures. I thought this was consistent with clojure.test, but maybe I missed something.
Hide
Permalink
Fogus added a comment -

Thanks for the clarification Stu. You're right that the exception behavior is consistent, the confusion was my own.

Show
Fogus added a comment - Thanks for the clarification Stu. You're right that the exception behavior is consistent, the confusion was my own.
Hide
Permalink
Stuart Sierra added a comment -

Patches have been applied as of September 1, 2012.

Show
Stuart Sierra added a comment - Patches have been applied as of September 1, 2012.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4#649-r158309) | Report a problem
Powered by a free Atlassian JIRA open source license for Clojure. Try JIRA - bug tracking software for your team.