<< Back to previous view

[CLJ-1048] add test.generative to Clojure's tests Created: 21/Aug/12  Updated: 01/Mar/13  Resolved: 07/Sep/12

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: None
Fix Version/s: Release 1.5

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File add-test-generative.patch    
Patch: Code and Test
Approval: Ok


The 0.1.5 release of test generative includes data generators and specs based on data generation, plus a runner that can run both c.t and c.t.g. tests under the same umbrella.

Comment by Stuart Halloway [ 21/Aug/12 10:26 PM ]

Note that this patch also removes the cyclic load tests. Those tests relied on leaving broken code on the test classpath, which is very hostile to tooling (in this case the test runner's ability to walk the code in the test directory.)

I believe such tests are an antipattern, and should be implemented differently or e.g. placed in an ancillary test suite.

Comment by Fogus [ 22/Aug/12 10:32 AM ]

I agree with you position on the cyclic load tests. The patch looks sane to me and ran without hitch. A couple points of note:

  • This requires one to re-run antsetup – not a problem, but it might trip a few people
  • The final output of the test run is different than before. In the case of a successful run we now see only the generative success stats. Is the regular clojure.test stats included? It's not clear to me, but I don't think they are. In the fail case we see something like the following. I actually like the map output better, but one advantage to the stack trace output was that it sometimes helped identify code bugs. In any case, it would be nice to see an aggregate score at the end. This is a test.generative feature request I know.
     [java] {:clojure.test/vars (test-equality),
     [java]  :thread/name "main",
     [java]  :pid 8682,
     [java]  :thread 1,
     [java]  :type :assert/fail,
     [java]  :level :warn,
     [java]  :test/actual (not (not true)),
     [java]  :test/expected (not (= nil nil)),
     [java]  :line 28,
     [java]  :tstamp 1345649256308,
     [java]  :file "data_structures.clj"}

In all, I like this! Now we need to start thinking up more specs.

Comment by Stuart Halloway [ 22/Aug/12 11:51 AM ]

The stats reports are aggregate, i.e. they rollup both the generative and clojure.test stats. It would be straightforward to separate these in the report. I will look at doing that in the next drop of c.t.g.

The output already includes stack traces for errors, but not for test failures. I thought this was consistent with clojure.test, but maybe I missed something.

Comment by Fogus [ 22/Aug/12 12:59 PM ]

Thanks for the clarification Stu. You're right that the exception behavior is consistent, the confusion was my own.

Comment by Stuart Sierra [ 07/Sep/12 11:34 AM ]

Patches have been applied as of September 1, 2012.

Generated at Wed Jan 18 18:45:05 CST 2017 using JIRA 4.4#649-r158309.