add test.generative to Clojure's tests
Description
Environment
Attachments
- 22 Aug 2012, 04:23 AM
Activity
Stuart Sierra September 7, 2012 at 5:34 PM
Patches have been applied as of September 1, 2012.
Fogus August 22, 2012 at 6:59 PM
Thanks for the clarification Stu. You're right that the exception behavior is consistent, the confusion was my own.
Stuart Halloway August 22, 2012 at 5:51 PM
The stats reports are aggregate, i.e. they rollup both the generative and clojure.test stats. It would be straightforward to separate these in the report. I will look at doing that in the next drop of c.t.g.
The output already includes stack traces for errors, but not for test failures. I thought this was consistent with clojure.test, but maybe I missed something.
Fogus August 22, 2012 at 4:32 PM
I agree with you position on the cyclic load tests. The patch looks sane to me and ran without hitch. A couple points of note:
This requires one to re-run
antsetup
– not a problem, but it might trip a few peopleThe final output of the test run is different than before. In the case of a successful run we now see only the generative success stats. Is the regular clojure.test stats included? It's not clear to me, but I don't think they are. In the fail case we see something like the following. I actually like the map output better, but one advantage to the stack trace output was that it sometimes helped identify code bugs. In any case, it would be nice to see an aggregate score at the end. This is a test.generative feature request I know.
[java] {:clojure.test/vars (test-equality),
[java] :thread/name "main",
[java] :pid 8682,
[java] :thread 1,
[java] :type :assert/fail,
[java] :level :warn,
[java] :test/actual (not (not true)),
[java] :test/expected (not (= nil nil)),
[java] :line 28,
[java] :tstamp 1345649256308,
[java] :file "data_structures.clj"}
In all, I like this! Now we need to start thinking up more specs.
Stuart Halloway August 22, 2012 at 4:26 AM
Note that this patch also removes the cyclic load tests. Those tests relied on leaving broken code on the test classpath, which is very hostile to tooling (in this case the test runner's ability to walk the code in the test directory.)
I believe such tests are an antipattern, and should be implemented differently or e.g. placed in an ancillary test suite.
Details
Assignee
UnassignedUnassignedReporter
Stuart HallowayStuart HallowayApproval
OkPatch
Code and TestPriority
MajorFix versions
Details
Details
Assignee
Reporter
Approval
Patch
Priority

The 0.1.5 release of test generative includes data generators and specs based on data generation, plus a runner that can run both c.t and c.t.g. tests under the same umbrella.