[TCHECK-9] Properly document how :max-size works Created: 04/Mar/14 Updated: 04/Jan/18 Resolved: 04/Jan/18
|Reporter:||Jean Niklas L'orange||Assignee:||Gary Fredericks|
I have had several issues where I thought increasing quick-check's num-tests argument would increase the time taken linearly, but surprisingly it grew quadratically. This is related to the auto-increasing of the size to call-gen, which I believe it isn't documented well enough. It also seems like other people have had trouble with this functionality, see the reply to announcement of clojure.test.check.
Preferably, it should be mentioned in the docstring and the tutorial that the size of the structures generated grows linearly with the amount of tests you have, up to 200. As a result, doubling the amount of tests does not necessarily double the amount of time taken to run the tests.
In some of my tests, this behaviour increases the running time exponentially, and I actually used quite some time figuring out why. The documentation on recursive generators decreases the size of the binary tree in half, but it is not explained why it is split in half. While obvious in hindsight, explicitly explaining that recursive structures should have an amount of nodes proportional to the input size argument and how it is done, would be beneficial.
|Comment by Reid Draper [ 04/Mar/14 8:50 PM ]|
+1. This is not well-documented now. Further, I'm still not 100% sure that halving inside of a recursive generator is the right thing to do. For some structures, log2 may be more appropriate. In the mailing list post you mention, gen/any is clearly not bounded enough at roughly size=90, consider it's using more than 1GB of heap.
|Comment by Reid Draper [ 14/Apr/14 8:58 PM ]|
I've made things a little better in 57df9b9b8f2fe61a798fbf683046e8d1aa128228.
|Comment by Gary Fredericks [ 04/Jan/18 8:23 AM ]|
I'm going to call this done based on the existence of this doc page.
I think other shortcomings, like exponential growth, or details of recursive-gen, are represented in other open tickets.