test.check

defspec makes it impossible to specify :max-size (and seed)

Details

  • Type: Enhancement Enhancement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

While defspec is very valuable for integration with clojure.test, it is almost impossible to specify :max-size for the quick-check call. The only way I've found around it right now is by using the following macro:

(defmacro ^:private change-max-size
  [name max-size]
  `(alter-meta! (var ~name) assoc
                :test (fn [] (#'ct/assert-check
                             (assoc (~name ct/*default-test-count*
                                           :max-size ~max-size)
                               :test-var (str '~name))))))

This macro alters the meta of a named test such that the quick-check-call uses a specified expression for max-size, but obviously this is a very hacky solution. Preferably, I would be able to specify the :max-size value directly through defspec. I imagine that something like this could work:

(defspec my-test
  {:max-size 40, :num-tests 400}
  (my-test-here))

Essentially, if the optional argument (currently named default-times) is a map, inspect the map for values given, or use defaults if not.

Activity

Hide
Reid Draper added a comment -

I like the idea of making the first (optional) argument a map. This allows us to add in more options in the future, without breaking existing code.

Show
Reid Draper added a comment - I like the idea of making the first (optional) argument a map. This allows us to add in more options in the future, without breaking existing code.
Hide
John Walker added a comment -

Do we want to maintain backwards compatibility with

(defspec my-test number-of-times (my-test-here))

?

Show
John Walker added a comment - Do we want to maintain backwards compatibility with
(defspec my-test number-of-times (my-test-here))
?
Hide
Reid Draper added a comment -

I spoke with Gary Fredericks about this earlier today, we're both thinking it seems reasonable to keep the current Int as a value, but also accept a map. The map will contain the same optional keys as the quick-check function, with an additional num-tests key. That seem reasonable?

Show
Reid Draper added a comment - I spoke with Gary Fredericks about this earlier today, we're both thinking it seems reasonable to keep the current Int as a value, but also accept a map. The map will contain the same optional keys as the quick-check function, with an additional num-tests key. That seem reasonable?
Hide
John Walker added a comment -

Yes, I think that is the right approach. I'll get a first draft in for us tonight.

Show
John Walker added a comment - Yes, I think that is the right approach. I'll get a first draft in for us tonight.
Hide
Reid Draper added a comment -

Great, thanks! As an aside, I've noticed that quick-check takes var-args, and not a last-argument-map. I'd probably like to change this as well, but am not sure how to keep backward compatibility there...

Show
Reid Draper added a comment - Great, thanks! As an aside, I've noticed that quick-check takes var-args, and not a last-argument-map. I'd probably like to change this as well, but am not sure how to keep backward compatibility there...
Hide
John Walker added a comment -

This patch adds a map as an alternative for defspec's options. Users can specify max-size, num-tests and seed using it. It also changes the way in which the function arguments produced by defspec are qualified. Example: Before it was max-size_auto_1448, now it is max-size.

Show
John Walker added a comment - This patch adds a map as an alternative for defspec's options. Users can specify max-size, num-tests and seed using it. It also changes the way in which the function arguments produced by defspec are qualified. Example: Before it was max-size_auto_1448, now it is max-size.
John Walker made changes -
Field Original Value New Value
Attachment defspec.diff [ 13311 ]
Hide
John Walker added a comment -

Great, thanks!

You're welcome! Thanks for test.check.

As an aside, I've noticed that quick-check takes var-args, and not a last-argument-map. I'd probably like to change this as well, but am not sure how to keep backward compatibility there...

Probably the best way to maintain backward compatibility here would be to do some parsing on the varargs ourselves. It would require a check to see if the input is a map, etc. It's not a lot of trouble, but it's definitely a compromise.

Show
John Walker added a comment -
Great, thanks!
You're welcome! Thanks for test.check.
As an aside, I've noticed that quick-check takes var-args, and not a last-argument-map. I'd probably like to change this as well, but am not sure how to keep backward compatibility there...
Probably the best way to maintain backward compatibility here would be to do some parsing on the varargs ourselves. It would require a check to see if the input is a map, etc. It's not a lot of trouble, but it's definitely a compromise.
Hide
Reid Draper added a comment -

Merged, with some white-space cleanup and added tests in c3d6134a5eef3a3c45369cc3f3f9416d23d30f61. Thanks!

Show
Reid Draper added a comment - Merged, with some white-space cleanup and added tests in c3d6134a5eef3a3c45369cc3f3f9416d23d30f61. Thanks!
Reid Draper made changes -
Resolution Completed [ 1 ]
Status Open [ 1 ] Resolved [ 5 ]

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: