test.check

Refactor c.t.c/quick-check as a state machine to provide more extension points

Details

  • Type: Enhancement Enhancement
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Patch:
    Code and Test

Description

We can think of the quick-check process as a state machine with a state flow like:

       started
          v
  trying, trying, [...]
          v
succeeded | failure
                 v
    shrinking, shrinking, [...]
                 v
               shrunk"

The process starts, runs trials based on the generated values and either it succeeds or it fails. If it succeeds, the process is done. If it fails, then it runs successive shrinks of the failed args until it gets to the terminal shrunk state.

Modelling the process this way, we can call a step-fn at interesting points of the process that serves two purposes:

  • Provide feedback to the user about the process (via side-effects)
  • Augment/modify the state being tracked by the process, making it easy to implement things like gracefully aborting the process before it finishes, calculating statistics on the generated values (TCHECK-87), add timestamps (TCHECK-8, TCHECK-95, TCHECK-96), etc.

The attached patch aims at being 100% backwards compatible. Adds a new clojure.test.check2/quick-check function. The "old" clojure.test.check/quick-check is re-implemented by calling the new one, maintaining all the current behavior by lifting the reporter-fn into a step-fn via a reporter-fn->step-fn adapter fn.

EDIT: Added an alternative patch (TCHECK126-test.check-refactor.patch) that refactors `c.t.c/quick-check` "in-place", instead of adding a new test.check2 namespace. The most important change is that it switches from the reporter-fn to a step-fn which gives the possibility to drive/augment the quick-check process by feeding back changes to the quick-check state (as the return value of step-fn).

Activity

Hide
Nicolás Berger added a comment -

Added an alternative patch (TCHECK126-test.check-refactor.patch) that refactors `c.t.c/quick-check` "in-place", instead of adding a new test.check2 namespace. The most important change is that it switches from the reporter-fn to a step-fn which gives the possibility to drive/augment the quick-check process by feeding back changes to the quick-check state (as the return value of step-fn).

Even though reporter-fn has been in master for quite some time, it hasn't been part of a release, so I'm proposing the switch from reporter-fn to this new step-fn with the hope that it can be incorporated before the next release.

Show
Nicolás Berger added a comment - Added an alternative patch (TCHECK126-test.check-refactor.patch) that refactors `c.t.c/quick-check` "in-place", instead of adding a new test.check2 namespace. The most important change is that it switches from the reporter-fn to a step-fn which gives the possibility to drive/augment the quick-check process by feeding back changes to the quick-check state (as the return value of step-fn). Even though reporter-fn has been in master for quite some time, it hasn't been part of a release, so I'm proposing the switch from reporter-fn to this new step-fn with the hope that it can be incorporated before the next release.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated: