test.check

size of args is conflated with depth in shrink tree

Details

  • Type: Defect Defect
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

This came up when working on TCHECK-114.

I had written a generator of the form (gen/frequency [[1 gen-1] [2 gen-2]]) and was trying to figure out why failures that originated in gen-2 didn't always shrink to gen-1.

When I examined the shrink tree, I realized that the shrinking algorithm did try shrinking to gen-1, but even after finding a failure there it would continue looking for alternatives and eventually shrunk to something from gen-2, because it was deeper in the shrink tree and therefore believed to be smaller.

So the shrinking algorithm has no way to compare the "size" of inputs except for their depth in the search tree, which is sometimes misleading. I doubt there's a general solution to this besides maybe adding an option to quick-check for a custom comparator function.

Activity

There are no comments yet on this issue.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated: