test.check

Permit a data-structure containing generators to be used as a generator

Details

  • Type: Enhancement Enhancement
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

I'm working through some of the examples in John Hughes' "QuickCheck Testing for Fun and Profit" paper. I'd like the structures I generate in those tests to look like the this:

[:apply `reg [(gen-name) (gen/elements pids)]]

Rather than the way they must be expressed now:

(gen/tuple (gen/return :apply) (gen/return `reg) (gen/tuple (gen-name) (gen/elements pids)))

I think a simple recursive function could be used to generate these, with the caveat that currently generators create a map {:gen #<fn...>}, which I'd propose to change to a record so that we could distinguish between a generator and a map data-structure.

This seems to be implemented to good effect in Quvig QuickCheck already:

In general, Quviq QuickCheck permits any data-structure containing embedded generators to be used as a generator for data-structures of that shape—something which is very convenient for users, but quite impossible in Haskell, where embedding a generator in a data-structure would normally result in a type error. This technique is used throughout the generators written at Ericsson. – Hughes

Activity

Hide
Darrick Wiebe added a comment -

Playing around in the REPL today, I came up with this code that seems to work well converting arbitrary literals into generators:

(defprotocol LiteralGenerator
  (-literal->generator [literal]))

(defn literal->generator [literal]
  (cond
    (satisfies? LiteralGenerator literal) (-literal->generator literal)
    (vector? literal) (apply gen/tuple (mapv literal->generator literal))
    (and (map? literal) (= [:gen] (keys literal)) (fn? (:gen literal))) literal
    (map? literal) (gen/fmap (partial into {}) (literal->generator (mapv vec literal)))
    (set? literal) (gen/fmap set (literal->generator (vec literal)))
    (list? literal) (gen/fmap (partial apply list) (literal->generator (vec literal)))
    :else (gen/return literal)))

Generating from a record is probably something that would be generally useful, so I added this as well:

(defn record->generator
  ([record]
   ; Is there a better way to do this?
   (let [ctor (eval (read-string (str "map->" (last (clojure.string/split (str (class record)) #"\.")))))]
     (record->generator ctor record)))
  ([ctor record]
   (gen/fmap ctor (literal->generator (into {} record)))))

Which enables me to extend a record like this:

(defrecord Foo [a b]
  LiteralGenerator
  (-literal->generator [this] (record->generator map->AbcDef this)))

I haven't looked at the possibility of baking this code into test.check at all yet. I'd like feedback on what I've got so far before pursuing this any further.

Show
Darrick Wiebe added a comment - Playing around in the REPL today, I came up with this code that seems to work well converting arbitrary literals into generators:
(defprotocol LiteralGenerator
  (-literal->generator [literal]))

(defn literal->generator [literal]
  (cond
    (satisfies? LiteralGenerator literal) (-literal->generator literal)
    (vector? literal) (apply gen/tuple (mapv literal->generator literal))
    (and (map? literal) (= [:gen] (keys literal)) (fn? (:gen literal))) literal
    (map? literal) (gen/fmap (partial into {}) (literal->generator (mapv vec literal)))
    (set? literal) (gen/fmap set (literal->generator (vec literal)))
    (list? literal) (gen/fmap (partial apply list) (literal->generator (vec literal)))
    :else (gen/return literal)))
Generating from a record is probably something that would be generally useful, so I added this as well:
(defn record->generator
  ([record]
   ; Is there a better way to do this?
   (let [ctor (eval (read-string (str "map->" (last (clojure.string/split (str (class record)) #"\.")))))]
     (record->generator ctor record)))
  ([ctor record]
   (gen/fmap ctor (literal->generator (into {} record)))))
Which enables me to extend a record like this:
(defrecord Foo [a b]
  LiteralGenerator
  (-literal->generator [this] (record->generator map->AbcDef this)))
I haven't looked at the possibility of baking this code into test.check at all yet. I'd like feedback on what I've got so far before pursuing this any further.
Hide
Reid Draper added a comment -

So, I have kind of mixed feelings about this. I agree it's convenient, but I also worry that being loose like that can allow more accidents to occur, and doesn't force users to make the distinction between values and generators. For example, what if you do something like this: [int int]. Did you mean to type [gen/int gen/int], or do you really intend to have written the equivalent of [(gen/return int) (gen/return int)]? If every function that expects a generator can also take a value, we can no longer start adding error-checking to make sure you're correctly passing generators. On that same token, I do see the benefit of being able to use data-structure literals. What if we wrote a function that you had to use to create a generator with this syntax, something like: {{(gen/literal [:age gen/int])}}. That way you could opt-in to this syntax, but only within the scope of gen/literal?

Show
Reid Draper added a comment - So, I have kind of mixed feelings about this. I agree it's convenient, but I also worry that being loose like that can allow more accidents to occur, and doesn't force users to make the distinction between values and generators. For example, what if you do something like this: [int int]. Did you mean to type [gen/int gen/int], or do you really intend to have written the equivalent of [(gen/return int) (gen/return int)]? If every function that expects a generator can also take a value, we can no longer start adding error-checking to make sure you're correctly passing generators. On that same token, I do see the benefit of being able to use data-structure literals. What if we wrote a function that you had to use to create a generator with this syntax, something like: {{(gen/literal [:age gen/int])}}. That way you could opt-in to this syntax, but only within the scope of gen/literal?
Hide
Darrick Wiebe added a comment -

I agree that is a concern, and since you're not guaranteed to see generator output, it might be better to be explicit about using gen/literal. It's still a nice clean solution. Fortunately, that's exactly what I've written! We'd just need to rename literal->generator to literal and install it into the clojure.test.check.generators namespace.

Show
Darrick Wiebe added a comment - I agree that is a concern, and since you're not guaranteed to see generator output, it might be better to be explicit about using gen/literal. It's still a nice clean solution. Fortunately, that's exactly what I've written! We'd just need to rename literal->generator to literal and install it into the clojure.test.check.generators namespace.
Hide
Reid Draper added a comment -

I think the first step is changing generators to use Records. Interested in making a patch for that?

Show
Reid Draper added a comment - I think the first step is changing generators to use Records. Interested in making a patch for that?
Hide
Darrick Wiebe added a comment -

A very simple patch to use a Generator record rather than a simple {:gen ƒ) map.

Show
Darrick Wiebe added a comment - A very simple patch to use a Generator record rather than a simple {:gen ƒ) map.
Hide
Reid Draper added a comment -

I've applied the record-patch in ef132b5f85a07879f01417c9104aa6dea771fdb4. Thanks. I've also added a generator? helper function.

Show
Reid Draper added a comment - I've applied the record-patch in ef132b5f85a07879f01417c9104aa6dea771fdb4. Thanks. I've also added a generator? helper function.
Hide
Philip Potter added a comment - - edited

I spotted something which seemed relevant: ztellman/collection-check defines a tuple* fn which is like gen/tuple but automatically wraps literals with gen/return.

Notably, despite not solving the general case, this would have solved the example in the issue description.

Show
Philip Potter added a comment - - edited I spotted something which seemed relevant: ztellman/collection-check defines a tuple* fn which is like gen/tuple but automatically wraps literals with gen/return. Notably, despite not solving the general case, this would have solved the example in the issue description.

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated: