<< Back to previous view

[CLJ-2378] [spec] instrument should check :ret and :fn specs Created: 16/Jul/18  Updated: 17/Jul/18  Resolved: 16/Jul/18

Status: Closed
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Feature Priority: Major
Reporter: jcr Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: spec


Problem: a function instrumented via clojure.spec.test.alpha/instrument checks the :args part of its spec, but not the :ret and :fn parts. It's confusing and forces people to use a 3rd-party library[1].

Solution: add an option to turn on :ret and :fn checking to clojure.spec.test.alpha/instrument.

[1]: https://github.com/jeaye/orchestra

Comment by Alex Miller [ 16/Jul/18 5:59 PM ]

The basic idea here is that instrument is designed to verify that a function is invoked correctly. For determining whether the internals of a function are correct, you can use `check`.

This has been discussed at great length, both on the core team and in the broader community and we’re familiar with all the arguments. While I’m not ruling out the possibility that we will do something in this area in the future, we are not currently planning on changing this, so I’m going to decline this for now. I’ll repeat, saying no to this now does not prevent us from saying yes later.

Comment by Ghadi Shayban [ 16/Jul/18 6:04 PM ]

There's also some commentary from Rich in Slack, but I don't have a link handy.
("we should do X" tickets are almost universally declined.)
dup of https://dev.clojure.org/jira/browse/CLJ-1961

Comment by jcr [ 17/Jul/18 10:44 AM ]

Sorry, I've only searched through open issues before creating this ticket.

Is it at all possible to keep this issue open so people could vote for it and subscribe to it, since (as Alex stated) it's not set in stone yet?

(The thing is, I was confused by this behavior and asked on irc and elsewhere, and was told by several people that it's weird and looks like a bug. I've filed it as a "feature" since the doc clearly states that's the intended behavior. Also, it's not quite clear how to use check to verify correctness of a function for a specific subset of possible inputs (think checking the fn in repl as you develop it) - maybe the spec guide could elaborate a bit more on what the supposed workflow should look like? Maybe that'd clear up some confusion; though, I understand that's out of the scope.)

Comment by Alex Miller [ 17/Jul/18 12:14 PM ]

As I said, we're aware of the request and why people want it. From a core team perspective, we don't need an issue to track it.

Use `check` if you want to use property based testing to verify the correctness of a function with respect to its spec. Generally if you want to test specific possible inputs, most people use example-based tests with clojure.test instead - that's not in the guide as it doesn't really have anything to do with spec. It is possible to override the args generator with a set spec of explicit inputs, but I'm not sure if the effort of that is worth the work most of the time.

Comment by jcr [ 17/Jul/18 12:57 PM ]

Well, fair enough!

Re example-based tests: what I had in mind is 1) calling a function (presumably in repl) with a specific input to see if the output still conforms to :ret and :fn specs; 2) testing a function A which calls spec'ed functions B1..Bn to see which of those fail their :ret/:fn specs (essentially it gives you free unit tests for each integration test). I agree that it's theoretically possible to do that in vanilla clojure.spec with a custom generator, but that certainly doesn't worth the effort.

Anyway, I guess you've already heard this argument, so I'll stop there.

[CLJ-2377] [spec] StackOverflowError in gen of recursive spec Created: 15/Jul/18  Updated: 15/Jul/18

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.10
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Bürgin Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: generator, spec

Clojure 1.10.0-alpha6

Approval: Triaged


Repro stripped down from larger piece of code:

clj -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha6"}, org.clojure/test.check {:mvn/version "0.10.0-alpha3"}}}'
(require '[clojure.spec.alpha :as s])

(s/def ::m (s/keys :req [::coll]))
(s/def ::coll (s/cat :m (s/? ::m)))

(s/conform ::m {::coll []})  ; => #:user{:coll {}}
(s/exercise ::m)  ; => StackOverflowError

Unlike in CLJ-2002, conform works fine here, but generation recurs for ever.

[CLJ-2376] [core.specs] Check early if let binding vector is even Created: 13/Jul/18  Updated: 16/Jul/18

Status: Open
Project: Clojure
Component/s: None
Affects Version/s: Release 1.9
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alexander Yakushev Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: spec

Attachments: Text File let-even.patch    
Patch: Code
Approval: Prescreened


If you miss a value in the let binding vector, Spec will return a rather verbose and misguiding message:

user=> (require '[clojure.spec.alpha :as s] '[clojure.core.specs.alpha :as cs])
user=> (s/explain ::cs/bindings '[a #_1 b 2])
In: [2] val: 2 fails spec: :clojure.core.specs.alpha/local-name at: [:binding :sym] predicate: simple-symbol?
In: [2] val: 2 fails spec: :clojure.core.specs.alpha/seq-binding-form at: [:binding :seq] predicate: vector?
In: [2] val: 2 fails spec: :clojure.core.specs.alpha/map-bindings at: [:binding :map] predicate: coll?
In: [2] val: 2 fails spec: :clojure.core.specs.alpha/map-special-binding at: [:binding :map] predicate: map?

The crux of the problem here is the odd number of parameters in the binding. Spec gets to it anyway, but if it first finds a parameter that is an invalid binding form, it will bark at that instead. With the suggested patch, the error looks like:

user=> (s/explain ::cs/bindings '[a #_1 b 2])
val: [a b 2] fails spec: :clojure.core.specs.alpha/bindings predicate: (even? (count %))

Prescreened by: Alex Miller

Generated at Thu Jul 19 14:27:40 CDT 2018 using JIRA 4.4#649-r158309.