Clojure

[spec] Generic spec walking for clojure.spec

Details

  • Type: Feature Feature
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: Release 1.9
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    [org.clojure/spec.alpha "0.1.134"]

Description

Problem

To do runtime coercion, specs need to be walked twice to strip away the branching information: s/conform + s/unform. This introduced extra latency (see the sample below).

Proposal

New versatile s/walk* to support generic spec walking.

Current status

Still, when running s/conform + s/unform, we walk the specs twice - which is performance-wise suboptimal. Below is a sample, with Late 2013 MacBook Pro with 2,5 GHz i7, with JVM running as -server.

(require '[clojure.spec.alpha :as s])

(s/def ::id int?)
(s/def ::name string?)
(s/def ::languages (s/coll-of #{:clj :cljs} :into #{}))
(s/def ::street string?)
(s/def ::zip string?)
(s/def ::number int?)

(s/def ::address (s/keys
                   :req-un [::street ::zip ::number]))

(s/def ::user (s/keys
                :req [::id]
                :req-un [::name ::address]
                :opt-un [::languages]))

(def value {::id 1
            :name "Liisa"
            :languages #{:clj :cljs}
            :address {:street "Hämeenkatu"
                      :number 24
                      :zip "33200"}})

; 2.0 µs
(cc/quick-bench
  (s/conform ::user value))

; 6.2 µs
(cc/quick-bench
  (s/unform ::user (s/conform ::user value)))

Despite s/conform is relatively fast, we triple the latency in the sample when running also s/unform. As we know already that we are not interested in the branching info, we could just not emit those.

Suggestion

s/walk* to replace both s/confrom* and s/unform*, maybe even s/explain*. It would take extra mode argument, which would be a Keyword of one of the following:

  • :validate - return false on first failing spec
  • :conform - like the current s/conform*, maybe also return s/explain results?
  • :unform - like the current s/unform*
  • :coerce - s/conform* + s/unform*, could be optimized (e.g. if no branching info, just return the value)

The public apis could be remain the same (+ optional extra argument with CLJ-2116), and a new s/coerce to call the s/walk* with :coerce.

Results

Single sweep validation & coercion. Happy runtime.

Activity

Hide
Tommi Reiman added a comment -

Renamed the issue. Instead of Keyword argument, it should take a function to walk the spec to support arbitrary walking applications.

Show
Tommi Reiman added a comment - Renamed the issue. Instead of Keyword argument, it should take a function to walk the spec to support arbitrary walking applications.
Hide
Marco Molteni added a comment -

hello, any news ?

Show
Marco Molteni added a comment - hello, any news ?
Hide
Alex Miller added a comment -

No plans to look at this before the next batch of implementation changes, so it will be a while.

Show
Alex Miller added a comment - No plans to look at this before the next batch of implementation changes, so it will be a while.
Hide
Tommi Reiman added a comment -

There are three different versions for ~this in spec-tools now:

1) spec-walker: using `s/form`, walks the specs and values and calls a callback function with both, returning a new value. It can be used for things like coercion. It doesn't validate, need to call `s/valid?` and `s/explain-data` separately (2-3 calls in total)

2) spec-visitor: just walks the forms and return new forms. Used in things like spec => json-schema transformations. Not transforming values.

3) wrapped specs with overridden `s/conform*`, returning new value or error. It's the saddest of the three, because everything needs to be wrapped, but still the only one that knows how to walk over properly regex specs. Bundled transform + validate, like with Schema.

...

https://cljdoc.org/d/metosin/spec-tools/0.9.0/doc/spec-coercion has examples of the first/coercion, hopefully highlighting why support for coercion is important.

Is this issue on a (near) roadmap? Can we help with it? If not on the roadmap, is there a suggested way to do runtime value transformations with spec?

Show
Tommi Reiman added a comment - There are three different versions for ~this in spec-tools now: 1) spec-walker: using `s/form`, walks the specs and values and calls a callback function with both, returning a new value. It can be used for things like coercion. It doesn't validate, need to call `s/valid?` and `s/explain-data` separately (2-3 calls in total) 2) spec-visitor: just walks the forms and return new forms. Used in things like spec => json-schema transformations. Not transforming values. 3) wrapped specs with overridden `s/conform*`, returning new value or error. It's the saddest of the three, because everything needs to be wrapped, but still the only one that knows how to walk over properly regex specs. Bundled transform + validate, like with Schema. ... https://cljdoc.org/d/metosin/spec-tools/0.9.0/doc/spec-coercion has examples of the first/coercion, hopefully highlighting why support for coercion is important. Is this issue on a (near) roadmap? Can we help with it? If not on the roadmap, is there a suggested way to do runtime value transformations with spec?

People

Vote (17)
Watch (9)

Dates

  • Created:
    Updated: