Clojure

'eval' of closures or fns with runtime metadata within a call expr yields "No matching ctor found" exceptions

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Declined
  • Affects Version/s: Release 1.5
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

I ran into some issues with 'eval' when writing compilation strategies for Graph. It seems these may have been known for some time [1], but I couldn't find a ticket for them, so here we are.

Clojure docs [2] say "If the operator is not a special form or macro, the call is considered a function call. Both the operator and the operands (if any) are evaluated, from left to right," and "Any object other than those discussed above will evaluate to itself." While bare fns do seem to evaluate to themselves in all cases, when in a call expression, the evaluation of the operator fails on fn objects that are closures or have run-time metadata applied:

;; raw non-closures are fine
user> (eval (fn [x] (inc x)))
#<user$eval30559$fn_30560 user$eval30559$fn_30560@354ee11c>

;; raw closures are fine
user> (eval (let [y 1] (fn [x] (+ x y))))
#<user$eval30511$fn_30512 user$eval30511$fn_30512@3bac3a34>

;; non-closures in exprs are fine
user> (eval `(~(fn [x] (inc x)) 1))
2

;; but closures in exprs cause an error
user> (eval `(~(let [y 1] (fn [x] (+ x y))) 1))
IllegalArgumentException No matching ctor found for class user$eval30535$fn__30536 clojure.lang.Reflector.invokeConstructor (Reflector.java:163)

;; as do fns with metadata in exprs
user> (eval `(~(with-meta (fn [x] (inc x)) {:x 1}) 1))
IllegalArgumentException No matching ctor found for class clojure.lang.AFunction$1 clojure.lang.Reflector.invokeConstructor (Reflector.java:163)

[1] http://stackoverflow.com/a/11287181
[2] http://clojure.org/evaluation

Activity

Hide
Dmitri Gaskin added a comment -

I have run into this as well. I ran into it while using https://github.com/plumatic/schema + a memoized predicate function + https://github.com/metosin/compojure-api. After hours of debugging, I was able to narrow it down to:

```
user=> (let [foo inc] (eval `(~foo 1)))
2
user=> (let [foo (memoize inc)] (eval `(~foo 1)))

IllegalArgumentException No matching ctor found for class clojure.core$memoize$fn__5708 clojure.lang.Reflector.invokeConstructor (Reflector.java:163)
```

Show
Dmitri Gaskin added a comment - I have run into this as well. I ran into it while using https://github.com/plumatic/schema + a memoized predicate function + https://github.com/metosin/compojure-api. After hours of debugging, I was able to narrow it down to: ``` user=> (let [foo inc] (eval `(~foo 1))) 2 user=> (let [foo (memoize inc)] (eval `(~foo 1))) IllegalArgumentException No matching ctor found for class clojure.core$memoize$fn__5708 clojure.lang.Reflector.invokeConstructor (Reflector.java:163) ```
Hide
Kevin Downey added a comment -

basically, this cannot work. as it stands now, for some set of fn objects (including closures) eval fails, because the compiler cannot embed arbitrary function objects in the emitted bytecode. for another set of fn objects eval succeeds, because there is a fall back in eval, where it tries to embed arbitrary objects in the byte code by invoking a no argument constructor on the class of the object. closures fail to work in that fall back because the classes generated for fns that are closures in clojure don't have 0-arg constructors.

don't eval function objects

Show
Kevin Downey added a comment - basically, this cannot work. as it stands now, for some set of fn objects (including closures) eval fails, because the compiler cannot embed arbitrary function objects in the emitted bytecode. for another set of fn objects eval succeeds, because there is a fall back in eval, where it tries to embed arbitrary objects in the byte code by invoking a no argument constructor on the class of the object. closures fail to work in that fall back because the classes generated for fns that are closures in clojure don't have 0-arg constructors. don't eval function objects
Hide
Kevin Downey added a comment -

for a work around maybe try using dynamic binding?

Show
Kevin Downey added a comment - for a work around maybe try using dynamic binding?
Hide
Jason Wolfe added a comment -

Here is a workaround that we use in graph: https://github.com/plumatic/plumbing/blob/master/src/plumbing/graph/positional.clj#L45

Basically, rather than evaluating a form with function objects inside, e.g. `(eval (... f))`, we do: `((eval (fn [x] (... x))) f)`.

Show
Jason Wolfe added a comment - Here is a workaround that we use in graph: https://github.com/plumatic/plumbing/blob/master/src/plumbing/graph/positional.clj#L45 Basically, rather than evaluating a form with function objects inside, e.g. `(eval (... f))`, we do: `((eval (fn [x] (... x))) f)`.
Hide
Steve Miner added a comment -

I've worked around someting like this using `intern`. My situation was a bit different as I was trying to use macros to precompile some calculations into a function to be called at runtime. My first attempt was to embed the actual function, which of course gave me the "no matching ctor" error.

Rewriting the original problem, and introducing a scratch var tmp_42, we have:

(eval `(~(let [y 1] (intern *ns* 'tmp_42 (fn [x] (+ x y)))) 1))
;=> 2

Note, intern returns the new var so we just return that as our function to be called. Maybe you'd want to use (gensym) instead of tmp_42 to be safer. And maybe you'd want to use a different namespace. The main point is that using a var to hold the function makes it easier to access in a different context.

Show
Steve Miner added a comment - I've worked around someting like this using `intern`. My situation was a bit different as I was trying to use macros to precompile some calculations into a function to be called at runtime. My first attempt was to embed the actual function, which of course gave me the "no matching ctor" error. Rewriting the original problem, and introducing a scratch var tmp_42, we have:
(eval `(~(let [y 1] (intern *ns* 'tmp_42 (fn [x] (+ x y)))) 1))
;=> 2
Note, intern returns the new var so we just return that as our function to be called. Maybe you'd want to use (gensym) instead of tmp_42 to be safer. And maybe you'd want to use a different namespace. The main point is that using a var to hold the function makes it easier to access in a different context.
Hide
Nicola Mometto added a comment -

I personally don't think this should be considered a bug at all.
The docstring for eval explicitely states that eval takes a data structure, a function object is not a data structure and thus should not be expected to be evaluable.

Show
Nicola Mometto added a comment - I personally don't think this should be considered a bug at all. The docstring for eval explicitely states that eval takes a data structure, a function object is not a data structure and thus should not be expected to be evaluable.
Hide
Alex Miller added a comment -

Yeah, I've reached essentially the same conclusion.

Show
Alex Miller added a comment - Yeah, I've reached essentially the same conclusion.
Alex Miller made changes -
Field Original Value New Value
Status Open [ 1 ] Closed [ 6 ]
Resolution Declined [ 2 ]
Hide
Kevin Downey added a comment -

if anything the bug is that it works with some function objects and not with others

Show
Kevin Downey added a comment - if anything the bug is that it works with some function objects and not with others
Hide
Richard Davies added a comment -

I agree with Kevin's last comment - if this bug is closed because functions are not valid data structures, then the cases where some functions do work would therefore be a bug.

Secondly, the error message is unhelpful. "No matching ctor found" should be replaced with something close to the actual cause such as "a function is not valid data structures for eval" or something similar.

Show
Richard Davies added a comment - I agree with Kevin's last comment - if this bug is closed because functions are not valid data structures, then the cases where some functions do work would therefore be a bug. Secondly, the error message is unhelpful. "No matching ctor found" should be replaced with something close to the actual cause such as "a function is not valid data structures for eval" or something similar.
Hide
Jason Wolfe added a comment -

Much of my initial confusion stemmed from http://clojure.org/reference/evaluation, which still ends with "Any object other than those discussed above will evaluate to itself" – which seems to contradict the "data structure" viewpoint of eval.

Show
Jason Wolfe added a comment - Much of my initial confusion stemmed from http://clojure.org/reference/evaluation, which still ends with "Any object other than those discussed above will evaluate to itself" – which seems to contradict the "data structure" viewpoint of eval.
Hide
Jason Wolfe added a comment -

To clarify, I don't particularly care either way (although it would make some rare instances more convenient if this worked). It just seems like this is a bit of a dark corner currently, so if the behavior isn't going to change, should I open an issue on https://github.com/clojure/clojure-site/blob/master/content/reference/evaluation.adoc instead? (I would offer to submit a PR, but I'm not quite clear on where the lines between "evaluates to itself", "errors", and "one of the above / UB" should be drawn.)

Show
Jason Wolfe added a comment - To clarify, I don't particularly care either way (although it would make some rare instances more convenient if this worked). It just seems like this is a bit of a dark corner currently, so if the behavior isn't going to change, should I open an issue on https://github.com/clojure/clojure-site/blob/master/content/reference/evaluation.adoc instead? (I would offer to submit a PR, but I'm not quite clear on where the lines between "evaluates to itself", "errors", and "one of the above / UB" should be drawn.)
Hide
Alex Miller added a comment -

@Richard - if someone wanted to file a ticket for the error message in this case, I think that would be reasonable. I can't imagine we would make something that works now stop working. I see your point and all, just in the grand scheme of how we spend our time, that does not seem to make sense to me.

@Jason - probably best just to file an issue so I can run something by Rich eventually.

Show
Alex Miller added a comment - @Richard - if someone wanted to file a ticket for the error message in this case, I think that would be reasonable. I can't imagine we would make something that works now stop working. I see your point and all, just in the grand scheme of how we spend our time, that does not seem to make sense to me. @Jason - probably best just to file an issue so I can run something by Rich eventually.
Hide
Jason Wolfe added a comment -
Show
Jason Wolfe added a comment - Thanks Alex, I just opened https://github.com/clojure/clojure-site/issues/95
Hide
Richard Davies added a comment -
Show
Richard Davies added a comment - Fair enough Alex. Opened http://dev.clojure.org/jira/browse/CLJ-1928

People

Vote (3)
Watch (4)

Dates

  • Created:
    Updated:
    Resolved: