Clojure

fn's created using defn should not lexical shadow the var that holds them

Details

  • Type: Enhancement Enhancement
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: Release 1.2
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

currently (defn foo [x] (foo x)) expands to something like (def foo
(fn foo [x] (foo x))) so the fn is bound to foo lexically in the scope
of the fn body.

because of this lexical shadowing self calls to fns defined with defn
do not incur the overhead of var dereferencing, I assume this is the
reason it was added.

the lexical shadowing also breaks memoization if you create the
memoized functions via (alter-var-root #'foo memoize) and several
macros here and there emit code in this style, since reusing defn is
the easiest way to get all the good juicy metadata bits like arglists,
etc.

1.3 has changes to eliminate some of the cost of going through the var.

since going through the var is much cheaper now it would be nice to
eliminate the lexical shadowing.

http://groups.google.com/group/clojure-dev/browse_thread/thread/33b52b24616967f?hl=en

Activity

Hide
Kevin Downey added a comment -

tests indicate that accessing the fn through the var vs. accessing through the lexical binding have very similar access times

https://gist.github.com/1013008

results in

$ java -jar clojure.jar ~/src/lexicaltest.clj
lexical binding run 0
"Elapsed time: 86.739 msecs"
lexical binding run 1
"Elapsed time: 8.062 msecs"
lexical binding run 2
"Elapsed time: 16.182 msecs"
var binding run 0
"Elapsed time: 61.136 msecs"
var binding run 1
"Elapsed time: 40.317 msecs"
var binding run 2
"Elapsed time: 11.641 msecs"
$

Show
Kevin Downey added a comment - tests indicate that accessing the fn through the var vs. accessing through the lexical binding have very similar access times https://gist.github.com/1013008 results in $ java -jar clojure.jar ~/src/lexicaltest.clj lexical binding run 0 "Elapsed time: 86.739 msecs" lexical binding run 1 "Elapsed time: 8.062 msecs" lexical binding run 2 "Elapsed time: 16.182 msecs" var binding run 0 "Elapsed time: 61.136 msecs" var binding run 1 "Elapsed time: 40.317 msecs" var binding run 2 "Elapsed time: 11.641 msecs" $
Hide
Jason Wolfe added a comment -

Here's another test that makes sure no JIT funny business (e.g. dead code elimination) is going on, and also looks at primitive hinted versions.

https://gist.github.com/1023816

var-obj result, time: 14930352 ,"Elapsed time: 965.577 msecs"
lex-obj result, time: 14930352 ,"Elapsed time: 957.741 msecs"
var-prim result, time: 14930352 ,"Elapsed time: 770.411 msecs"
lex-prim result, time: 14930352 ,"Elapsed time: 113.412 msecs"
nil

I'm not sure how to properly hint the var in the var-prim version to get truly comparable results. In any case, this use case should definitely be kept in mind.

Show
Jason Wolfe added a comment - Here's another test that makes sure no JIT funny business (e.g. dead code elimination) is going on, and also looks at primitive hinted versions. https://gist.github.com/1023816 var-obj result, time: 14930352 ,"Elapsed time: 965.577 msecs" lex-obj result, time: 14930352 ,"Elapsed time: 957.741 msecs" var-prim result, time: 14930352 ,"Elapsed time: 770.411 msecs" lex-prim result, time: 14930352 ,"Elapsed time: 113.412 msecs" nil I'm not sure how to properly hint the var in the var-prim version to get truly comparable results. In any case, this use case should definitely be kept in mind.
Hide
Kevin Downey added a comment -

I think the point is we want to make sure the jvm can optimize both just as well, so making the code purposefully unoptimizable is kind of silly, yes?

Show
Kevin Downey added a comment - I think the point is we want to make sure the jvm can optimize both just as well, so making the code purposefully unoptimizable is kind of silly, yes?
Hide
Jason Wolfe added a comment -

@Kevin In your gist, the return value of foo is not used, and foo has no side effects. A sufficiently smart compiler could optimize the calls out entirely, which would not tell us much about the runtime of foo.

Show
Jason Wolfe added a comment - @Kevin In your gist, the return value of foo is not used, and foo has no side effects. A sufficiently smart compiler could optimize the calls out entirely, which would not tell us much about the runtime of foo.
Hide
Aaron Bedra added a comment -

Rich: Given the performance testing (in the email thread) what is the next step here?

  • more thorough testing
  • patch?

Are there things that this might break that we need to consider?

Show
Aaron Bedra added a comment - Rich: Given the performance testing (in the email thread) what is the next step here?
  • more thorough testing
  • patch?
Are there things that this might break that we need to consider?
Hide
Rich Hickey added a comment -

Someone needs to make a patch, and then test perf with and without the patch. These simulated tests aren't necessarily indicative. Naive fib is an ok test once you have a patch. And yes, more things might break - needs careful assessment.

Show
Rich Hickey added a comment - Someone needs to make a patch, and then test perf with and without the patch. These simulated tests aren't necessarily indicative. Naive fib is an ok test once you have a patch. And yes, more things might break - needs careful assessment.
Hide
Nicola Mometto added a comment -

If I understand this ticket correctly, it looks like this has been done at some point in time.

Clojure 1.7.0-master-SNAPSHOT
user=> (macroexpand-1 '(defn x []))
(def x (clojure.core/fn ([])))
Show
Nicola Mometto added a comment - If I understand this ticket correctly, it looks like this has been done at some point in time.
Clojure 1.7.0-master-SNAPSHOT
user=> (macroexpand-1 '(defn x []))
(def x (clojure.core/fn ([])))

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated: