Clojure

Indirect function calls through Var instances fail to clear locals

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Completed
  • Affects Version/s: Release 1.4
  • Fix Version/s: Release 1.6
  • Component/s: None
  • Labels:
  • Environment:
    Probably all, but observed on Ubuntu 12.04, OpenJDK 6
  • Patch:
    Code
  • Approval:
    Ok

Description

If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

(defn total [xs] (reduce + 0 xs))
(total (range 1000000000))   ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error

Solution: Similar to RestFn, wrap each argN in Var inside a Util.ret1(argN, argN = null).

Patch: var-clear-locals-patch-v2.diff

Screened by: Alex Miller

  1. var-clear-locals.diff
    29/Nov/12 2:57 PM
    19 kB
    Timothy Baldridge
  2. var-clear-locals-patch-v2.diff
    22/Oct/13 9:13 AM
    18 kB
    Alex Miller
  3. var-clear-locals-patch-v2.txt
    05/Sep/13 6:18 PM
    18 kB
    Andy Fingerhut

Activity

Timothy Baldridge made changes -
Field Original Value New Value
Assignee Timothy Baldridge [ halgari ]
Timothy Baldridge made changes -
Description If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

(defn total [xs] (reduce + 0 xs))
(sum (range 1000000000)) ; this works, though takes a while
(#'sum (range 1000000000)) ; this dies with out of memory error

I can provide a patch if it would be useful. The fix should be trivial, something along the lines of wrapping each argN in clojure/lang/Var.java inside a Util.ret1(argN, argN = null) as is done in RestFn.java.
If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error

I can provide a patch if it would be useful. The fix should be trivial, something along the lines of wrapping each argN in clojure/lang/Var.java inside a Util.ret1(argN, argN = null) as is done in RestFn.java.
Timothy Baldridge made changes -
Approval Vetted [ 10003 ]
Assignee Timothy Baldridge [ halgari ]
Timothy Baldridge made changes -
Assignee Timothy Baldridge [ halgari ]
Timothy Baldridge made changes -
Attachment var-clear-locals.diff [ 11729 ]
Timothy Baldridge made changes -
Assignee Timothy Baldridge [ halgari ]
Timothy Baldridge made changes -
Patch Code [ 10001 ]
Alex Miller made changes -
Approval Vetted [ 10003 ] Triaged [ 10120 ]
Rich Hickey made changes -
Approval Triaged [ 10120 ] Vetted [ 10003 ]
Fix Version/s Release 1.6 [ 10157 ]
Andy Fingerhut made changes -
Attachment var-clear-locals-patch-v2.txt [ 12238 ]
Alex Miller made changes -
Approval Vetted [ 10003 ] Screened [ 10004 ]
Description If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error

I can provide a patch if it would be useful. The fix should be trivial, something along the lines of wrapping each argN in clojure/lang/Var.java inside a Util.ret1(argN, argN = null) as is done in RestFn.java.
If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

{code}
(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error
{code}

*Solution:* Similar to RestFn, wrap each argN in Var inside a Util.ret1(argN, argN = null).

*Patch:* var-clear-locals-patch-v2.txt

*Screened by:* Alex Miller
Alex Miller made changes -
Description If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

{code}
(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error
{code}

*Solution:* Similar to RestFn, wrap each argN in Var inside a Util.ret1(argN, argN = null).

*Patch:* var-clear-locals-patch-v2.txt

*Screened by:* Alex Miller
If you make a function call indirectly by invoking a Var object (which derefs itself and invokes the result), the invocation parameters remain in the thread's local stack for the duration of the function call, even though they are needed only long enough to be passed into the deref'd function. As a result, passing a lazy seq into a function invoked in its Var form may run out of memory if the seq is forced inside that function. For example:

{code}
(defn total [xs] (reduce + 0 xs))
(total (range 1000000000)) ; this works, though takes a while
(#'total (range 1000000000)) ; this dies with out of memory error
{code}

*Solution:* Similar to RestFn, wrap each argN in Var inside a Util.ret1(argN, argN = null).

*Patch:* var-clear-locals-patch-v2.diff

*Screened by:* Alex Miller
Attachment var-clear-locals-patch-v2.diff [ 12366 ]
Rich Hickey made changes -
Approval Screened [ 10004 ] Ok [ 10007 ]
Stuart Halloway made changes -
Resolution Completed [ 1 ]
Status Open [ 1 ] Closed [ 6 ]

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: