Clojure

StackOverflowError on exception in reducef for PersistentHashMap fold

Details

  • Type: Defect Defect
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Completed
  • Affects Version/s: Release 1.5
  • Fix Version/s: Release 1.6
  • Component/s: None
  • Labels:
  • Environment:
    Clojure 1.5.0-alpha4, Sun Java 1.6.0_35, with [org.codehaus.jsr166-mirror/jsr166y "1.7.0"]
  • Patch:
    Code and Test
  • Approval:
    Ok

Description

If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem.

user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError   clojure.lang.Numbers.add (Numbers.java:126)

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt

Cause: The problem area in the code catches Exception and swallows it - this is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L444

Approach: Throw the Exception and allow it to propagate instead of swallowing it.

Patch: sneakyThrow-clj-1058.diff

Screened by: Alex Miller

Activity

Timothy Baldridge made changes -
Field Original Value New Value
Assignee Timothy Baldridge [ halgari ]
Timothy Baldridge made changes -
Approval Vetted [ 10003 ]
Priority Minor [ 4 ] Major [ 3 ]
Assignee Timothy Baldridge [ halgari ]
Rich Hickey made changes -
Fix Version/s Release 1.6 [ 10157 ]
Alex Miller made changes -
Labels reducers
Bruce Adams made changes -
Attachment patch-for-clj-1058.diff [ 12266 ]
Andy Fingerhut made changes -
Patch Code [ 10001 ]
Bruce Adams made changes -
Attachment declare-throws-clj-1058.diff [ 12269 ]
Bruce Adams made changes -
Description If reducef throws an exception, the PHM fold code can descend into an infinite loop, causing a stack overflow which masks the problem. This situation is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).

To reproduce:
{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
;; boom
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt
If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem. The problem area in the code is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).

To reproduce:
{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt
Bruce Adams made changes -
Assignee Bruce Adams [ bruceadams ]
Bruce Adams made changes -
Attachment patch-with-tests-clj-1058.diff [ 12334 ]
Bruce Adams made changes -
Attachment patch-for-clj-1058.diff [ 12266 ]
Alex Miller made changes -
Patch Code [ 10001 ] Code and Test [ 10002 ]
Alex Miller made changes -
Description If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem. The problem area in the code is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).

To reproduce:
{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt
If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem.

{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt

*Cause:* The problem area in the code catches Exception and swallows it - this is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L444

*Approach:* Throw the Exception instead of swallowing it to allow it to propagate.

*Patch:* patch-with-tests-clj-1058.diff

*Screened by:*
Alex Miller made changes -
Approval Vetted [ 10003 ] Incomplete [ 10006 ]
Bruce Adams made changes -
Attachment declare-throws-clj-1058.diff [ 12269 ]
Bruce Adams made changes -
Attachment patch-with-tests-clj-1058.diff [ 12334 ]
Bruce Adams made changes -
Attachment declarative-clj-1058.diff [ 12694 ]
Attachment private-interface-INode.diff [ 12693 ]
Bruce Adams made changes -
Attachment sneakyThrow-clj-1058.diff [ 12707 ]
Alex Miller made changes -
Approval Incomplete [ 10006 ] Vetted [ 10003 ]
Description If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem.

{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt

*Cause:* The problem area in the code catches Exception and swallows it - this is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L444

*Approach:* Throw the Exception instead of swallowing it to allow it to propagate.

*Patch:* patch-with-tests-clj-1058.diff

*Screened by:*
If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem.

{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt

*Cause:* The problem area in the code catches Exception and swallows it - this is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L444

*Approach:* Throw the Exception and allow it to propagate instead of swallowing it.

*Patch:* sneakyThrow-clj-1058.diff

*Screened by:*
Alex Miller made changes -
Approval Vetted [ 10003 ] Incomplete [ 10006 ]
Bruce Adams made changes -
Attachment declarative-clj-1058.diff [ 12694 ]
Bruce Adams made changes -
Attachment private-interface-INode.diff [ 12693 ]
Bruce Adams made changes -
Attachment sneakyThrow-clj-1058.diff [ 12707 ]
Bruce Adams made changes -
Attachment sneakyThrow-clj-1058.diff [ 12765 ]
Alex Miller made changes -
Approval Incomplete [ 10006 ] Vetted [ 10003 ]
Alex Miller made changes -
Approval Vetted [ 10003 ] Screened [ 10004 ]
Description If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem.

{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt

*Cause:* The problem area in the code catches Exception and swallows it - this is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L444

*Approach:* Throw the Exception and allow it to propagate instead of swallowing it.

*Patch:* sneakyThrow-clj-1058.diff

*Screened by:*
If reducef throws an exception, the exception is swallowed sometimes (that is: not propagated up to the caller of r/fold). This can lead to infinite recursion, causing a stack overflow which masks the problem.

{code}
user> (require '[clojure.core.reducers :as r])
nil
user> (r/fold (fn ([]) ([ret k v] (+ 3 "foo") ret)) (into {} (map (juxt identity identity) (range 10000))))
StackOverflowError clojure.lang.Numbers.add (Numbers.java:126)
{code}

This results in a stack like: https://raw.github.com/gist/3bab917287a7fd635a84/f38bfe3e270556e467f3fc02062af7ea10781390/gistfile1.txt

*Cause:* The problem area in the code catches Exception and swallows it - this is commented "aargh" in PersistentHashMap.java line 444 (as of 412a51d).
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentHashMap.java#L444

*Approach:* Throw the Exception and allow it to propagate instead of swallowing it.

*Patch:* sneakyThrow-clj-1058.diff

*Screened by:* Alex Miller
Rich Hickey made changes -
Approval Screened [ 10004 ] Ok [ 10007 ]
Stuart Halloway made changes -
Resolution Completed [ 1 ]
Status Open [ 1 ] Closed [ 6 ]

People

Vote (1)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: