Clojure

clojure.core/set should use transients

Details

  • Type: Enhancement Enhancement
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: Release 1.5, Release 1.6
  • Fix Version/s: Release 1.7
  • Component/s: None
  • Labels:
  • Patch:
    Code
  • Approval:
    Screened

Description

CLJ-1384-p2 uses transients for both create and createWithCheck. This is consistent with the current implementation for map.

clojure.core/vec calls (more or less) PersistentVector.create(...), which uses a transient vector to build up the result.

clojure.core/set on the other hand, calls PersistentHashSet.create(...), which repeatedly calls .cons on a PersistentHashSet, with all the associated speed/GC issues.

Operation count now w/transients
set 5 1.771924 µs 1.295637 µs
into 5 1.407925 µs 1.402995 µs
set 1000000 2.499264 s 1.196653 s
into 1000000 0.977555 s 1.006951 s

Patch: CLJ-1384-p2.patch
Screened by: Stu

  1. CLJ-1384-p1.patch
    15/Mar/14 10:46 PM
    2 kB
    Gary Fredericks
  2. CLJ-1384-p2.patch
    27/Jun/14 12:57 PM
    4 kB
    Stuart Halloway
  3. set-bench.tar
    11/Apr/14 11:15 AM
    6 kB
    Ambrose Bonnaire-Sergeant

Activity

Gary Fredericks made changes -
Field Original Value New Value
Description {{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.
{{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

h2. Benchmarks

(Using criterium)

h3. Currently

h4. Small collections (5 elements)

{{set}} averages 1.771924 µs
{{into}} averages 1.407925 µs

h4. Large collections (1,000,000 elements)

{{set}} averages 2.499264 sec
{{into}} averages 977.555243 ms
Alex Miller made changes -
Approval Triaged [ 10120 ]
Labels performance
Gary Fredericks made changes -
Description {{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

h2. Benchmarks

(Using criterium)

h3. Currently

h4. Small collections (5 elements)

{{set}} averages 1.771924 µs
{{into}} averages 1.407925 µs

h4. Large collections (1,000,000 elements)

{{set}} averages 2.499264 sec
{{into}} averages 977.555243 ms
{{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

h2. Benchmarks

(Using criterium)

h3. Currently

h4. Small collections (5 elements)

{{set}} averages 1.771924 µs
{{into}} averages 1.407925 µs

h4. Large collections (1,000,000 elements)

{{set}} averages 2.499264 sec
{{into}} averages 977.555243 ms

h3. After transients

h4. Small collections (5 elements)

{{set}} averages 1.295637 µs
{{into}} averages 1.402995 µs

h4. Large collections (1,000,000 elements)

{{set}} averages 1.196653 sec
{{into}} averages 1.006951 sec
Gary Fredericks made changes -
Attachment CLJ-1384-p1.patch [ 12880 ]
Andy Fingerhut made changes -
Patch Code [ 10001 ]
Ambrose Bonnaire-Sergeant made changes -
Attachment set-bench.tar [ 12932 ]
Alex Miller made changes -
Description {{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

h2. Benchmarks

(Using criterium)

h3. Currently

h4. Small collections (5 elements)

{{set}} averages 1.771924 µs
{{into}} averages 1.407925 µs

h4. Large collections (1,000,000 elements)

{{set}} averages 2.499264 sec
{{into}} averages 977.555243 ms

h3. After transients

h4. Small collections (5 elements)

{{set}} averages 1.295637 µs
{{into}} averages 1.402995 µs

h4. Large collections (1,000,000 elements)

{{set}} averages 1.196653 sec
{{into}} averages 1.006951 sec
{{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

|Operation|count|now|w/transients|
|set|5|1.771924 µs|1.295637 µs|
|into|5|1.407925 µs|1.402995 µs|
|set|1000000|2.499264 s|1.196653 s|
|into|1000000|0.977555 s|1.006951 s|
Alex Miller made changes -
Priority Minor [ 4 ] Major [ 3 ]
Rich Hickey made changes -
Approval Triaged [ 10120 ] Vetted [ 10003 ]
Rich Hickey made changes -
Fix Version/s Release 1.7 [ 10250 ]
Stuart Halloway made changes -
Approval Vetted [ 10003 ] Screened [ 10004 ]
Description {{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

|Operation|count|now|w/transients|
|set|5|1.771924 µs|1.295637 µs|
|into|5|1.407925 µs|1.402995 µs|
|set|1000000|2.499264 s|1.196653 s|
|into|1000000|0.977555 s|1.006951 s|
CLJ-1384-p2 uses transients for both create and createWithCheck. This is consistent with the current implementation for map.

{{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

|Operation|count|now|w/transients|
|set|5|1.771924 µs|1.295637 µs|
|into|5|1.407925 µs|1.402995 µs|
|set|1000000|2.499264 s|1.196653 s|
|into|1000000|0.977555 s|1.006951 s|
Attachment CLJ-1384-p2.patch [ 13104 ]
Alex Miller made changes -
Description CLJ-1384-p2 uses transients for both create and createWithCheck. This is consistent with the current implementation for map.

{{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

|Operation|count|now|w/transients|
|set|5|1.771924 µs|1.295637 µs|
|into|5|1.407925 µs|1.402995 µs|
|set|1000000|2.499264 s|1.196653 s|
|into|1000000|0.977555 s|1.006951 s|
CLJ-1384-p2 uses transients for both create and createWithCheck. This is consistent with the current implementation for map.

{{clojure.core/vec}} calls (more or less) {{PersistentVector.create(...)}}, which uses a transient vector to build up the result.

{{clojure.core/set}} on the other hand, calls {{PersistentHashSet.create(...)}}, which repeatedly calls {{.cons}} on a {{PersistentHashSet}}, with all the associated speed/GC issues.

|Operation|count|now|w/transients|
|set|5|1.771924 µs|1.295637 µs|
|into|5|1.407925 µs|1.402995 µs|
|set|1000000|2.499264 s|1.196653 s|
|into|1000000|0.977555 s|1.006951 s|

*Patch:* CLJ-1384-p2.patch
*Screened by:* Stu

People

Vote (4)
Watch (3)

Dates

  • Created:
    Updated: