<< Back to previous view

[TBENCH-16] Add performance testing for STM Created: 23/Feb/12  Updated: 23/Feb/12

Status: Open
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Stefan Kamphausen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: performance, test
Environment:

64bit Linux, Quad-core


Attachments: File stm-perf-TBENCH-16.diff    

 Description   

Write a test-suite to detect regressions in STM performance.



 Comments   
Comment by Stefan Kamphausen [ 23/Feb/12 2:38 PM ]

Patch which adds an stm-namespace containing a collection of performance test functions.





[TBENCH-15] Alioth nbody Created: 21/Jan/12  Updated: 22/Jan/12  Resolved: 22/Jan/12

Status: Resolved
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by David Nolen [ 22/Jan/12 12:36 PM ]

Done, https://github.com/clojure/test.benchmark/commit/b0772ea5858a3924b15d87003cb039af20c0cfa7





[TBENCH-14] Alioth fannkuch-redux Created: 21/Jan/12  Updated: 22/Jan/12

Status: In Progress
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: David Nolen
Resolution: Unresolved Votes: 0
Labels: None





[TBENCH-13] Alioth pidgits Created: 21/Jan/12  Updated: 21/Jan/12  Resolved: 21/Jan/12

Status: Resolved
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Comments   
Comment by David Nolen [ 21/Jan/12 1:08 PM ]

TBENCH-4





[TBENCH-12] Alioth k-nucleotide Created: 21/Jan/12  Updated: 23/Feb/12

Status: Open
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Comments   
Comment by Mike Anderson [ 21/Feb/12 5:00 AM ]

Have added my latest version of the k-nucleotide benchmark in the following repository:

https://github.com/mikera/test.benchmark/

Comment by David Nolen [ 21/Feb/12 8:00 AM ]

Please create a patch of your changes and add it to the ticket, thanks!

Comment by Paul Michael Bauer [ 23/Feb/12 3:43 PM ]

If you add a patch, I can take a look at this tonight.

Comment by Paul Michael Bauer [ 23/Feb/12 10:45 PM ]

Mike,
I did pull down your repo and ran some tests.
This version of k-nucleotide does run faster than the currently submitted version in alioth.
However, we've got a ways to go to catch the faster Java implementations.

I've added a few comments in your repo.
Of note, the (set! unchecked-math true) directive at the top of files allows us to use inc, dec, etc directly without explicitly specifying unchecked-inc, etc.

http://shootout.alioth.debian.org/u64q/program.php?test=knucleotide&lang=java&id=9 is the fastest java version, but is a bit gnarly.
http://shootout.alioth.debian.org/u64q/program.php?test=knucleotide&lang=java&id=2 is a little slower, but in the same order of magnitude for performance, and significantly more straightforward.

You might check those out for some inspiration to boost the performance of clojure's knucleotide.
Good luck and thanks for the help!





[TBENCH-11] Alioth reverse-complement Created: 21/Jan/12  Updated: 13/Feb/12

Status: Open
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Paul Michael Bauer
Resolution: Unresolved Votes: 0
Labels: None





[TBENCH-10] Alioth spectral-norm Created: 21/Jan/12  Updated: 21/Jan/12  Resolved: 21/Jan/12

Status: Resolved
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by David Nolen [ 21/Jan/12 1:02 PM ]

Fixed, https://github.com/clojure/test.benchmark/commit/642279639c99ce222c10a4d1f1f69aa5565845c6





[TBENCH-9] Alioth regex-dna Created: 15/Nov/11  Updated: 13/Feb/12  Resolved: 13/Feb/12

Status: Resolved
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Paul Michael Bauer Assignee: Paul Michael Bauer
Resolution: Completed Votes: 0
Labels: None

Patch: Code and Test

 Description   

Implement regex-dna as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Paul Michael Bauer [ 13/Feb/12 11:11 PM ]

re-worked A. Fingerhut's implementation to use nio and a hand-rolled reduce loop for code replacement.
Result is a ~2x speedup and 92% as fast as the fastest Java version.

https://github.com/clojure/test.benchmark/blob/0977c41e430b85f452f0064aad24562e9be48fff/src/main/clojure/alioth/regexdna.clj





[TBENCH-8] Alioth mandelbrot Created: 13/Sep/11  Updated: 15/Nov/11  Resolved: 15/Nov/11

Status: Closed
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Paul Michael Bauer Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: File mandelbrotR1.diff    
Patch: Code and Test
Approval: Ok

 Description   

Implement mandelbrot as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Paul Michael Bauer [ 13/Sep/11 6:06 AM ]

This is a first pass at a mandelbrot that takes advantage of clojure 1.3 performance features.

Approach:
Avoid idiomatic functional code.
Extensive iteration, in-place mutation, primitive arrays, manage our own threads etc. Not "pretty."
(currently .8 to .9 % of fastest java version).

Tests:
Correctness tests, verified against a set of known-good resource files.

Caveats:
This is the first pass and my first time writing optimized clojure 1.3. I'm sure there are things that can be improved, so any feedback welcome.

Future:
I've included a modified version of the fastest java program in the patch as I've found it useful for comparison
(# time script/run_java alioth.java.mandelbrot 16000)
One thing we don't have is repeatable performance metrics/tests.
One approach would be to have tests that fail whenever the clojure version falls below a certain percentage of the java version.
That's something I'm working on.

Comment by Paul Michael Bauer [ 13/Sep/11 9:00 PM ]

Per discussion on mailing list (https://groups.google.com/d/topic/clojure-dev/Kr5teQh69Pw/discussion) removed java files.
Files pushed.

Comment by Stuart Halloway [ 15/Nov/11 7:29 PM ]

Paul, it is fine for you to close tickets yourself on contrib projects (all the notes about "core team only" apply to Clojure itself, not the contribs).





[TBENCH-7] Auto-generate maven-classpath on build, script/* refactoring Created: 10/Sep/11  Updated: 15/Nov/11  Resolved: 15/Nov/11

Status: Closed
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Paul Michael Bauer Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File 0001-Refactored-script-run-repl-auto-generate-script-mave.patch    
Patch: Code
Waiting On: Stuart Halloway

 Description   
  • Update pom to auto-generate maven-classpath file on build
  • script refactoring, can run from any folder, misc improvements
  • Update pom to depend on 1.3 beta 3


 Comments   
Comment by Paul Michael Bauer [ 13/Sep/11 9:01 PM ]

This can be closed. Patch pushed.





[TBENCH-6] thread-ring Created: 29/Oct/10  Updated: 15/Nov/11  Resolved: 15/Nov/11

Status: Closed
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Paul Michael Bauer
Resolution: Completed Votes: 0
Labels: None

Patch: Code and Test

 Description   

Implement thread-ring as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Stuart Halloway [ 03/Dec/10 11:46 AM ]

these were assigned to me by accident

Comment by Paul Michael Bauer [ 07/Oct/11 1:45 PM ]

Could you assign this to me ... working on it.

Comment by Paul Michael Bauer [ 15/Nov/11 7:36 PM ]

Fixed:
https://github.com/clojure/test.benchmark/commit/f4c0802af58f5f3341816be40bf1388029bc0d3e
https://github.com/clojure/test.benchmark/commit/f4c0802af58f5f3341816be40bf1388029bc0d3e

Comment by Paul Michael Bauer [ 15/Nov/11 7:36 PM ]

https://github.com/clojure/test.benchmark/commit/f4c0802af58f5f3341816be40bf1388029bc0d3e





[TBENCH-5] spectral-norm Created: 29/Oct/10  Updated: 21/Jan/12  Resolved: 21/Jan/12

Status: Resolved
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

Implement spectral-norm as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Stuart Halloway [ 03/Dec/10 11:46 AM ]

these were assigned to me by accident

Comment by David Nolen [ 21/Jan/12 1:07 PM ]

Done, https://github.com/clojure/test.benchmark/commit/642279639c99ce222c10a4d1f1f69aa5565845c6





[TBENCH-4] Alioth pidigits Created: 29/Oct/10  Updated: 03/Dec/10

Status: Open
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Implement pidigits as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Stuart Halloway [ 03/Dec/10 11:46 AM ]

these were assigned to me by accident





[TBENCH-3] Alioth meteor-contest Created: 29/Oct/10  Updated: 03/Dec/10

Status: Open
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Implement meteor-contest as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Stuart Halloway [ 03/Dec/10 11:46 AM ]

these were assigned to me by accident





[TBENCH-2] Alioth chameneos-redux Created: 29/Oct/10  Updated: 03/Dec/10

Status: Open
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Implement chameneos-redux as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Stuart Halloway [ 03/Dec/10 11:46 AM ]

these were assigned to me by accident





[TBENCH-1] Alioth binary-trees Created: 29/Oct/10  Updated: 21/Jan/12  Resolved: 21/Jan/12

Status: Resolved
Project: test.benchmark
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

Implement binary-trees as described at http://shootout.alioth.debian.org/. See http://dev.clojure.org/display/testbenchmark/Alioth+Benchmarks for an overview.



 Comments   
Comment by Stuart Halloway [ 03/Dec/10 11:46 AM ]

these were assigned to me by accident

Comment by David Nolen [ 21/Jan/12 11:22 AM ]

done as far I can tell.





Generated at Mon Nov 24 14:36:32 CST 2014 using JIRA 4.4#649-r158309.