Clojure

Need to expand tests to cover transducers

Details

  • Type: Enhancement Enhancement
  • Status: Closed Closed
  • Priority: Critical Critical
  • Resolution: Completed
  • Affects Version/s: Release 1.7
  • Fix Version/s: Release 1.7
  • Component/s: None
  • Labels:
  • Patch:
    Code and Test
  • Approval:
    Ok

Description

Attached patch contains both some generative and example tests for transducers. The generative tests build a series of sequence functions (take 5, filter odd?, etc) and apply them to a random vector of numbers as seq transformations, sequence of transducer, into of transducer, and transduce of transducer. The results are compared.

Note: these tests depend on the patch in CLJ-1349 to run as tests.

Patch: clj-1554-5.patch

  1. clj-1554.patch
    21/Oct/14 11:53 AM
    9 kB
    Alex Miller
  2. clj-1554-2.patch
    22/Oct/14 12:17 PM
    10 kB
    Alex Miller
  3. clj-1554-3.patch
    24/Oct/14 2:52 PM
    10 kB
    Alex Miller
  4. clj-1554-4.patch
    03/Nov/14 10:46 AM
    10 kB
    Alex Miller
  5. clj-1554-5.patch
    12/Nov/14 12:08 PM
    10 kB
    Alex Miller

Activity

Hide
Fogus added a comment - - edited

I downloaded and applied this patch and its dependent patch (1349) and ran the tests. The coverage is a good start and the approach of verifying results against results gathered from other approaches is important. One note of style is that the use of `doall` is inconsistent in the `apply-as-*` functions. i would recommend that at least one other person screen this patch as my grasp of test.check is tenuous.

Show
Fogus added a comment - - edited I downloaded and applied this patch and its dependent patch (1349) and ran the tests. The coverage is a good start and the approach of verifying results against results gathered from other approaches is important. One note of style is that the use of `doall` is inconsistent in the `apply-as-*` functions. i would recommend that at least one other person screen this patch as my grasp of test.check is tenuous.
Hide
Alex Miller added a comment -

Updated patch slightly to clean up the doall stuff.

Show
Alex Miller added a comment - Updated patch slightly to clean up the doall stuff.
Hide
Guangyu Zhang added a comment -

What is clojure.test.check? You require it but never use it. This namespace doesn't exist, so I can't do individual test by (require 'clojure.test-clojure.transducers).

The error message:
CompilerException java.io.FileNotFoundException: Could not locate clojure/test/check__init.class or clojure/test/check.clj on classpath., compiling:(clojure/test_clojure/transducers.clj:1:1)

The way I used to do individual test is described in http://dev.clojure.org/display/community/Developing+Patches.

But there is no error when I run 'mvn package'.

Show
Guangyu Zhang added a comment - What is clojure.test.check? You require it but never use it. This namespace doesn't exist, so I can't do individual test by (require 'clojure.test-clojure.transducers). The error message: CompilerException java.io.FileNotFoundException: Could not locate clojure/test/check__init.class or clojure/test/check.clj on classpath., compiling:(clojure/test_clojure/transducers.clj:1:1) The way I used to do individual test is described in http://dev.clojure.org/display/community/Developing+Patches. But there is no error when I run 'mvn package'.
Hide
Alex Miller added a comment -

As noted in the description, this patch depends on CLJ-1349 to be applied first.

Show
Alex Miller added a comment - As noted in the description, this patch depends on CLJ-1349 to be applied first.
Hide
Alex Miller added a comment -

After you apply CLJ-1349 you will also need to rerun antsetup.sh as it adds new dependencies.

Show
Alex Miller added a comment - After you apply CLJ-1349 you will also need to rerun antsetup.sh as it adds new dependencies.
Hide
Guangyu Zhang added a comment -

I did what you say, but the error still exists.
I can pass this test via 'ant test-example', but I can not do individual test.

To reproduce this problem:
Apply CLJ-1349 and CLJ-1554
$ ./antsetup.sh
$ ant
$ java -cp test:clojure-1.7.0-master-SNAPSHOT.jar clojure.main
user=> (require 'clojure.test-clojure.transducers)
CompilerException java.io.FileNotFoundException: Could not locate clojure/test/check__init.class or clojure/test/check.clj on classpath., compiling:(clojure/test_clojure/transducers.clj:1:1)

This should work:
$ java -cp /Users/guangyu/.m2/repository/org/clojure/test.check/0.5.9/test.check-0.5.9.jar:/Users/guangyu/.m2/repository/org/clojure/test.generative/0.5.1/test.generative-0.5.1.jar:test:clojure.jar clojure.main
user=> (require 'clojure.test-clojure.transducers)
nil

Maybe the document (http://dev.clojure.org/display/community/Developing+Patches) needs to be updated.

Show
Guangyu Zhang added a comment - I did what you say, but the error still exists. I can pass this test via 'ant test-example', but I can not do individual test. To reproduce this problem: Apply CLJ-1349 and CLJ-1554 $ ./antsetup.sh $ ant $ java -cp test:clojure-1.7.0-master-SNAPSHOT.jar clojure.main user=> (require 'clojure.test-clojure.transducers) CompilerException java.io.FileNotFoundException: Could not locate clojure/test/check__init.class or clojure/test/check.clj on classpath., compiling:(clojure/test_clojure/transducers.clj:1:1) This should work: $ java -cp /Users/guangyu/.m2/repository/org/clojure/test.check/0.5.9/test.check-0.5.9.jar:/Users/guangyu/.m2/repository/org/clojure/test.generative/0.5.1/test.generative-0.5.1.jar:test:clojure.jar clojure.main user=> (require 'clojure.test-clojure.transducers) nil Maybe the document (http://dev.clojure.org/display/community/Developing+Patches) needs to be updated.
Hide
Guangyu Zhang added a comment -

There is no need to require clojure.test.check . I remove it and nothing happens.

Show
Guangyu Zhang added a comment - There is no need to require clojure.test.check . I remove it and nothing happens.
Hide
Alex Miller added a comment -

That page is out of date with respect to running tests with either test.generative or test.check (which doesn't actually exist yet until CLJ-1349).

More complete recipe:

1. Apply CLJ-1349 and CLJ-1554 patches
2. ./antsetup.sh
3. ant
4. java -cp `cat maven-classpath`:target/classes:src:test clojure.main
5. (require 'clojure.test-clojure.transducers)
6. (clojure.test/run-tests 'clojure.test-clojure.transducers)

Works for me.

Confusingly, the patch in this test uses test.check, which is a generative test but run in the build (post CLJ-1349) as an example-based test. Stu and I are still talking about the best way to address that. One issue is that test.generative tests are time-based for intensity while test.check is iteration-based.

I will update the patch to remove the require of test.check.

Show
Alex Miller added a comment - That page is out of date with respect to running tests with either test.generative or test.check (which doesn't actually exist yet until CLJ-1349). More complete recipe: 1. Apply CLJ-1349 and CLJ-1554 patches 2. ./antsetup.sh 3. ant 4. java -cp `cat maven-classpath`:target/classes:src:test clojure.main 5. (require 'clojure.test-clojure.transducers) 6. (clojure.test/run-tests 'clojure.test-clojure.transducers) Works for me. Confusingly, the patch in this test uses test.check, which is a generative test but run in the build (post CLJ-1349) as an example-based test. Stu and I are still talking about the best way to address that. One issue is that test.generative tests are time-based for intensity while test.check is iteration-based. I will update the patch to remove the require of test.check.
Hide
Alex Miller added a comment -

I updated that testing page to cover test.generative as well.

Show
Alex Miller added a comment - I updated that testing page to cover test.generative as well.
Hide
Stuart Halloway added a comment -

Alex, would like to discuss two possible changes

  • make fbind create a symbolic rep of the work to do, so that failure messages are easier to read
  • whitelist the exceptions we expect, and check with a predicate in seq-and-transducer-same-result
Show
Stuart Halloway added a comment - Alex, would like to discuss two possible changes
  • make fbind create a symbolic rep of the work to do, so that failure messages are easier to read
  • whitelist the exceptions we expect, and check with a predicate in seq-and-transducer-same-result
Hide
Alex Miller added a comment -

Added new patch that whitelists only IllegalArgumentException and ClassCastException as the possible allowed exceptions in the transducer tests (they may vary between the transducer and non-transducer form).

The fbind does build a semantic description already in the :desc key which is used on error. Here's an example error - see the :actions key. That will be a list of the transformations applied (although shrinking often minimizes that list):

[java] Testing clojure.test-clojure.transducers
     [java] {:test-var seq-and-transducer, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [3 5 5 5 -2], :actions take 6, :s (3 5 5 5 -2), :xs (3 5 5), :xi [3 5 5], :xt [3 5 5]}>, :seed 1415806766835, :failing-size 6, :num-tests 7, :fail [[3 5 5 5 -2] [{:desc take 6, :xf #<core$take$fn__4550 clojure.core$take$fn__4550@4d186c57>, :seq #<core$partial$fn__4490 clojure.core$partial$fn__4490@44709ca4>}]], :shrunk {:total-nodes-visited 46, :depth 10, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [0 0], :actions take 2, :s (0 0), :xs (0), :xi [0], :xt [0]}>, :smallest [[0 0] [{:desc take 2, :xf #<core$take$fn__4550 clojure.core$take$fn__4550@5b938615>, :seq #<core$partial$fn__4490 clojure.core$partial$fn__4490@556733e4>}]]}}
Show
Alex Miller added a comment - Added new patch that whitelists only IllegalArgumentException and ClassCastException as the possible allowed exceptions in the transducer tests (they may vary between the transducer and non-transducer form). The fbind does build a semantic description already in the :desc key which is used on error. Here's an example error - see the :actions key. That will be a list of the transformations applied (although shrinking often minimizes that list):
[java] Testing clojure.test-clojure.transducers
     [java] {:test-var seq-and-transducer, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [3 5 5 5 -2], :actions take 6, :s (3 5 5 5 -2), :xs (3 5 5), :xi [3 5 5], :xt [3 5 5]}>, :seed 1415806766835, :failing-size 6, :num-tests 7, :fail [[3 5 5 5 -2] [{:desc take 6, :xf #<core$take$fn__4550 clojure.core$take$fn__4550@4d186c57>, :seq #<core$partial$fn__4490 clojure.core$partial$fn__4490@44709ca4>}]], :shrunk {:total-nodes-visited 46, :depth 10, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [0 0], :actions take 2, :s (0 0), :xs (0), :xi [0], :xt [0]}>, :smallest [[0 0] [{:desc take 2, :xf #<core$take$fn__4550 clojure.core$take$fn__4550@5b938615>, :seq #<core$partial$fn__4490 clojure.core$partial$fn__4490@556733e4>}]]}}

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: