Clojure

Improve reporting in transducers generative test.

Details

  • Type: Enhancement Enhancement
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Completed
  • Affects Version/s: Release 1.7
  • Fix Version/s: Release 1.7
  • Component/s: None
  • Labels:
  • Patch:
    Code and Test
  • Approval:
    Ok

Description

If the transducers generative test breaks, you get output like this:

[java] {:test-var seq-and-transducer, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [-16 10 -8 8 -5], :actions map clojure.core$dec@782a4056,take 5,partition-by clojure.core$even_QMARK_@2200d281,partition-all 9,map clojure.core$inc@643b798d,drop 9,remove clojure.core$empty_QMARK_@4600f352,remove clojure.core$odd_QMARK_@32dd05af, :s #<ClassCastException java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.Number>, :xs (), :xi [], :xt []}>, :seed 1419199634890, :failing-size 21, :num-tests 22, :fail [[-16 10 -8 8 -5] [{:desc map clojure.core$dec@782a4056, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@28449652>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@506b8505>} {:desc take 5, :xf #<core$take$fn__3712 clojure.core$take$fn__3712@38934406>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@27ce0b6d>} {:desc partition-by clojure.core$even_QMARK_@2200d281, :xf #<core$partition_by$fn__5568 clojure.core$partition_by$fn__5568@5287c159>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@70961c7b>} {:desc partition-all 9, :xf #<core$partition_all$fn__5590 clojure.core$partition_all$fn__5590@3f869b0>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@6f99ed9f>} {:desc map clojure.core$inc@643b798d, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@2f2c41d3>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@2fdbef8d>} {:desc drop 9, :xf #<core$drop$fn__3728 clojure.core$drop$fn__3728@4f7b4b50>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@214b9b5>} {:desc remove clojure.core$empty_QMARK_@4600f352, :xf #<core$filter$fn__3696 clojure.core$filter$fn__3696@6846d654>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@7df231c7>} {:desc remove clojure.core$odd_QMARK_@32dd05af, :xf #<core$filter$fn__3696 clojure.core$filter$fn__3696@5a8ce6dd>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@34ee9000>}]], :shrunk {:total-nodes-visited 32, :depth 12, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [0], :actions map clojure.core$inc@643b798d, :s (1), :xs (0), :xi [0], :xt [0]}>, :smallest [[0] [{:desc map clojure.core$inc@643b798d, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@2f2c41d3>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@2fdbef8d>}]]}}
     [java]
     [java] ERROR in (seq-and-transducer) (core.clj:4566)
     [java] Uncaught exception, not in assertion.
     [java] expected: nil
     [java]   actual: clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results.
     [java]  at clojure.core$ex_info.invoke (core.clj:4566)
     [java]     clojure.test_clojure.transducers$seq_and_transducer_same_result.invoke (transducers.clj:103)
     [java]     clojure.lang.AFn.applyToHelper (AFn.java:156)
...etc etc

This has a few problems:

  • when clojure functions are given as arguments, they're full object printouts, with classnames and memory addresses, this is kind of hard to read
  • the combination of the first problem found with the shrunk version means there's a lot of content to read
  • lack of pretty printing makes that content very hard to read
  • the traceback isn't actually that helpful – we know what failed already.

Approach: The attached patch encodes more descriptive info in the actions and does a better job of reporting the difference in an understandable manner:

[java] FAIL in (seq-and-transducer) (transducers.clj:135)
     [java] {:coll [0],
     [java]  :actions (->> coll (map inc)),
     [java]  :s (1),
     [java]  :xs (0),
     [java]  :xi [0],
     [java]  :xt [0]}

Patch: transducer-reporting-v1.patch

Screened by: Alex Miller

Activity

Michael Blume made changes -
Field Original Value New Value
Attachment transducer-reporting-v1.patch [ 13655 ]
Patch Code and Test [ 10002 ]
Alex Miller made changes -
Labels transducers
Fix Version/s Release 1.7 [ 10250 ]
Affects Version/s Release 1.7 [ 10250 ]
Approval Vetted [ 10003 ]
Alex Miller made changes -
Description If I break the map transducer and then run the transducers.clj generative test, the output I get looks like so

{code}
 
     [java] {:test-var seq-and-transducer, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [-16 10 -8 8 -5], :actions map clojure.core$dec@782a4056,take 5,partition-by clojure.core$even_QMARK_@2200d281,partition-all 9,map clojure.core$inc@643b798d,drop 9,remove clojure.core$empty_QMARK_@4600f352,remove clojure.core$odd_QMARK_@32dd05af, :s #<ClassCastException java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.Number>, :xs (), :xi [], :xt []}>, :seed 1419199634890, :failing-size 21, :num-tests 22, :fail [[-16 10 -8 8 -5] [{:desc map clojure.core$dec@782a4056, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@28449652>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@506b8505>} {:desc take 5, :xf #<core$take$fn__3712 clojure.core$take$fn__3712@38934406>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@27ce0b6d>} {:desc partition-by clojure.core$even_QMARK_@2200d281, :xf #<core$partition_by$fn__5568 clojure.core$partition_by$fn__5568@5287c159>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@70961c7b>} {:desc partition-all 9, :xf #<core$partition_all$fn__5590 clojure.core$partition_all$fn__5590@3f869b0>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@6f99ed9f>} {:desc map clojure.core$inc@643b798d, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@2f2c41d3>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@2fdbef8d>} {:desc drop 9, :xf #<core$drop$fn__3728 clojure.core$drop$fn__3728@4f7b4b50>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@214b9b5>} {:desc remove clojure.core$empty_QMARK_@4600f352, :xf #<core$filter$fn__3696 clojure.core$filter$fn__3696@6846d654>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@7df231c7>} {:desc remove clojure.core$odd_QMARK_@32dd05af, :xf #<core$filter$fn__3696 clojure.core$filter$fn__3696@5a8ce6dd>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@34ee9000>}]], :shrunk {:total-nodes-visited 32, :depth 12, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [0], :actions map clojure.core$inc@643b798d, :s (1), :xs (0), :xi [0], :xt [0]}>, :smallest [[0] [{:desc map clojure.core$inc@643b798d, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@2f2c41d3>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@2fdbef8d>}]]}}
     [java]
     [java] ERROR in (seq-and-transducer) (core.clj:4566)
     [java] Uncaught exception, not in assertion.
     [java] expected: nil
     [java] actual: clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results.
     [java] at clojure.core$ex_info.invoke (core.clj:4566)
     [java] clojure.test_clojure.transducers$seq_and_transducer_same_result.invoke (transducers.clj:103)
     [java] clojure.lang.AFn.applyToHelper (AFn.java:156)
     [java] clojure.lang.AFn.applyTo (AFn.java:144)
     [java] clojure.core$apply.invoke (core.clj:626)
     [java] clojure.test.check.properties$apply_gen$fn__25343$fn__25344.invoke (properties.clj:16)
     [java] clojure.test.check.properties$apply_gen$fn__25343.invoke (properties.clj:16)
     [java] clojure.test.check.rose_tree$fmap.invoke (rose_tree.clj:46)
     [java] clojure.core$partial$fn__3652.invoke (core.clj:2489)
     [java] clojure.test.check.generators$gen_fmap$fn__25169.invoke (generators.clj:70)
     [java] clojure.test.check.generators$call_gen.invoke (generators.clj:56)
     [java] clojure.test.check$quick_check.doInvoke (check.clj:57)
     [java] clojure.lang.RestFn.invoke (RestFn.java:425)
     [java] clojure.lang.AFn.applyToHelper (AFn.java:156)
     [java] clojure.lang.RestFn.applyTo (RestFn.java:132)
     [java] clojure.core$apply.invoke (core.clj:630)
     [java] clojure.test_clojure.transducers$seq_and_transducer.doInvoke (transducers.clj:109)
     [java] clojure.lang.RestFn.invoke (RestFn.java:410)
     [java] clojure.test_clojure.transducers$seq_and_transducer.invoke (transducers.clj:109)
     [java] clojure.test_clojure.transducers/fn (transducers.clj:109)
     [java] clojure.test$test_var$fn__7689.invoke (test.clj:704)
     [java] clojure.test$test_var.invoke (test.clj:704)
     [java] clojure.test$test_vars$fn__7711$fn__7716.invoke (test.clj:722)
     [java] clojure.test$default_fixture.invoke (test.clj:674)
     [java] clojure.test$test_vars$fn__7711.invoke (test.clj:722)
     [java] clojure.test$default_fixture.invoke (test.clj:674)
     [java] clojure.test$test_vars.invoke (test.clj:718)
     [java] clojure.test$test_all_vars.invoke (test.clj:728)
     [java] clojure.test$test_ns.invoke (test.clj:747)
     [java] clojure.core$map$fn__3673.invoke (core.clj:2614)
     [java] clojure.lang.LazySeq.sval (LazySeq.java:40)
     [java] clojure.lang.LazySeq.seq (LazySeq.java:49)
     [java] clojure.lang.Cons.next (Cons.java:39)
     [java] clojure.lang.RT.next (RT.java:644)
     [java] clojure.core/next (core.clj:64)
     [java] clojure.core$reduce1.invoke (core.clj:905)
     [java] clojure.core$reduce1.invoke (core.clj:896)
     [java] clojure.core$merge_with.doInvoke (core.clj:2931)
     [java] clojure.lang.RestFn.applyTo (RestFn.java:139)
     [java] clojure.core$apply.invoke (core.clj:628)
     [java] clojure.test$run_tests.doInvoke (test.clj:762)
     [java] clojure.lang.RestFn.applyTo (RestFn.java:137)
     [java] clojure.core$apply.invoke (core.clj:626)
     [java] user$eval27152.invoke (run_test.clj:7)
     [java] clojure.lang.Compiler.eval (Compiler.java:6768)
     [java] clojure.lang.Compiler.load (Compiler.java:7195)
     [java] clojure.lang.Compiler.loadFile (Compiler.java:7151)
     [java] clojure.main$load_script.invoke (main.clj:274)
     [java] clojure.main$script_opt.invoke (main.clj:336)
     [java] clojure.main$main.doInvoke (main.clj:420)
     [java] clojure.lang.RestFn.invoke (RestFn.java:408)
     [java] clojure.lang.Var.invoke (Var.java:379)
     [java] clojure.lang.AFn.applyToHelper (AFn.java:154)
     [java] clojure.lang.Var.applyTo (Var.java:700)
     [java] clojure.main.main (main.java:37)
{code}

This has a few problems

* when clojure functions are given as arguments, they're full object printouts, with classnames and memory addresses, this is kind of hard to read
* the combination of the first problem found with the shrunk version means there's a lot of content to read
* lack of pretty printing makes that content very hard to read
* the traceback isn't actually that helpful -- we know what failed already.

Instead it might be nice if the test printed something like

{code}
 
     [java] FAIL in (seq-and-transducer) (transducers.clj:135)
     [java] {:coll [0],
     [java] :actions (->> coll (map inc)),
     [java] :s (1),
     [java] :xs (0),
     [java] :xi [0],
     [java] :xt [0]}
{code}
If the transducers generative test breaks, you get output like this:

{code}
     [java] {:test-var seq-and-transducer, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [-16 10 -8 8 -5], :actions map clojure.core$dec@782a4056,take 5,partition-by clojure.core$even_QMARK_@2200d281,partition-all 9,map clojure.core$inc@643b798d,drop 9,remove clojure.core$empty_QMARK_@4600f352,remove clojure.core$odd_QMARK_@32dd05af, :s #<ClassCastException java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.Number>, :xs (), :xi [], :xt []}>, :seed 1419199634890, :failing-size 21, :num-tests 22, :fail [[-16 10 -8 8 -5] [{:desc map clojure.core$dec@782a4056, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@28449652>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@506b8505>} {:desc take 5, :xf #<core$take$fn__3712 clojure.core$take$fn__3712@38934406>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@27ce0b6d>} {:desc partition-by clojure.core$even_QMARK_@2200d281, :xf #<core$partition_by$fn__5568 clojure.core$partition_by$fn__5568@5287c159>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@70961c7b>} {:desc partition-all 9, :xf #<core$partition_all$fn__5590 clojure.core$partition_all$fn__5590@3f869b0>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@6f99ed9f>} {:desc map clojure.core$inc@643b798d, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@2f2c41d3>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@2fdbef8d>} {:desc drop 9, :xf #<core$drop$fn__3728 clojure.core$drop$fn__3728@4f7b4b50>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@214b9b5>} {:desc remove clojure.core$empty_QMARK_@4600f352, :xf #<core$filter$fn__3696 clojure.core$filter$fn__3696@6846d654>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@7df231c7>} {:desc remove clojure.core$odd_QMARK_@32dd05af, :xf #<core$filter$fn__3696 clojure.core$filter$fn__3696@5a8ce6dd>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@34ee9000>}]], :shrunk {:total-nodes-visited 32, :depth 12, :result #<ExceptionInfo clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results. {:coll [0], :actions map clojure.core$inc@643b798d, :s (1), :xs (0), :xi [0], :xt [0]}>, :smallest [[0] [{:desc map clojure.core$inc@643b798d, :xf #<core$map$fn__3669 clojure.core$map$fn__3669@2f2c41d3>, :seq #<core$partial$fn__3652 clojure.core$partial$fn__3652@2fdbef8d>}]]}}
     [java]
     [java] ERROR in (seq-and-transducer) (core.clj:4566)
     [java] Uncaught exception, not in assertion.
     [java] expected: nil
     [java] actual: clojure.lang.ExceptionInfo: Applied actions to coll as seq, sequence transducer, and into transducer and got different results.
     [java] at clojure.core$ex_info.invoke (core.clj:4566)
     [java] clojure.test_clojure.transducers$seq_and_transducer_same_result.invoke (transducers.clj:103)
     [java] clojure.lang.AFn.applyToHelper (AFn.java:156)
...etc etc
{code}

This has a few problems:

* when clojure functions are given as arguments, they're full object printouts, with classnames and memory addresses, this is kind of hard to read
* the combination of the first problem found with the shrunk version means there's a lot of content to read
* lack of pretty printing makes that content very hard to read
* the traceback isn't actually that helpful -- we know what failed already.

*Approach:* The attached patch encodes more descriptive info in the actions and does a better job of reporting the difference in an understandable manner:

{code}
     [java] FAIL in (seq-and-transducer) (transducers.clj:135)
     [java] {:coll [0],
     [java] :actions (->> coll (map inc)),
     [java] :s (1),
     [java] :xs (0),
     [java] :xi [0],
     [java] :xt [0]}
{code}

*Patch:* transducer-reporting-v1.patch

*Screened by:* Alex Miller
Approval Vetted [ 10003 ] Screened [ 10004 ]
Rich Hickey made changes -
Approval Screened [ 10004 ] Ok [ 10007 ]
Stuart Halloway made changes -
Status Open [ 1 ] Closed [ 6 ]
Resolution Completed [ 1 ]

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: