<< Back to previous view

[MATCH-57] Non deterministic match behavior for seqs when AOT Created: 05/Apr/12  Updated: 28/Jul/13  Resolved: 15/Jun/13

Status: Closed
Project: core.match
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ronen Narkis Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: None
Environment:

Clojure 1.3

Leiningen 1.7.1 on Java 1.6.0_26

core.match "0.2.0-alpha9"



 Description   

The following source example:

https://github.com/narkisr/match-issue/blob/master/src/match_issue/core.clj

The defrule macro calls a function called lhs that use the match library,

this function calls it self recursively and print out the rest of body left to be matched,

When running lein compile two different results appear (in non deterministic fashion):

(when $message :> Message (== level 6) :from (entry-point event-stream))
($message :> Message (== level 6) :from (entry-point event-stream))
(Message (== level 6) :from (entry-point event-stream))
((== level 6) :from (entry-point event-stream))

This is the bug, as (== level 6) should be matched by:

[([([(o :when operator?) f s] :seq) :as c & r] :seq)]

In other cases the output is (the correct one):

(when $message :> Message (== level 6) :from (entry-point event-stream))
($message :> Message (== level 6) :from (entry-point event-stream))
(Message (== level 6) :from (entry-point event-stream))
((== level 6) :from (entry-point event-stream))
(:from (entry-point event-stream))
(entry-point event-stream)
()
()



 Comments   
Comment by David Nolen [ 06/Apr/12 11:03 AM ]

All AOT related issues w/o patches are low priority for the foreseeable future. They are simply too time consuming to track down and I don't have the bandwidth at the moment. I will get to them eventually, but if you want movement on this please submit a patch. I will happily apply it!

Comment by Ronen Narkis [ 07/Apr/12 7:53 PM ]

Hey David,

After going through the source code and some more in depth look I managed to understand what was going on, first iv stepped through the different stages

(def m (emit-matrix ['body]
'( [(['when & r] :seq)] (lhs r)
[(['accumulate & r] :seq)] (<< "accumulate(~(lhs r))")
[([bind ':> & r] :seq)] (<< "~{bind}:~(lhs r)"); pattern with bind
[([(t :when is-type?) & r] :seq)] (<< "{t}(lhs r)"); pattern
[([([(f :when acc-fn?) & args] :seq) & r] :seq)] (<< ",{f}{args}~(lhs r)" ); accumulate function
[([':from dest & r] :seq)] (<< "from ~(lhs dest) ~(lhs r)")
[([':over win & r] :seq)] (<< "over ~(window win) ~(lhs r)")
[([([(o :when operator?) f s] :seq) :as c & r] :seq)] (<< "(~(reduce str (map pr-str (to-infix c)))) ~(lhs r)")
[(['entry-point point & r] :seq)] (<< "entry-point \"~{point}\"~(lhs r)")
:else ""
)))

(def c (cm/compile m))

(pprint c)

(pprint (executable-form c))

The last pprint failed aot compilation:

$ lein compile

...
Caused by: java.lang.ClassCastException: clojure.core.match.WildcardPattern cannot be cast to java.lang.String

Which was very weird as I could clearly see that:

(deftype WildcardPattern [sym _meta]
IWildcardPattern
(sym [_] sym)
clojure.lang.IObj
(meta [_] _meta)
(withMeta [_ new-meta]
(WildcardPattern. sym new-meta))
Object
(toString [_]
(str sym)))

Then iv also tried to enabled trace:

(set-trace! true)

Which resulted with:

$ lein compile

...

TRACE: DAG: Column 0 : [#<SeqPattern #<LiteralPattern window> #<LiteralPattern :time> #<WildcardPattern t> #<WildcardPattern unit>>]
TRA at clojure.lang.Compiler.analyzeSeq(Compiler.java:6416)
at clojure.lang.Compiler.analyze(Compiler.java:6216)
at clojure.lang.Compiler.analyze(Compiler.java:6177)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5572)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5008)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3629)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6407)
at clojure.lang.Compiler.analyze(Compiler.java:6216)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6397)
at clojure.lang.Compiler.analyze(Compiler.java:6216)
at clojure.lang.Compiler.access$100(Compiler.java:37)
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:492)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6409)
at clojure.lang.Compiler.analyze(Compiler.java:6216)
at clojure.lang.Compiler.analyze(Compiler.java:6177)
at clojure.lang.Compiler.compile1(Compiler.java:6980)
at clojure.lang.Compiler.compile(Compiler.java:7046)
at clojure.lang.RT.compile(RTCE: DAG: Add switch-node on occurrence exp

The most weird thing was that on second compile it all went well, this raised a flag and my suspicion was that the core.match classes got compiled on the first run:

$ ls classes/

match$analyze_actions$analyze_action__1166.class
match$analyze_actions.class
match$backtrack_expr.class
match$bind_node.class
match$catch_error.class
match$check_matrix_args$check_pattern__1150.class
...

Then iv decided to AOT compile clojure.core match:

$ git checkout core.match-0.2.0-alpha9
$ git diff project.clj
-(defproject match "0.2.0-alpha10-SNAPSHOT"
+(defproject match "0.2.0-alpha10-aot"
:description "Optimized pattern matching and predicate dispatch for Clojure"
:source-path "src/main/clojure"

  • :test-path "src/test/clojure")
    + :test-path "src/test/clojure"
    + :aot [clojure.core.match]
    + )

After using the aot jar it all went smooth including passing my non deterministic case (which is deterministic but very confusing to track down),

In order to reproduce it its important to run:

$ lein clean
$ rm -rf lib

And only then run

$ lein compile

Otherwise the classes dir might contain AOT'ed classes and make it seem that even non-aot jar works fine (its enough to run lein compile once to cause the classes to get compiled),

The fix in my case is simple, just AOT ns.

Im not sure if this Is a bug in the way Clojure AOT its classes but I think that adding the AOT'ed classes to the default core match distro is a reasonable workaround

BTW when using master (and not the tag) iv stumbled upon:

Exception in thread "main" java.lang.AssertionError: Assert failed: Unknown predicate in [is-type?]

Which is a pred defined in my ns.

Comment by David Nolen [ 15/Jun/13 10:30 PM ]

This sounds very similar to the other AOT bug reported by Kevin Lynagh MATCH-53 which is fixed in master, unless I hear different this is resolved.

Generated at Sun Sep 21 09:17:10 CDT 2014 using JIRA 4.4#649-r158309.