<< Back to previous view

[TANAL-101] t.a.j fails to detect wrong tag on defn with destructured map arg Created: 31/Oct/14  Updated: 31/Oct/14  Resolved: 31/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Not Reproducible Votes: 0
Labels: None


 Description   

Latest t.a.j calls the wrong-tag handler (if supplied) for these functions:

(defn avlf4 ^LinkedList [coll] (java.util.LinkedList. coll))
(defn avlf4b ^LinkedList [& coll] (java.util.LinkedList. coll))

but not for this one:

(defn avlf4c ^LinkedList [& {:keys [coll]}] (java.util.LinkedList. coll))


 Comments   
Comment by Andy Fingerhut [ 31/Oct/14 2:16 PM ]

Now that I think about it, I haven't checked whether the last case above would trigger CLJ-1232 behavior or not.

Comment by Andy Fingerhut [ 31/Oct/14 2:20 PM ]

I have now, and it does:

user=> (import '(java.util List))
java.util.List
user=> (defn ll1 ^List [] (java.util.LinkedList.))
#'user/ll1
user=> (.size (ll1))
0
user=> (defn ll2 ^List [& {:keys [coll]}] (java.util.LinkedList. coll))
#'user/ll2
user=> (ll2 :coll [4 5 -1])
(4 5 -1)
user=> (.size (ll2 :coll [4 5 -1]))
3
user=> (in-ns 'user2)
#<Namespace user2>
user2=> (clojure.core/refer 'user)
nil
user2=> (.size (ll1))

CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: List, compiling:(/private/var/folders/v5/7hpqbpk51td3v351377gl6yw0000gn/T/form-init6319381494916415605.clj:1:1) 
user2=> (.size (ll2 :coll [4 5 -1]))

CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: List, compiling:(/private/var/folders/v5/7hpqbpk51td3v351377gl6yw0000gn/T/form-init6319381494916415605.clj:1:1)
Comment by Nicola Mometto [ 31/Oct/14 2:25 PM ]

I've tried testing the last function with a dummy wrong-tag-handler that just prints a message when invoked and it looks like it is being invoked:

clojure.tools.analyzer.jvm> (do (analyze '(defn avlf4 ^LinkedList [& {:keys [coll]}] (java.util.LinkedList. coll))                                                                                                                
                                         (empty-env) {:passes-opts {:validate/wrong-tag-handler (ƒ [_ ast] (println "invoked"))}})                                                                                                
                                ∅)
invoked
invoked
invoked
invoked
∅
clojure.tools.analyzer.jvm>




[TANAL-100] Fix method matcher Created: 31/Oct/14  Updated: 31/Oct/14

Status: Open
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Blocker
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Unresolved Votes: 0
Labels: None


 Description   

places to fix:

jvm.utils/try-best-match
passes.jvm.annotate-methods
passes.jvm.validate/validate-call






[TANAL-99] Many new reflection warnings analyzing core.rrb-vector in latest t.a(.jvm) vs. 0.6.1 Created: 30/Oct/14  Updated: 30/Oct/14  Resolved: 30/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

Steps to reproduce:
pull latest Eastwood master

% cd crucible
% ./clone.sh  (skip if already pulled crucible repos)
% cd repos/core.rrb-vector-2014-02-14

% lein check
(no reflection warnings)
% lein eastwood '{:debug #{:compare-forms}}'
(many reflection warnings)

The forms emitted by t.a(.jvm) version 0.6.1 did not cause Clojure 1.6.0 to produce these reflection warnings.

The :compare-forms debug option to Eastwood will cause forms-read.txt and forms-emitted.txt files to be written, where the latter may be helpful in determining what it is about the emitted forms that are causing Clojure to reflection-warn about them. Sorry I haven't tracked this further yet.



 Comments   
Comment by Nicola Mometto [ 30/Oct/14 1:04 PM ]

it looks like tools.reader might have a bugged implementation of syntax-quote as it's losing the metadata.

Comment by Nicola Mometto [ 30/Oct/14 2:23 PM ]

I'm closing this since it's a tools.reader bug rather than a tools.analyzer one.
I still have not decided what's the best way to solve this and TBH I'm still investigating whether this might be a clojure bug or not.
In the meantime reverting https://github.com/clojure/tools.reader/commit/bb744f4e5513cea57d85638c83bc193a2390f9b9 should fix it

Comment by Andy Fingerhut [ 30/Oct/14 2:35 PM ]

Should I file a similar ticket for tools.reader ?

Comment by Nicola Mometto [ 30/Oct/14 2:38 PM ]

not necessary, I just pushed a fix

Comment by Andy Fingerhut [ 30/Oct/14 3:27 PM ]

In case there is a need to refer to this in the future, the fix went in with this commit, shortly after tools.reader 0.8.10 was released: https://github.com/clojure/tools.reader/commit/7812e704ceef683970ff2e28fd099bafefe4eba0





[TANAL-98] Exception thrown for an extend-protocol form Created: 30/Oct/14  Updated: 30/Oct/14  Resolved: 30/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Steps to reproduce:

pull latest master of Eastwood, which now includes latest master of t.a(.jvm) as of this ticket being created.

% cd crucible
% ./clone.sh  (skip if already pulled crucible repos)
% cd repos/cassaforte-2014-03-11

% lein eastwood '{:namespaces [clojurewerkz.cassaforte.conversion]}'
objc[5290]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
== Eastwood 0.1.5-SNAPSHOT Clojure 1.5.1 JVM 1.7.0_45
== Linting clojurewerkz.cassaforte.conversion ==
Reflection warning, clojurewerkz/cassaforte/bytes.clj:56:3 - call to deserialize can't be resolved.
Exception thrown during phase :analyze+eval of linting namespace clojurewerkz.cassaforte.conversion
IllegalArgumentException No implementation of method: :typename of protocol: #'clojure.reflect/TypeReference found for class: nil
	clojure.core/-cache-protocol-fn (core_deftype.clj:541)
	clojure.reflect/fn--9008/G--9003--9013 (reflect.clj:48)
	clojure.reflect.JavaReflector (java.clj:169)
	clojure.reflect/fn--8990/G--8986--8993 (reflect.clj:44)
	clojure.reflect/fn--8990/G--8985--8997 (reflect.clj:44)
	clojure.core/apply (core.clj:619)
	clojure.core/partial/fn--4190 (core.clj:2396)
	clojure.reflect/type-reflect (reflect.clj:100)
	clojure.core/apply (core.clj:623)
	eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm.utils/type-reflect (utils.clj:22)
	eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm.utils/members* (utils.clj:246)
	clojure.core/apply (core.clj:617)

[... most lines of stack trace omitted ...]

The following form was being processed during the exception:
(extend-protocol
 DefinitionToMap
 ResultSet
 (to-map
  [input]
  (into [] (for [row input] (into {} (for [cd #] (let # #))))))
 Host
 (to-map
  [host]
  {:datacenter (.getDatacenter host),
   :address (.getHostAddress (.getAddress host)),
   :rack (.getRack host)}))

[... more lines omitted ...]


 Comments   
Comment by Nicola Mometto [ 30/Oct/14 12:04 PM ]

Fixed:
https://github.com/clojure/tools.analyzer.jvm/commit/50c5d8dc83ae0254695ffd1dd86e84f64addc2b7
https://github.com/clojure/tools.analyzer.jvm/commit/4e40ae890633c0220b3401f91eb07606099955e0





[TANAL-97] analyze+eval throws exception with Clojure 1.7.0-alpha3 but not 1.6.0 for project utf8 Created: 28/Oct/14  Updated: 28/Oct/14  Resolved: 28/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

I haven't tracked down what is going on here. I am using tools.analyzer(.jvm) 0.6.1 with the latest Eastwood where I've noticed this (but also saw it with the 0.2.x version of tools.analyzer(.jvm) used by Eastwood 0.1.4). I haven't yet checked whether the latest released tools.analyzer(.jvm) improves on this behavior.

This might also be a bug introduced in Clojure 1.7.0-alpha3 vs. 1.6.0.

To see the issue, get latest Eastwood, pull all of the crucible projects, or at least the one that gets renamed utf8-2013-11-15, and run these two commands after doing 'mvm install' on the latest Clojure master as of 1.7.0-alpha3. The last command below will use 1.7.0-master-SNAPSHOT of Clojure, so it needs to be installed in your ~/.m2 named as that, or change the project.clj file to name 1.7.0-alpha3 instead.

% lein clean
% lein with-profile +1.6 eastwood
% lein clean
% leon with-profile +1.7 eastwood

With the last command I see an exception like this:

Exception thrown during phase :analyze+eval of linting namespace pjstadig.utf8
Got exception with extra ex-data:
    msg='Could not resolve var: Charset'
    (keys dat)=(:end-line :line :column :end-column :file :var)
ExceptionInfo Could not resolve var: Charset
	clojure.core/ex-info (core.clj:4566)
	eastwood.copieddeps.dep2.clojure.tools.analyzer.passes.jvm.validate/eval1973/fn--1975 (validate.clj:29)
	clojure.lang.MultiFn.invoke (MultiFn.java:229)


 Comments   
Comment by Nicola Mometto [ 28/Oct/14 12:03 PM ]

This appears to be caused because clojure 1.7.0-alpha3 fails to compile nio.core, thus the ns expression is not evaluated.

This is a regression introduced with clojure 1.7.0-alpha2, I'm investigating the cause and will open a CLJ ticket

Comment by Nicola Mometto [ 28/Oct/14 12:14 PM ]

turns out it's not even a clojure reggression, it's a bug in the last released version of nio that has been fixed in the SNAPSHOT version: https://github.com/pjstadig/nio/issues/4

clojure <=1.7.0-alpha2 silently ignored this bug but commit https://github.com/clojure/clojure/commit/85169b785c5dd59e89c0bd12600eebe5f6172874 had the side effect of exposing the bug

Comment by Andy Fingerhut [ 28/Oct/14 2:24 PM ]

Thanks for tracking that down! (inc Bronsa)

Comment by Andy Fingerhut [ 28/Oct/14 5:18 PM ]

And this helped me discover that although I tried to stop Eastwood's linking at the point that analyze+eval returns an AST indicating that eval threw an exception, I was only checking the top-level AST. I should have been checking all sub-ASTs of top-level do forms, and do forms nested inside those, etc. Soon Eastwood will stop earlier, closer to the real problem in this case.





[TANAL-85] enhance validations in t.a.js Created: 10/Aug/14  Updated: 17/Oct/14

Status: Open
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Critical
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This includes:

  • validating :new nodes
  • validating protocol/types usage
  • better error messages on non-valid 'ns forms





[TANAL-96] Change munging scheme to match the one introduced with CLJ-1330 Created: 13/Oct/14  Updated: 13/Oct/14  Resolved: 13/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None





[TANAL-95] java.lang.IllegalAccessError: butlast+last does not exist when (use 'clojure.tools.analyzer.jvm) with dependencies [org.clojure/tools.analyzer.jvm "0.6.1-SNAPSHOT"] and [org.clojure/core.typed "0.2.72"] Created: 11/Oct/14  Updated: 11/Oct/14  Resolved: 11/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Richard Davies Assignee: Nicola Mometto
Resolution: Not Reproducible Votes: 0
Labels: None
Environment:

project.clj:

(defproject bug-report "0.1.0-SNAPSHOT"
:repositories {"sonatype-oss-public" "https://oss.sonatype.org/content/groups/public/"}
:dependencies [[org.clojure/clojure "1.6.0"]
[org.clojure/core.typed "0.2.72"]
[org.clojure/tools.analyzer.jvm "0.6.1-SNAPSHOT"]])



 Description   

With the above project.clj, lauch a REPL

(require 'clojure.tools.analyzer.jvm)
;-> CompilerException java.lang.IllegalAccessError: butlast+last does not exist, compiling:(clojure/tools/analyzer/jvm.clj:1:1)

This exception does not occur if either the version of tools.analyzer.jvm is dropped back to 0.6.0 or if the dependency on core.typed is removed.



 Comments   
Comment by Nicola Mometto [ 11/Oct/14 5:08 AM ]

I could not reproduce with a clean repo and nothing between 0.6.1-SNAPSHOT and 0.6.0 hash changed wrt butlast+last.
Make sure you don't have any stale dependency by cleaning your target/ and eventually ~/.m2 directory





[TANAL-87] Move some passes to the emitter Created: 06/Sep/14  Updated: 10/Oct/14  Resolved: 10/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Don't belong in tools.analyzer.jvm:

  • annotate-class-id
  • annotate-internal-name

Require some though

  • classify-invoke
  • box/ensure-tag
  • collect
  • collect-closed-overs
  • clear-locals





[TANAL-94] emit-form does not qualify def symbols when using :qualified-symbols option Created: 09/Oct/14  Updated: 10/Oct/14  Resolved: 10/Oct/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Richard Davies Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None
Environment:

All



 Description   
(ns bug
  (:require [clojure.tools.analyzer.jvm :as j])
  (:require [clojure.tools.analyzer.passes.jvm.emit-form :as e]))

(def the-original-code '((def x 1) (def y x)))

(def the-ast (j/analyze the-original-code))

(def the-unqualified-code (e/emit-form (j/analyze the-original-code)))

(def the-qualified-code (e/emit-form (j/analyze the-original-code) #{:qualified-symbols}))

the-unqualified-code
;((def x 1) (def y x))

the-qualified-code
;((def x 1) (def y bug/x))

In the second example, x and y are only qualified where it is used, not where they are declared.

Desired output:
the-qualified-code
;((def bug/x 1) (def bug/y bug/x))



 Comments   
Comment by Nicola Mometto [ 10/Oct/14 7:14 AM ]

I'm not sure about considering this a bug.
While it is true that (def foo/bar 1) is a valid form (when foo is *ns*), def always interns vars in the current ns *ns* so there is no ambiguity wrt what var bar in (def bar 1) will refer to.

Do you have a specific use case where emitting (def foo/bar 1) is ok but (def bar 1) is problematic?

Comment by Richard Davies [ 10/Oct/14 8:09 AM ]

Hi Nicola,

Here's my use case:

I am currently customising the "-emit-form" defmethods to give me a mapping of [original-form -> qualified-symbol] or [original-form -> hygenic-local] to build a graph data structure (which I want to visualise) of my code. (I'm also interested in tracking how the code is transformed by macros as I'd like to be able to display either the original code or the macro-transformed code.)

I'm not sure using the emitter is the best approach but it's easier for me to work with the emitter than to write something that parses the AST from scratch, especially as I want the all the emitted code to be regular Clojure except for the parts that give me the [original-form -> hygenic-local/qualified-symbol] mapping.

This is a quick summary of what I'm doing:

First I attach a unique id to the metadata of each form so that I can track forms from the original code to whatever it has been transformed to by any macros:

(postwalk (fn [x] (if (instance? clojure.lang.IObj x) (with-meta x {:id (swap! c inc)}) x)) form)

Then I modify any emitter defmethods to insert special nodes where there is a form that need to be made hygenic:

(defrecord Out [emit form])

(defmethod e/-emit-form :default
[ast opts]
(let [e (default/-emit-form ast opts) r (first (:raw-forms ast)) f (:form ast) o (Out. e f) i (:id (meta f))
o1 (if r (assoc o :raw r) o)
o2 (if i (assoc o1 :id i) o1)]
o2
))

So it would be nice if this approach also worked with all var symbols. I can work around it given that it's quite easy to figure out what namespace a def'ed symbol is in as you say. However, in favour of supporting this fix, it would make the :qualified-symbols emitter option more true to its description and as a consequence, it would make the emitter easier to tweak. Last argument in favour is that if you turn the :qualified-symbols on and then pass the emitted code back, the ns information is lost.

When I've added a (defmethod e/-emit-form :def ....) it doesn't seem to descend to the symbol in the def although I can of course pull the namespace info from the AST so it's all possible but it felt like a bit of a hack to get the mapping for the def symbol working.

BTW, if there's a simpler/cleaner way to do what I'm trying to do, I'm open to suggestions

Thanks,
Richard

Comment by Nicola Mometto [ 10/Oct/14 8:52 AM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/9315dc482cd90a6a76d3ad7e9be739ee0b16744f





[TANAL-93] Incorrect earmuff warnings Created: 25/Sep/14  Updated: 25/Sep/14  Resolved: 25/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I see non-dynamic var earmuff warnings for vars that do have :dynamic true in their metadata, like so:

(def ^:dynamic *smile-factory* nil)

I think this might be fixable with the following change:

diff --git a/src/eastwood/copieddeps/dep1/clojure/tools/analyzer/passes/warn_earmuff.clj b/src/eastwood/copieddeps/dep1/clojure/tools/analyzer/passes/warn_earmuff.clj
index ddb1488..ecf587d 100644
--- a/src/eastwood/copieddeps/dep1/clojure/tools/analyzer/passes/warn_earmuff.clj
+++ b/src/eastwood/copieddeps/dep1/clojure/tools/analyzer/passes/warn_earmuff.clj
@@ -19,7 +19,7 @@
                (> (count name) 2)  ;; Allow * and ** as non-dynamic names
                (.startsWith name "*")
                (.endsWith name "*")
-               (not (dynamic? (:var ast))))
+               (not (dynamic? (:var ast) (-> ast :meta :val))))
       (binding [*out* *err*]
         (println "Warning:" name "not declared dynamic and thus is not dynamically rebindable,"
                  "but its name suggests otherwise."


 Comments   
Comment by Nicola Mometto [ 25/Sep/14 1:24 PM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/3af99e2a8055c22609de2d64b1e0009e949c252d





[TANAL-92] Latest t.a.j throws exception on defrecord Created: 25/Sep/14  Updated: 25/Sep/14  Resolved: 25/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I get an exception thrown when calling analyze+eval on the last form of this input file:

(ns testcases.tanal-9)

(defrecord RecordTest [a b])

Stack trace (from unchecked-in working copy of Eastwood with latest t.a.j):

Exception in thread "main" java.lang.VerifyError: (class: testcases/tanal_9$reify__5886, method: create signature: (Lclojure/lang/IPersistentMap;)Ltestcases/tanal_9$reify__5886;) Expecting to find unitialized object on stack, compiling:(/private/var/folders/sf/wkqqkhmj1wb4322ttmrx8cl00000gn/T/form-init8013061043522900563.clj:1:142)
	at clojure.lang.Compiler.load(Compiler.java:7142)
	at clojure.lang.Compiler.loadFile(Compiler.java:7086)
	at clojure.main$load_script.invoke(main.clj:274)
	at clojure.main$init_opt.invoke(main.clj:279)
	at clojure.main$initialize.invoke(main.clj:307)
	at clojure.main$null_opt.invoke(main.clj:342)
	at clojure.main$main.doInvoke(main.clj:420)
	at clojure.lang.RestFn.invoke(RestFn.java:421)
	at clojure.lang.Var.invoke(Var.java:383)
	at clojure.lang.AFn.applyToHelper(AFn.java:156)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)
Caused by: java.lang.VerifyError: (class: testcases/tanal_9$reify__5886, method: create signature: (Lclojure/lang/IPersistentMap;)Ltestcases/tanal_9$reify__5886;) Expecting to find unitialized object on stack
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:190)
	at testcases.tanal_9$eval5888.invoke(form-init8013061043522900563.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:6703)
	at clojure.lang.Compiler.eval(Compiler.java:6666)
	at clojure.core$eval.invoke(core.clj:2927)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$_deftype.invoke(jvm.clj:300)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3902$fn__3904.invoke(jvm.clj:312)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:285)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:286)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze.invoke(analyzer.clj:123)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3924$fn__3926$fn__3933.invoke(jvm.clj:364)
	at clojure.core.protocols$fn__6074.invoke(protocols.clj:79)
	at clojure.core.protocols$fn__6031$G__6026__6044.invoke(protocols.clj:13)
	at clojure.core$reduce.invoke(core.clj:6289)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3924$fn__3926.invoke(jvm.clj:362)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:285)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval368$fn__370.invoke(analyzer.clj:297)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_body.invoke(analyzer.clj:368)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_let.invoke(analyzer.clj:522)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval464$fn__465.invoke(analyzer.clj:532)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:285)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:286)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:286)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval368$fn__370.invoke(analyzer.clj:297)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_body.invoke(analyzer.clj:368)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_let.invoke(analyzer.clj:522)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval464$fn__465.invoke(analyzer.clj:532)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:285)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:286)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval368$fn__370.invoke(analyzer.clj:297)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_body.invoke(analyzer.clj:368)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_fn_method.invoke(analyzer.clj:608)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$analyze_method_impls.invoke(jvm.clj:281)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3911$fn__3913$fn__3920.invoke(jvm.clj:342)
	at clojure.core$mapv$fn__6311.invoke(core.clj:6353)
	at clojure.core.protocols$fn__6086.invoke(protocols.clj:143)
	at clojure.core.protocols$fn__6057$G__6052__6066.invoke(protocols.clj:19)
	at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
	at clojure.core.protocols$fn__6080.invoke(protocols.clj:48)
	at clojure.core.protocols$fn__6031$G__6026__6044.invoke(protocols.clj:13)
	at clojure.core$reduce.invoke(core.clj:6289)
	at clojure.core$mapv.invoke(core.clj:6353)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3911$fn__3913.invoke(jvm.clj:342)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:285)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_in_env$fn__322.invoke(analyzer.clj:135)
	at clojure.core$mapv$fn__6311.invoke(core.clj:6353)
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:63)
	at clojure.core.protocols$fn__6093.invoke(protocols.clj:98)
	at clojure.core.protocols$fn__6057$G__6052__6066.invoke(protocols.clj:19)
	at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
	at clojure.core.protocols$fn__6076.invoke(protocols.clj:60)
	at clojure.core.protocols$fn__6031$G__6026__6044.invoke(protocols.clj:13)
	at clojure.core$reduce.invoke(core.clj:6289)
	at clojure.core$mapv.invoke(core.clj:6353)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval368$fn__370.invoke(analyzer.clj:296)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_body.invoke(analyzer.clj:368)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze_let.invoke(analyzer.clj:522)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval464$fn__465.invoke(analyzer.clj:532)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$eval3810$fn__3811.invoke(jvm.clj:63)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval364$fn__365.invoke(analyzer.clj:285)
	at clojure.lang.MultiFn.invoke(MultiFn.java:236)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$eval287$fn__288.invoke(analyzer.clj:68)
	at clojure.lang.MultiFn.invoke(MultiFn.java:231)
	at eastwood.copieddeps.dep1.clojure.tools.analyzer$analyze.invoke(analyzer.clj:123)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$analyze$fn__3947.invoke(jvm.clj:474)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.core$apply.invoke(core.clj:624)
	at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1862)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$analyze.invoke(jvm.clj:470)
	at eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm$analyze_PLUS_eval.invoke(jvm.clj:516)
	at eastwood.analyze_ns$analyze_file$fn__4284.invoke(analyze_ns.clj:260)
	at eastwood.analyze_ns$analyze_file.doInvoke(analyze_ns.clj:258)
	at clojure.lang.RestFn.invoke(RestFn.java:486)
	at eastwood.analyze_ns$analyze_ns.doInvoke(analyze_ns.clj:302)
	at clojure.lang.RestFn.invoke(RestFn.java:439)
	at eastwood.core$lint_ns.invoke(core.clj:325)
	at eastwood.core$run_eastwood$fn__5834.invoke(core.clj:584)
	at eastwood.core$run_eastwood.invoke(core.clj:583)
	at clojure.lang.Var.invoke(Var.java:379)
	at eastwood.versioncheck$run_eastwood.invoke(versioncheck.clj:15)
	at user$eval21.invoke(form-init8013061043522900563.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:6703)
	at clojure.lang.Compiler.eval(Compiler.java:6693)
	at clojure.lang.Compiler.load(Compiler.java:7130)
	... 11 more


 Comments   
Comment by Nicola Mometto [ 25/Sep/14 11:50 AM ]

This was caused by the last commit on t.a.jvm, I had the erraneous impression that __meta should be visible from inside reifies, but it looks like I was wrong.
Reverted the commit: https://github.com/clojure/tools.analyzer.jvm/commit/2f3499280dc2bb62d6f291738ecc04f91b0d7dbb





[TANAL-91] analyze+eval throws exception about wrong tag when it should not Created: 24/Sep/14  Updated: 25/Sep/14  Resolved: 25/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

At least, it seems to me that it does so. I'll let you decide, and decline this ticket if I'm wrong somewhere here, but I'd be curious to know how.

All three of the def forms in the sample REPL session below cause tools.analyzer(.jvm)'s latest versions to throw an exception with a "Wrong tag" message. The first two I agree with, but the last seems like it is correct code that should not throw such an exception. Clojure 1.6.0 (and maybe older versions) does not give any reflection warnings when the Var l4 is used as an argument in Java interop.

user=> (clojure-version)
"1.6.0"
user=> (set! *warn-on-reflection* true)
true
user=> (def ^long l2 -2)
#'user/l2
user=> (Math/abs l2)
CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: clojure.core$long@4201beb6, compiling:(NO_SOURCE_PATH:5:1) 
user=> (def ^{:tag long} l3 -2)
#'user/l3
user=> (Math/abs l3)
CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: clojure.core$long@4201beb6, compiling:(NO_SOURCE_PATH:7:1) 
user=> (def ^{:tag 'long} l4 -2)
#'user/l4
user=> (Math/abs l4)
2


 Comments   
Comment by Nicola Mometto [ 25/Sep/14 2:42 AM ]

fixed: https://github.com/clojure/tools.analyzer.jvm/commit/be662dfa604eea1282ded033f2dab09bbaf026a8





[TANAL-90] Exception thrown attempting to analyze :arglists containing keywords or booleans Created: 21/Sep/14  Updated: 21/Sep/14  Resolved: 21/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Steps to reproduce:

# git clone/pull latest version of Eastwood as of Sep 21 2014
# It should include version 0.6.0 of both tools.analyzer(.jvm), including fix for TANAL-88 and TANAL-89

git checkout update-tanal
lein install

# Edit ~/.lein/profiles.clj to have this in {:user {:plugins [ ... ]}} vector: [jonase/eastwood "0.1.5-SNAPSHOT"]

cd crucible
# This step is unnecessary if you have done it before
./clone.sh
cd repos/java.jdbc-2014-03-07
lein with-profile +1.6 eastwood

Exception I see looks like this:

Exception thrown during phase :analyze+eval of linting namespace clojure.java.jdbc
ClassCastException clojure.lang.Keyword cannot be cast to clojure.lang.IObj
	clojure.core/with-meta (core.clj:214)
	clojure.core/vary-meta (core.clj:640)
	eastwood.copieddeps.dep2.clojure.tools.analyzer.jvm/qualify-argvec/fix-tag--3837 (jvm.clj:185)
	clojure.core/mapv/fn--6311 (core.clj:6353)
[ ... more stack trace lines omitted ... ]

The exception goes away if I delete the :arglists of the function query, which is the form being analyzed when the exception occurs. It also goes away if I change the keywords and booleans in the :arglists to symbols, or remove them entirely (done not as a proposed change to java.jdbc, but to do a little bit of investigation to see what causes the exception).



 Comments   
Comment by Nicola Mometto [ 21/Sep/14 2:26 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/698096cebc271d7b5f44eefe4e5b79ebde9df8e2





[TANAL-89] analyze+eval throws exception with message 'Cannot set! non-assignable target' for code that shouldn't Created: 20/Sep/14  Updated: 20/Sep/14  Resolved: 20/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Steps to reproduce:

# git clone/pull latest version of Eastwood as of Sep 20 2014
# It should include version 0.6.0 of both tools.analyzer(.jvm), including fix for TANAL-88

git checkout update-tanal
lein install

# Edit ~/.lein/profiles.clj to have this in {:user {:plugins [ ... ]}} vector: [jonase/eastwood "0.1.5-SNAPSHOT"]

cd crucible
# This step is unnecessary if you have done it before
./clone.sh
cd repos/tools.reader-2014-03-05
lein with-profile +1.6 eastwood

I get a similar exception with tools.nrepl



 Comments   
Comment by Nicola Mometto [ 20/Sep/14 7:28 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/e54155a3c7aeacdcd7321ff57ddba4f0788161b0





[TANAL-88] analyze+eval throws NullPointerException, perhaps related to proxy expressions Created: 20/Sep/14  Updated: 20/Sep/14  Resolved: 20/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I haven't narrowed it down completely, and it may be a coincidence that several of the places I have seen this occur involve proxy expressions.

Steps to reproduce:

# git clone/pull latest version of Eastwood as of Sep 20 2014
# It should include version 0.6.0 of both tools.analyzer(.jvm)

git checkout update-tanal
lein install

# Edit ~/.lein/profiles.clj to have this in {:user {:plugins [ ... ]}} vector: [jonase/eastwood "0.1.5-SNAPSHOT"]

cd crucible
# This step is unnecessary if you have done it before
./clone.sh
cd repos/tools.reader-2014-03-05
lein with-profile +1.6 eastwood

I get a similar exception with tools.logging and tools.nrepl, plus some other crucible projects, but those should be enough to go on.



 Comments   
Comment by Nicola Mometto [ 20/Sep/14 5:37 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/e0b686a33e89d7d74d5131e3f4356a0a9aea20ff





[TANAL-86] add a function to convert a cljs.analyzer namespace map to a t.a.js one Created: 10/Aug/14  Updated: 05/Sep/14  Resolved: 05/Sep/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None





[TANAL-82] new/catch classes should be analyzed Created: 17/Jul/14  Updated: 31/Aug/14  Resolved: 20/Jul/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None





[TANAL-24] clojure.tools.analyzer.passes.jvm/validate throws an exception when :tag metadata cannot be resolved to a Class Created: 04/Dec/13  Updated: 09/Aug/14  Resolved: 09/Aug/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 1
Labels: None


 Description   

While the Clojure Compiler silently ignores :tag metadata in some forms when it cannot be resolved to a class, c.t.a.passes.jvm/validate throws an exception every time.

Some libraries (test.generators) use :tag metadata to store instructions for generating test input data, those can be everything, from symbols to quoted clojure expressions – the Clojure Compiler allows for this but validate throws an exception.

Generally however, when :tag cannot be resolved to a class it's not beacause of a design choice, but because of a Clojure bug or a user-code bug.

Some examples are:

  • tagging a defn form in the fn name with a primitive type hint:
    user=> (defn ^long x [] 1)
    #'user/x
    user=> (:tag (meta #'x))
    #<core$long clojure.core$long@6596f6ef>
  • using extend-type with a run-time resolved class http://dev.clojure.org/jira/browse/CLJ-1308
    user=> (defprotocol p (f [_]))
    p
    user=> (binding [*print-meta* true] (pr (macroexpand-1 '(extend-type (class 1) p (f [_])))))
    (clojure.core/extend (class 1) p {:f (fn ([^(class 1) _]))})
    nil
  • type hinting a form in a namespace with an imported class doesn't qualify the class in :tag http://dev.clojure.org/jira/browse/CLJ-1232
    user=> (ns foo (:import clojure.lang.RT))
    nil
    foo=> (defn x ^RT [])
    #'foo/x
    foo=> (ns bar (:use foo))
    nil
    bar=> (:tag (meta #'x))
    RT
    bar=> (.hashCode (x))
    CompilerException java.lang.IllegalArgumentException: Unable to resolve classname: RT, compiling:(NO_SOURCE_PATH:5:1)

A clojure-dev post has been opened asking whether tools.analyzer.jvm is correct in assuming that non class-resolvible tags are an error and thus an exception could be thrown, or if the current behaviour of the Cloure implementation of throwing in some special places and silently ignoring malformed :tag values is by design. It hasn't have received an official reply yet. https://groups.google.com/d/msg/clojure-dev/hRZFuaiB_50/mzKLirgZWmUJ



 Comments   
Comment by Nicola Mometto [ 05/Dec/13 9:23 AM ]

This is somewhat an undefined behaviour in clojure, it's not clear to me from the documentation wether tagging elements with things that cannot be resolved to a class should work or not.

I'll ask the clojure-dev ML about this before doing anything.

Thanks

Comment by Nicola Mometto [ 05/Dec/13 2:00 PM ]

clojure-dev post: https://groups.google.com/d/msg/clojure-dev/hRZFuaiB_50/mzKLirgZWmUJ

Comment by Andy Fingerhut [ 02/Apr/14 1:24 PM ]

Ticket TGEN-5 was created on test.generative as a result of this issue, too, with a suggested patch so that test.generative no longer 'leaks' such tag values.

Comment by Nicola Mometto [ 09/Aug/14 10:10 AM ]

A workaround is now available in tools.analyzer.jvm master, the default behaviour is still to throw on "wrong" tags but it's now available a wrong-tag handler that allows user to customize the behaviour of the validate pass on wrong tags.
PoC: https://github.com/clojure/tools.analyzer.jvm/blob/master/src/test/clojure/clojure/tools/analyzer/jvm/passes_test.clj#L209-L216





[TANAL-42] Document the AST nodes && the passes Created: 03/Jan/14  Updated: 02/Aug/14

Status: Open
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Unresolved Votes: 0
Labels: None


 Comments   
Comment by Max Kreminski [ 19/Apr/14 5:11 PM ]

I put together a quick reference to the AST node structures a few days ago when trying to identify discrepancies between the tools.analyzer AST format and the internal AST format used by the ClojureScript compiler. It's nowhere near as complete as it could be, but I figured I'd post it here just in case it winds up being useful when writing official docs.

http://mkremins.github.io/clojure-ast-ref/

Comment by Nicola Mometto [ 19/Apr/14 5:36 PM ]

At first glance it looks really well done.
I'll take some time to review it and contribute if necessary, in the mean time if you want you can add a link pointing at your quickref in the tools.analyzer wiki.

Thanks

Comment by Ian Tegebo [ 29/May/14 12:59 AM ]

It appears incomplete, e.g. :static-call is missing. This appears to be because it's added not by tools.analyzer but by tools.analyzer.jvm; is the intention that platform-specific AST nodes are to be considered hidden implementation details?

If not, should a separate issue requesting documentation be filed with the "concrete" platforms, e.g. jvm.tools.analyzer?

Comment by Max Kreminski [ 02/Jun/14 11:49 AM ]

I haven't yet had time to document the platform-specific node types added by tools.analyzer.{jvm,js}, although I do intend to add them to the quickref in the near future.

jvm.tools.analyzer appears to use a similar but incompatible AST format, which I currently don't plan on trying to document in the quickref. Filing a separate issue on that project might be the best way to move forward there.

Comment by Andy Fingerhut [ 03/Jun/14 11:21 AM ]

jvm.tools.analyzer is probably not worth documenting at this time, except for historical purposes. tools.analyzer.{jvm,js} are far more likely to be maintained and enhanced in the future.

Comment by Ian Tegebo [ 03/Jun/14 11:58 AM ]

Oops, I think I introduced some confusion when I mistakenly referenced jvm.tools.analyzer. I did so because I didn't see a JIRA project for tools.analyzer.jvm.

On 29/May14 12:59 AM, I meant "... the 'concrete' platforms, e.g. tools.analyzer.{jvm,js} ...".

As an aside, had anyone considered a slight rewrite of the code so that AST (reference) documentation can be extracted rather than separately maintained?

Comment by Nicola Mometto [ 02/Aug/14 7:12 AM ]

This has been addressed mostly with the official AST quickrefs: http://clojure.github.io/tools.analyzer/doc/quickref.html and http://clojure.github.io/tools.analyzer.jvm/doc/quickref.html

I'm changing the priority to minor





[TANAL-84] Simplify tag handling for castings/nil insertion/value popping Created: 25/Jul/14  Updated: 25/Jul/14

Status: Open
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently the compiler uses the op type, o-tag, tag and the context to figure out where to put casts, where to insert nils or where to pop a value off the stack.
This doesn't work well as it's really hard to figure out when it's actually necessary to e.g. pop a value or if it has already been removed from the stack.
Ideally the analyzer should provide more information to the emitter so that it's easier to understand when and where a casting/nil insertion/value popping is necessary






[TANAL-83] Make it possible for try expression to return primitives Created: 24/Jul/14  Updated: 24/Jul/14  Resolved: 24/Jul/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None





[TANAL-81] adds end-line and end-column information to source-info Created: 14/Jun/14  Updated: 16/Jun/14  Resolved: 16/Jun/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Benedek Fazekas Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File 0001-adds-end-line-and-end-column-information-to-source-i.patch     Text File end-line_and_end-column.patch    
Patch: Code and Test

 Description   

goal particularly is to use this info in clj-refactor but this additional info can be useful for other code analyzer and/or munging libraries.

currently we are experimenting to turn clj-refactor into a nrepl middleware based, editor agnostic clojure refactor tool, see cider-and-ast branch. for this we use tools.analyzer and work on the AST. the feature we use in the experiment intends to remove debug functions like println, prn etc. in order to easily integrate with our client which is emacs-lisp code for now we need end-line information, please see this and this snippets particularly.



 Comments   
Comment by Nicola Mometto [ 14/Jun/14 8:38 AM ]

I'll be more than happy to take this patch, can you please attach patch created with git format-patch?
Also since this is a contrib library you need to sign the elettronic Clojure CA, https://secure.echosign.com/public/hostedForm?formid=95YMDL576B336E
Thanks

Comment by Benedek Fazekas [ 14/Jun/14 8:53 AM ]

hi,

I just signed electronic Clojure CA 30 mins ago. Let me sort out the format patch thing too...

Comment by Benedek Fazekas [ 14/Jun/14 8:59 AM ]

hope this suffices

Comment by Nicola Mometto [ 14/Jun/14 9:22 AM ]

Ok, it's going to take a couple of days until your name appears in the contributors list (http://clojure.org/contributing) once it's there I'll merge this.
I was planning on releasing a new version of ta/taj today, I'll wait a couple of extra days to get this included.

Thanks again

Comment by Benedek Fazekas [ 14/Jun/14 9:55 AM ]

sounds great, thx!

Comment by Nicola Mometto [ 16/Jun/14 1:29 PM ]

Pushed: https://github.com/clojure/tools.analyzer/commit/47841ee516b1caccbf7dcfc10de17119a5b4593c





[TANAL-80] Update the namespace in the env on each recursive all to analyze+eval Created: 12/Jun/14  Updated: 12/Jun/14  Resolved: 12/Jun/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-80-v1.diff     File tanal-80-v2.diff    

 Description   

Without a change similar to the one in the attached patch, I do not see how to get the correct namespace associated with some warnings in Eastwood, particularly the ones for :unlimited-use. The value bound to ns does change as soon as an in-ns form is evaluated, which is in the middle of handling the Gilardi scenario.



 Comments   
Comment by Andy Fingerhut [ 12/Jun/14 6:02 AM ]

Patch tanal-80-v1.diff is one modification I have tested, and it seems to do the job. With this, Eastwood passes its few unit tests, and the crucible test results, while not 100% identical to what they were before using ana.jvm/analyze+eval, are very very close.

The introduction of butlast+last is not a functional change, but merely giving name to some code inside analyze+eval and defining it separately. Certainly not a necessary part of the desired change, but I thought I'd suggest it.

Comment by Nicola Mometto [ 12/Jun/14 6:17 AM ]

Is there any reason why you replaced the mapv with a loop/recur? I don't think there's any behavioural difference and if so can you update the patch to switch back to mapv?

Otherwise the patch looks reasonable and I'm eager to apply it.

Comment by Andy Fingerhut [ 12/Jun/14 6:37 AM ]

tanal-80-v2.diff is same as -v1, but switches loop back to mapv.

Comment by Nicola Mometto [ 12/Jun/14 6:40 AM ]

Fixed, thanks: https://github.com/clojure/tools.analyzer.jvm/commit/cbc5ef209bebe29f814993dfc022bd5fd74be05c





[TANAL-79] Add changelog Created: 12/Apr/14  Updated: 12/Apr/14  Resolved: 12/Apr/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None





[TANAL-75] t.a(.jvm) form emitted from AST has ^{:tag Object} where original source had ^clojure.lang.IPersistentCollection Created: 30/Mar/14  Updated: 31/Mar/14  Resolved: 31/Mar/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

With latest t.a(.jvm) (0.1.0-beta9 for both), latest Eastwood master, and latest core.rrb-vector repo as of today, 'lein check' gives no reflection warnings, but the following command does:

% lein with-profile +1.6 eastwood '{:namespaces [clojure.core.rrb-vector]}'
Reflection warning, clojure/data/priority_map.clj:215:19 - call to method equiv on java.lang.Object can't be resolved (no such method).
Reflection warning, clojure/core/memoize.clj:72:23 - reference to field cache can't be resolved.
== Eastwood 0.1.2-SNAPSHOT Clojure 1.6.0 JVM 1.7.0_51
== Linting clojure.core.rrb-vector ==
Reflection warning, clojure/core/rrb_vector.clj:100:17 - call to method cons on java.lang.Object can't be resolved (no such method).
Reflection warning, clojure/core/rrb_vector.clj:150:17 - call to method cons on java.lang.Object can't be resolved (no such method).

Ignore the first two reflection warnings. It is the two in clojure/core/rrb_vector.clj that are the reason for this ticket. Use the following command to get voluminous debug output from Eastwood, including the emitted forms just before they are eval's, to see the java.lang.Object tag in place of the clojure.lang.IPersistentCollection in the source code:

% lein with-profile +1.6 eastwood '{:namespaces [clojure.core.rrb-vector] :debug #{:all}}' > out.txt


 Comments   
Comment by Nicola Mometto [ 30/Mar/14 3:02 AM ]

Thanks for the feedback, I need to do a complete overhaul of the validate-loop-locals pass, hopefully there might be room for a small performance enhancement too.

Comment by Nicola Mometto [ 30/Mar/14 10:57 PM ]

This fixes it, https://github.com/clojure/tools.analyzer.jvm/commit/a95b72a1ce519150816d45b8fea0317d35e7ca32

Since I honestly can't figure out why I was overriding :form there, I'll let this ticket open until more testing confirms that this commit hasn't introduced some other issues.

Comment by Nicola Mometto [ 31/Mar/14 12:53 PM ]

More testing hasn't found any case where this commit caused a regression so I feel confident to close this ticket.

Comment by Andy Fingerhut [ 31/Mar/14 1:09 PM ]

Agreed that after this change I have seen no new problems in the Eastwood crucible set of projects using t.a(.jvm).





[TANAL-77] Different reflection warning from Clojure 1.6 for compiling interop expression directly vs. after emitting from t.a(.jvm) Created: 31/Mar/14  Updated: 31/Mar/14  Resolved: 31/Mar/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

Latest t.a(.jvm) as of today (after TANAL-75 commit), Clojure 1.6.0, latest Eastwood as of today, latest version of 'fs' project as of Jan 29 2014 (and today):

% lein with-profile +1.6 eastwood '{:namespaces [ me.raynes.fs ]}'
Reflection warning, clojure/data/priority_map.clj:215:19 - call to method equiv on java.lang.Object can't be resolved (no such method).
Reflection warning, clojure/core/memoize.clj:72:23 - reference to field cache can't be resolved.
== Eastwood 0.1.2-SNAPSHOT Clojure 1.6.0 JVM 1.7.0_51
== Linting me.raynes.fs ==
Reflection warning, me/raynes/fs.clj:156:23 - reference to field toUrl on java.io.File can't be resolved.
Reflection warning, me/raynes/fs.clj:489:42 - reference to field getName can't be resolved.

It is the warning about 'reference to field toUrl' that is different here than when you run 'lein check', when the corresponding warning output line is:

Reflection warning, me/raynes/fs.clj:156:23 - call to toUrl can't be resolved.

I think that 'lein check' is warning about the macroexpanded version (. (. this (toFile)) (toUrl)), but t.a(.jvm) is emitting the form (.toUrl (.toFile this)).



 Comments   
Comment by Nicola Mometto [ 31/Mar/14 9:34 AM ]

There's nothing much I can do for the cases where tools.analyzer.jvm avoids some reflection that clojure does.

It's an enhancement that tools.analyzer.jvm provides over the current Clojure and I'd frankly consider this a bug of clojure if it cannot statically figure out the types of simple expressions like (.. this (toFile) (toUrl))

Comment by Andy Fingerhut [ 31/Mar/14 11:52 AM ]

Sorry for the noise. I think I see what you mean now. However, I would point out that this is a case where Clojure gives a reflection warning on the original code, and also on the code emitted by t.a(.jvm), but at least with Clojure 1.6.0 it is a different warning, simply because the Java interop form is different in the two cases. I think this is due to ambiguity in Java interop between whether (. symbol instance) means that symbol is a Java field, or a no-arg method.

Comment by Nicola Mometto [ 31/Mar/14 11:58 AM ]

Oh, sorry, I misread the description.
I'll look into this then

Comment by Nicola Mometto [ 31/Mar/14 12:10 PM ]

Ok so, still nothing we can do.
tools.analyzer.jvm macroexpands the . form differntly than clojure, but it's still a valid expansion.

Anyway, looking into why this causes a reflection warning I discovered that the fs library has a typo in there, the method is called "toURL" rather then "toUrl"





[TANAL-78] Forms emitted from t.a(.jvm) produce reflection warning not present in original code Created: 31/Mar/14  Updated: 31/Mar/14  Resolved: 31/Mar/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Latest t.a(.jvm) as of today (after TANAL-75 commit), Clojure 1.6.0, latest Eastwood as of today, latest version of 'tools.nrepl' lib as of Feb 28 2014 (and today):

% lein with-profile +1.6 eastwood '{:namespaces [ clojure.tools.nrepl.middleware.session ]}'
Reflection warning, clojure/data/priority_map.clj:215:19 - call to method equiv on java.lang.Object can't be resolved (no such method).
Reflection warning, clojure/core/memoize.clj:72:23 - reference to field cache can't be resolved.
== Eastwood 0.1.2-SNAPSHOT Clojure 1.6.0 JVM 1.7.0_51
== Linting clojure.tools.nrepl.middleware.session ==
Reflection warning, clojure/tools/logging.clj:270:1 - call to method isLoggable can't be resolved (target class is unknown).
Reflection warning, clojure/tools/logging.clj:270:1 - call to method log can't be resolved (target class is unknown).
Reflection warning, clojure/tools/logging.clj:270:1 - call to method log can't be resolved (target class is unknown).
Reflection warning, clojure/tools/nrepl/middleware/session.clj:230:30 - call to method put can't be resolved (target class is unknown).
Reflection warning, clojure/tools/nrepl/middleware/session.clj:230:30 - call to method put can't be resolved (target class is unknown).

The only lines in the output I am concerned about here are the last 2, about 'call to method put can't be resolved'. Those warnings do not appear in the output of 'lein with-profile +1.6 check' on the same library.

All other reflection warnings produced by Eastwood are the same set as those produced by 'lein check'



 Comments   
Comment by Nicola Mometto [ 31/Mar/14 11:51 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/c323142cb670f54e7879a906e3b9f295c6d8b5f2

tools.analyzer was discarding the macroexpanded form's meta, now it merges the &form meta into the macroexpanded form's one





[TANAL-76] Add link to Tim Baldridge's tools.analyzer talk to README Created: 30/Mar/14  Updated: 30/Mar/14  Resolved: 30/Mar/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-76-v1.diff    
Patch: Code

 Description   

While not official documentation, anyone interested in tools.analyzer would likely want to watch it.



 Comments   
Comment by Nicola Mometto [ 30/Mar/14 10:58 AM ]

https://github.com/clojure/tools.analyzer/commit/1eb64bba9e7e8df11076d60f7d0ddb4cc26ce1b3
Thanks





[TANAL-73] add a pass to warn on reflection when *warn-on-reflection* is true Created: 16/Feb/14  Updated: 06/Mar/14  Resolved: 06/Mar/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by Nicola Mometto [ 06/Mar/14 11:57 AM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/df3ee5765195f4163c93a4f76dfba7074b04f5d2





[TANAL-74] locals clearing for loop closed-overs does not nest Created: 24/Feb/14  Updated: 24/Feb/14  Resolved: 24/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

This currently returns nil, should return true

(-> (a/analyze '(loop [] (do (let [a 1] (loop [] (if 1 a (recur)))) (recur)) (a/empty-env)) :body :ret :statements first :body :ret :body :ret :then :to-clear?)


 Comments   
Comment by Nicola Mometto [ 24/Feb/14 3:14 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/683ec31723aac225b83b1c718d3c2666993d37bf





[TANAL-67] Restore some data to AST that Eastwood uses for line/col numbers for some linters Created: 10/Feb/14  Updated: 17/Feb/14  Resolved: 14/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None

Attachments: File tanal-67-v1.diff    

 Description   

Warning: Treat this suggested change as the ravings of a hacker who is poking at things without understanding how they work, and isn't even looking to see if there are bad consequences that may result.

This commit to tools.analyzer removed some data from the resulting ASTs that Eastwood was using for printing line/col numbers for some linters: https://github.com/clojure/tools.analyzer/commit/96fff1041bdebf75f3b539c439a41f3c5f496c7c

I did a quick experiment with the latest t.a(.jvm) and Eastwood as of Feb 10 2014, and 'lein test' passes with Eastwood again with the attached patch.

If there are good reasons to avoid applying this patch, I can certainly find another way to get the desired line/col numbers in Eastwood's error messages. However, if you consider keeping this data in the AST harmless, it would be nice to have.



 Comments   
Comment by Nicola Mometto [ 10/Feb/14 7:51 PM ]

This change was made for tools.emitter.jvm to generate correctly munged class-names for functions, I'm ok with accepting this patch but I'll need to fix the behaviour on tools.emitter.jvm first then.

Comment by Nicola Mometto [ 14/Feb/14 12:50 PM ]

I actually went completely on another direction with this after some thought: https://github.com/clojure/tools.analyzer/commit/ae2b7ee34a326905b332d6a77cd7229f69626fb9

Now :name is a string and represents the munged name for a fn.
This decision was made to make :name more consistent.

It should be trivial to write a pass for eastwood to restore the previous functionallity though, and if you point me out to where this is needed I'll be glad to do that myself.

Comment by Andy Fingerhut [ 17/Feb/14 2:08 PM ]

I created a Github issue for Eastwood at the link below with steps to reproduce, and where in the Eastwood source code it is attempting to get line/col numbers but getting nils, when it formerly got line/col numbers with older t.a(.jvm) versions.

https://github.com/jonase/eastwood/issues/58





[TANAL-65] Enhance clear-locals to clear loop closed-overs in return position Created: 26/Jan/14  Updated: 15/Feb/14  Resolved: 15/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   
(fn [x] (loop [a 1] (if (zero? a) x (recur (dec a)))))

x is safe to clear.



 Comments   
Comment by Nicola Mometto [ 15/Feb/14 12:32 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/6abcad1b81bdfbed6280480ac3c5129f98d3c2cf





[TANAL-71] Memoization on reflective utils breaks deftype redefinition Created: 11/Feb/14  Updated: 11/Feb/14  Resolved: 11/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Blocker
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by Nicola Mometto [ 11/Feb/14 5:31 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/dc02dc61b083913d21574a7a03c8d34666a2c122





[TANAL-72] t.a(.jvm) throw exception during analysis of namespace clojure.core.async.impl.channels due to ^int type hint on constant Created: 11/Feb/14  Updated: 11/Feb/14  Resolved: 11/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Duplicate Votes: 0
Labels: None


 Description   

To reproduce with latest t.a(.jvm) and Eastwood:

$ cd path/to/core.async
$ lein eastwood '{:namespaces [ clojure.core.async.impl.channels ]}'

The exception is no longer thrown if you remove the int type hint from MAX-QUEUE-SIZE in the namespace where it is defined.

This type hint was there in core.async since August 2013, so earlier versions of t.a(.jvm) were not throwing this exception with this code, I am pretty sure.



 Comments   
Comment by Nicola Mometto [ 11/Feb/14 11:28 AM ]

Duplicate of http://dev.clojure.org/jira/browse/TANAL-24





[TANAL-70] emit-form throws exception on AST produced from clojure.java.browse-ui namespace Created: 11/Feb/14  Updated: 11/Feb/14  Resolved: 11/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

To reproduce with latest t.a(.jvm) and Eastwood:

$ lein eastwood '{:namespaces [ clojure.java.browse-ui ]}'
== Eastwood 0.1.1-SNAPSHOT Clojure 1.5.1 JVM 1.7.0_15
== Linting clojure.java.browse-ui ==
Exception thrown during phase :emit-form of linting namespace clojure.java.browse-ui
ClassCastException clojure.lang.Symbol cannot be cast to java.lang.Class
        clojure.tools.analyzer.passes.jvm.emit-form/eval3479/fn--3481 (emit_form.clj:94)
        clojure.lang.MultiFn.invoke (MultiFn.java:231)
        clojure.tools.analyzer.passes.jvm.emit-form/-emit-form* (emit_form.clj:16)
        clojure.tools.analyzer.passes.jvm.emit-form/eval3486/fn--3488/fn--3490 (emit_form.clj:99)
        clojure.core/mapv/fn--6258 (core.clj:6241)

[... rest of stack trace deleted for brevity ...]


 Comments   
Comment by Nicola Mometto [ 11/Feb/14 10:01 AM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/e70bee842e943fc7d465265b2be03ee9e3e14674





[TANAL-69] t.a(.jvm) returns ASTs with method calls having arg type nil in some cases Created: 10/Feb/14  Updated: 11/Feb/14  Resolved: 11/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

I haven't found a very short example of this, but on easy way to see such an AST is to update to the latest t.a(.jvm) and Eastwood (after 2 commits made late on Feb 10 2014 to Eastwood for extra debug info in this case), and run the following command while in the Eastwood project root directory:

$ lein eastwood '{:namespaces [ eastwood.util ]}'
== Eastwood 0.1.1-SNAPSHOT Clojure 1.5.1 JVM 1.7.0_15
== Linting eastwood.util ==
Error: Bad arg-type nil for method named identical for class clojure.lang.Util, full arg type list (java.lang.Object, nil).  ast pprinted below for debugging tools.analyzer:

[ ... rest of ast deleted for brevity ... ]

Perhaps the type info needed for Eastwood to do its deprecation checks on Java method calls is elsewhere in the AST now, but I haven't found it yet if so.



 Comments   
Comment by Nicola Mometto [ 11/Feb/14 5:28 AM ]

I fixed this on the eastwood side with https://github.com/jonase/eastwood/commit/6fa94fef3c25bc6b98fec9dc139f63d7b1db49fd

A tag for "nil" doesn't really make any sense, to avoid this errors ensure-tag replaces all nil tags with Object.





[TANAL-68] doseq form causes t.a(.jvm) emit-form to generate code that Clojure cannot compile Created: 10/Feb/14  Updated: 10/Feb/14  Resolved: 10/Feb/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I can reproduce this with a namespace that contains only a single function like this, with latest t.a(.jvm) and Eastwood as of Feb 10 2014:

(defn foo []
  (doseq [i (range 10)]
    (println i)))

That in a namespace and 'lein eastwood '{:namespaces [ my.ns ] :debug #{:eval}}' shows the output of emit-form that causes the Clojure compiler to throw an exception.



 Comments   
Comment by Nicola Mometto [ 10/Feb/14 8:02 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/1c3e1cdb823c34f7117fcc7f917163a0fa5a289f





[TANAL-66] Fix locals tag handling Created: 27/Jan/14  Updated: 27/Jan/14  Resolved: 27/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by Nicola Mometto [ 27/Jan/14 5:32 PM ]

Handled on the tools.emitter.jvm side https://github.com/clojure/tools.emitter.jvm/commit/c08106a1d77aa8866a9392e33d707c6eeef82dbd





[TANAL-64] Enhance infer-tag to ignore untyped nodes Created: 26/Jan/14  Updated: 27/Jan/14  Resolved: 27/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

just as we do for :recur, ignore :throw and all the other untyped nodes



 Comments   
Comment by Nicola Mometto [ 27/Jan/14 12:24 PM ]

Fixed for :throw: https://github.com/clojure/tools.analyzer.jvm/commit/91c76b45e84ae5bef2375b12b50a695ec2c01857

The remaining nodes :monitor-enter/:monitor-exit/:deftype should never be in return position.





[TANAL-63] clear-locals is broken Created: 26/Jan/14  Updated: 26/Jan/14  Resolved: 26/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Blocker
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

needs to be rewritten.



 Comments   
Comment by Nicola Mometto [ 26/Jan/14 9:49 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/8720fbc2d693dba4626ec992941f042073ae6ab6
https://github.com/clojure/tools.analyzer.jvm/commit/d674dcb0d8d8ed1b61421e254070ee5ed6214a9a





[TANAL-62] An exception is no longer thrown while linting project pantomime namespace pantomime.media Created: 23/Jan/14  Updated: 26/Jan/14  Resolved: 26/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

This is not necessarily a bug, but it is a change that resulted from the most recent commits (last 1 or 2 days) to tools.analyzer(.jvm). With current latest versions of these and Eastwood, and pantomime (part of Eastwood crucible), this command causes no exception to be thrown during analysis:

lein eastwood '{:namespaces [ pantomime.media ]}'

Before the most recent commits to tools.analyzer(.jvm), there was an exception due to line 97 in the source file because of a ^Map type hint on a locally bound symbol in a when-let form.



 Comments   
Comment by Nicola Mometto [ 24/Jan/14 5:31 AM ]

Andy, I'm currently working on making tools.analyzer.jvm and tools.emitter.jvm play nice with each other again, the tag handling situation is a bit problematic and I'm working on making it simpler for t.e.j to handle casting.

Expect things to be broken in master for a couple of days until thing settle down.

Comment by Andy Fingerhut [ 24/Jan/14 9:55 AM ]

No problem, and sorry for the noise. If you remember, let me know when you think things are settled down and I will update and test then.

Comment by Nicola Mometto [ 25/Jan/14 11:47 AM ]

Should be back to the previous behaviour now.
One thing to notice is that I removed :ret-tag and :bind-tag, now every expression has both :tag and :o-tag.
:o-tag represents the effective type of the expression, while :tag represents the "needed" tag of the expression.

E.g

^clojure.lang.IPersistentVector []

will have clojure.lang.IPersistentVector as :tag and clojure.lang.PersistentVector as :o-tag.

This means that that we're not throwing any :tag-kind anymore since it all got unified. This will probably require some changes to eastwood.

Comment by Andy Fingerhut [ 26/Jan/14 12:22 PM ]

OK, latest t.a(.jvm) is back to throwing an exception on that code now. I have updated Eastwood to no longer use :tag-kind for deciding what kind of error messages to print when a "Can't find class" exception is thrown. I believe nothing else in Eastwood depended upon :ret-tag or :bind-tag

Comment by Nicola Mometto [ 26/Jan/14 12:42 PM ]

Ok, I'm closing the ticket then.





[TANAL-61] Exception 'Ambiguous method signature for method: cons' when Eastwood analyzes namespace clojure.core.match Created: 21/Jan/14  Updated: 21/Jan/14  Resolved: 21/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

With latest tools.analyzer(.jvm), Eastwood, and core.match:

$ lein eastwood '{:namespaces [ clojure.core.match ]}'
[ ... many lines deleted ... ]
Exception thrown during phase :analyze of linting namespace clojure.core.match
Got exception with extra ex-data:
    msg='Ambiguous method signature for method: cons'
    (keys dat)=(:file :method :interfaces :form :params :matches)
    (:form dat)=
(^{:line 307, :column 4, :end-line 307, :end-column 8} cons
 [^{:line 307, :column 10, :end-line 307, :end-column 11} _
  ^{:line 307, :column 12, :end-line 307, :end-column 13} x]
 (^{:line 308, :column 6, :end-line 308, :end-column 17} PatternRow.
  (^{:line 308, :column 19, :end-line 308, :end-column 23} conj
   ^{:line 308, :column 24, :end-line 308, :end-column 26} ps
   ^{:line 308, :column 27, :end-line 308, :end-column 28} x)
  ^{:line 308, :column 30, :end-line 308, :end-column 36} action
  ^{:line 308, :column 37, :end-line 308, :end-column 45} bindings))
ExceptionInfo Ambiguous method signature for method: cons
	clojure.core/ex-info (core.clj:4355)
	clojure.tools.analyzer.passes.jvm.validate/eval2516/fn--2518 (validate.clj:289)
	clojure.lang.MultiFn.invoke (MultiFn.java:227)
	clojure.tools.analyzer.passes.jvm.validate/validate (validate.clj:326)
	clojure.tools.analyzer.ast/cycling/fn--53 (ast.clj:19)

[ ... many lines deleted ... ]


 Comments   
Comment by Nicola Mometto [ 21/Jan/14 5:29 PM ]

https://github.com/clojure/tools.analyzer.jvm/commit/f820fe3da5e37e4532beb04de814f45eb81b7919

I rewrote the logic for try-best-match, now it's much simplier and should cover all the cases.





[TANAL-60] Exception 'Ambiguous method signature for method: assoc' when Eastwood analyzes code containing 'defrecord' Created: 21/Jan/14  Updated: 21/Jan/14  Resolved: 21/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I haven't narrowed it down to exactly which commit may have caused this to change, but it was probably within the last 3-4 days.

Example, with latest tools.analyzer(.jvm) and Eastwood, but I believe that just about any namespace with a defrecord in it will trigger this issue:

$ lein eastwood '{:namespaces [ clojurewerkz.neocons.rest ]}'

[ ... some lines deleted here ... ]

Exception thrown during phase :analyze of linting namespace clojurewerkz.neocons.rest
Got exception with extra ex-data:
    msg='Ambiguous method signature for method: assoc'
    (keys dat)=(:file :method :interfaces :form :params :matches)
    (:form dat)=

[ ... many lines deleted here ... ]


 Comments   
Comment by Nicola Mometto [ 21/Jan/14 1:00 PM ]

I fixed this 2 days ago but somehow forgot to commit it.

https://github.com/clojure/tools.analyzer.jvm/commit/398dc661dc78821da8d15317b45915ad30ee62e8





[TANAL-59] eval'ing emit-form result gives 2 reflection warnings that 'lein check' does not, for core.incubator namespace Created: 17/Jan/14  Updated: 19/Jan/14  Resolved: 17/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

With latest t.a(.jvm), Eastwood, and core.incubator, here is output I see:

% lein eastwood '{:namespaces [ clojure.core.incubator-test ]}'
Reflection warning, clojure/data/priority_map.clj:215:19 - call to equiv can't be resolved.
Reflection warning, clojure/core/memoize.clj:72:23 - reference to field cache can't be resolved.
== Eastwood 0.1.1-SNAPSHOT Clojure 1.5.1 JVM 1.7.0_15
== Linting clojure.core.incubator-test ==
Reflection warning, clojure/core/incubator.clj:90:7 - reference to field getClass can't be resolved.
Reflection warning, clojure/core/incubator.clj:90:7 - reference to field isArray can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:17:39 - reference to field toString can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:20:39 - reference to field toString can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:21:39 - call to get can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:21:39 - reference to field toString can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:25:15 - reference to field toString can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:31:15 - reference to field toString can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:36:15 - reference to field toString can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:37:15 - call to get can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:37:15 - reference to field toString can't be resolved.
{:linter :unlimited-use,
 :msg
 "Unlimited use of (clojure.test clojure.core.incubator) in clojure.core.incubator-test",
 :line 11,
 :column 5}

== Warnings: 1 (not including reflection warnings)  Exceptions thrown: 0
Subprocess failed

Most of those reflection warnings also appear in the output of 'lein check' for that namespace, but 2 do not. They are:

Reflection warning, clojure/core/incubator_test.clj:21:39 - call to get can't be resolved.
Reflection warning, clojure/core/incubator_test.clj:37:15 - call to get can't be resolved.


 Comments   
Comment by Nicola Mometto [ 17/Jan/14 6:00 PM ]

This took me a lot to figure out.
It's not a tools.analyzer bug but a Clojure bug.

This will resolve the method at compile time:

(.method [..] ..)

This will not:

(.method ^{..} [..] ..)

In this case, lein check is not reporting any reflection warning because the only meta on [0] is :line/:column that gets stripped, while in eastwood [0] has :line/:column/:end-line/:end-column since it gets read by tools.reader.

Comment by Andy Fingerhut [ 17/Jan/14 6:43 PM ]

So to see if I understand correctly, ideally Clojure should resolve the method correctly even if there is metadata like in the (.method ^{..} [..] ..) case, as long as that metadata gives no :tag at all, or a correct :tag for the method resolved to?

Comment by Andy Fingerhut [ 17/Jan/14 6:44 PM ]

And if the answer to my question above is "yes", is it worth filing a ticket for the Clojure compiler, with any details you have learned about the source of the issue?

Comment by Nicola Mometto [ 17/Jan/14 7:08 PM ]

Yes.
I'll definitely open a ticket for this once I fully grasp what's going on.

Looks like tools.analyzer/emitter is, even though in less cases (not in this for example), affected by a similar bug.

Comment by Nicola Mometto [ 19/Jan/14 9:03 AM ]

Should be fixed for tools.analyzer, here's the ticket for clojure http://dev.clojure.org/jira/browse/CLJ-1326





[TANAL-58] analyze gives no error for arg vector [a & &] Created: 15/Jan/14  Updated: 16/Jan/14  Resolved: 16/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Discovered during some manual testing of which arg vectors gave errors:

(defn foo [a & &]
  (inc a))


 Comments   
Comment by Nicola Mometto [ 16/Jan/14 5:17 AM ]

Fixed https://github.com/clojure/tools.analyzer/commit/0be70788d42917c1cea9e99d2249623d6229c70c





[TANAL-57] Reflection warning from emit-form eval that Clojure compiler by itself does not Created: 15/Jan/14  Updated: 15/Jan/14  Resolved: 15/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

OK, this one I have verified is a difference even after 'lein clean' on the project. Latest version of Eastwood and t.a(.jvm), and of tools.emitter.jvm, where the extra Eastwood warning occurs. I added a :1.6 profile to the project.clj that uses latest 1.6.0-master-SNAPSHOT, but I think this difference exists even with Clojure 1.5.1, too.

lein with-profile 1.6 eastwood '{:namespaces [ clojure.tools.emitter.jvm.emit ] :debug #{:eval}}'

[ ... ]

Reflection warning, clojure/tools/emitter/jvm/emit.clj:39:29 - call to getMethod can't be resolved.


 Comments   
Comment by Nicola Mometto [ 15/Jan/14 4:12 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/f676e1d877d2c3885e52b29ce990045db80b98c3





[TANAL-55] Another difference in reflection warnings from eval'ing emit-form vs. plain Clojure Created: 15/Jan/14  Updated: 15/Jan/14  Resolved: 15/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I think the list of differences between reflection warnings issues by 'lein check' and Eastwood for projects in Eastwood's crucible is getting fairly short now, but there are still a few.

I have switched to doing my comparison with Clojure 1.6.0-master-SNAPSHOT after realizing that a few of the differences go away when using that, I think due to the change introduced by CLJ-1202.

Get latest 'reply' project (it is one pulled by Eastwood crucible), and edit its project.clj file to add these things:

:global-vars {*warn-on-reflection* true}

   :profiles {:dev {:dependencies ~dev-deps}
              :1.6 {:dependencies [[org.clojure/clojure "1.6.0-master-SNAPSHOT"]]

Then run the command below, assuming you have done 'mvn install' on latest Clojure 1.6.0-master-SNAPSHOT:

lein with-profile 1.6 eastwood '{:namespaces [reply.reader.jline.JlineInputReader] :debug #{:eval}}'

[ ... ]

Reflection warning, reply/reader/jline/JlineInputReader.clj:15:10 - reference to field state can't be resolved.
Reflection warning, reply/reader/jline/JlineInputReader.clj:18:18 - reference to field readLine can't be resolved.
Reflection warning, reply/reader/jline/JlineInputReader.clj:30:10 - reference to field state can't be resolved.

No reflection warnings appear for that namespace in the output of 'lein with-profile 1.6 check'



 Comments   
Comment by Nicola Mometto [ 15/Jan/14 5:09 AM ]

Honestly I have absolutely no idea how that code doesn't trigger a reflection warning in Clojure.
There must be something going on under the wood of gen-class that I'm not aware of, I'll need some time to take a look & understand.

Thanks for the report

Comment by Nicola Mometto [ 15/Jan/14 5:14 AM ]

Upon further investigation, I think that for some reason Clojure is not printing the reflection warning even though reflection is actually needed.

Empirical evidence: try to change a method call to .foo and no reflection warning will be printed.

Comment by Nicola Mometto [ 15/Jan/14 5:17 AM ]

Verified that if the (set! warn-on-reflection true) is put in the ns, I get the same reflection warnings from clojure.

I have no idea why it won't print when warn-on-reflection is set! outside the ns though.

Comment by Andy Fingerhut [ 15/Jan/14 10:14 AM ]

Weird, I was trying to reproduce this again to see why 'lein check' was not producing those warnings, but now it is. I can't think what might have changed since yesterday.

Comment by Andy Fingerhut [ 15/Jan/14 10:20 AM ]

Not sure, but the :gen-class may have something to do with it. If the class has already been generated, perhaps Leiningen is not recompiling that file. 'lein clean' first should make things more consistent.

Comment by Andy Fingerhut [ 15/Jan/14 10:22 AM ]

Confirmed. If you do 'lein clean' then 'lein check' twice with no changes to the source files, the warnings are produced the first time, but not the second. If you edit the source file, it is newer than the target class file and so gets recompiled.





[TANAL-56] Update versions in project.clj files Created: 15/Jan/14  Updated: 15/Jan/14  Resolved: 15/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

So they are 0.1.0-SNAPSHOT for both t.a and t.a.jvm.

Also nice to update the dependency on t.a.jvm in tools.emitter.jvm's project.clj



 Comments   
Comment by Nicola Mometto [ 15/Jan/14 4:58 AM ]

https://github.com/clojure/tools.analyzer/commit/ce23c47a243d663e04143f2ff98a4dd86b685f60
https://github.com/clojure/tools.analyzer.jvm/commit/64ecab702a3a666f0a14562f0fa4c5339267bd07
https://github.com/clojure/tools.emitter.jvm/commit/5f230c7ae5363445812c9c3a49e1142d029b2cc3





[TANAL-54] Reflection warnings from emitted forms, but type tags given on loop locals should prevent them? Created: 14/Jan/14  Updated: 14/Jan/14  Resolved: 14/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I'm guessing at the common denominator here. I am using latest Eastwood and tools.analyzer(.jvm) on latest core.async library code. The first two reflection warnings are normal, from Eastwood's use of core.memoize and data.priority-map, I believe. It is the 5 in clojure.core.async.impl.channels that are the concern here. None of them appear in the output of 'lein check' for core.async

% lein eastwood '{:namespaces [ clojure.core.async.impl.channels ]}'
Reflection warning, clojure/data/priority_map.clj:215:19 - call to equiv can't be resolved.
Reflection warning, clojure/core/memoize.clj:72:23 - reference to field cache can't be resolved.
== Eastwood 0.1.1-SNAPSHOT Clojure 1.5.1 JVM 1.7.0_15
== Linting clojure.core.async.impl.channels ==
Reflection warning, clojure/core/async/impl/channels.clj:190:15 - reference to field lock can't be resolved.
Reflection warning, clojure/core/async/impl/channels.clj:192:17 - reference to field unlock can't be resolved.
Reflection warning, clojure/core/async/impl/channels.clj:65:56 - reference to field lock can't be resolved.
Reflection warning, clojure/core/async/impl/channels.clj:66:40 - reference to field lock can't be resolved.
Reflection warning, clojure/core/async/impl/channels.clj:70:36 - reference to field unlock can't be resolved.


 Comments   
Comment by Nicola Mometto [ 14/Jan/14 9:01 PM ]

The issue here was that validate-loop-locals was trying to be "smarter" than clojure, by validating non-primitive values too, making it possible for

(loop [x (Integer. 1)] (if (number? x) (recur "foo") (.hashCode x))

to compile (Clojure throws an exception at runtime for this code)

However it looks like this "enhancements" breaks the assumption of that piece of core.async code that loop args will be automatically tagged, so this has been reverted and we now match Clojure behaviour.
https://github.com/clojure/tools.analyzer.jvm/commit/8ff4ff8df01f5d0638030bc78fc7b8cad712d272





[TANAL-53] Should type-hint exceptions inside catch expressions? Created: 14/Jan/14  Updated: 14/Jan/14  Resolved: 14/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Clojure 1.5.1 gives no reflection warning at all for the following source file, but eval'ing the result of emit-form gives this warning:

Reflection warning, reflection/core4.clj:11:16 - reference to field getMessage can't be resolved.

Here is the code:

(ns reflection.core4)

(set! *warn-on-reflection* true)

;; copied & trimmed down from namespace reply.main

(defn foo [arg]
  (try
    (read-string arg)
    (catch Exception e
      (println (.getMessage e)))))


 Comments   
Comment by Nicola Mometto [ 14/Jan/14 3:55 PM ]

Well this one is weird.
Looks like if the classname to a catch is a Class, the local will not get type hinted but the catch will work fine anyway.

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/c79c5f8e8cb96a323360fb75c86bf10c41b83082





[TANAL-52] More tags missing from emit-form result Created: 14/Jan/14  Updated: 14/Jan/14  Resolved: 14/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

If you don't mind, I will keep putting up one of these every time I find one, but keep at most one pending at a time. The main reason for me doing one-at-a-time is I am not taking the time to determine which of these are due to the same root cause, and which are not, so the one-at-a-time method should reduce duplication of reports.

In ogre project, part of Eastwood crucible, the following command with latest tools.analyzer(.jvm) and Eastwood produces a reflection warning from Eastwood, but not from the Clojure compiler (using 'lein check'):

lein eastwood '{:namespaces [ ogre.filter ] :debug #{:eval}}'

[ ... most output deleted ... ]
Reflection warning, ogre/filter.clj:27:12 - call to interval can't be resolved.


 Comments   
Comment by Nicola Mometto [ 14/Jan/14 11:45 AM ]

I don't mind at all, keep'em coming!
Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/6371e4384ae0b6948ca1243f03690a3a9be9f669





[TANAL-51] Another case where emit-form seems to omit a type tag leading to reflection warning Created: 13/Jan/14  Updated: 14/Jan/14  Resolved: 13/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

To reproduce, go to latest version of core.async source tree and with latest tools.analyzer(.jvm) and latest Eastwood 0.1.1-SNAPSHOT:

lein eastwood '{:namespaces [ clojure.core.async.impl.timers ] :debug #{:eval}}'

I get exactly one reflection warning, but none appear when compiling the code normally:

Reflection warning, clojure/core/async/impl/timers.clj:1:251 - reference to field channel can't be resolved.

It is inside defn timeout



 Comments   
Comment by Nicola Mometto [ 13/Jan/14 9:19 PM ]

Should be fixed with https://github.com/clojure/tools.analyzer.jvm/commit/bb4c9fca086b3cbeba382df7435990f493148869 and https://github.com/clojure/tools.analyzer/commit/fe76b18248a336566965ff9b235e97af65f1f1f1 but it's possible there are still some edge-cases missing.

Please open a new ticket if you find any, thanks

Comment by Andy Fingerhut [ 13/Jan/14 9:43 PM ]

That change to tools.analyzer causes it to fail one of its unit tests run by 'lein test'

Comment by Nicola Mometto [ 13/Jan/14 9:50 PM ]

Sorry about that. I was relying on hudson to check whether or not it break anything but the commit didn't trigger a new build and I didn't notice I was looking at an old build.

Should be fixed now.

Comment by Andy Fingerhut [ 13/Jan/14 10:48 PM ]

Hmm. Fails more unit tests of 'lein test' now.

Comment by Nicola Mometto [ 14/Jan/14 5:20 AM ]

That was intended, I finished that minor refactor with https://github.com/clojure/tools.analyzer/commit/d52e8400b460c1475646f084d53f948d7fae5b34
Tests are now passing again.





[TANAL-50] emit-form completely omits some metadata that causes reflection warning if eval'd Created: 13/Jan/14  Updated: 13/Jan/14  Resolved: 13/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

To reproduce, do 'lein new reflection' and replace contents of src/reflection/core.clj with the following:

(ns reflection.core
  (:import [java.net URL]))

(set! *warn-on-reflection* true)

;; copied from namespace clojurewerkz.neocons.rest.helpers

(defn extract-id
  [^String location]
  (let [url (URL. location)]
    (Long/valueOf ^String (first (re-seq #"\d+$" (.getPath url))))))

Update to the latest Eastwood master, install, make sure eastwood version 0.1.1-SNAPSHOT is in your ~/.lein/profiles.clj, and run this command:

lein eastwood '{:debug #{:eval}}'

It should show each form that will be eval'd, starting with the not-analyzed ns form, followed by the 2 other forms that are each analyzed and are the result of emit-form, shown with metadata.

When I do that, I see the arg to Long/valueOf with no metadata at all (no String tag), which causes eval in Eastwood to generate a reflection warning. There is no reflection warning with 'lein check'.



 Comments   
Comment by Nicola Mometto [ 13/Jan/14 5:05 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/d9e9cd2e385a5f1354a8c5fbd26826e04ae549e4





[TANAL-49] emit-form missing some metadata, leading to reflection warnings when eval'd? Created: 12/Jan/14  Updated: 12/Jan/14  Resolved: 12/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I will keep investigating to see if this problem exists only in Eastwood's use of tools.analyzer(.jvm), but here are the symptoms. Use 'lein new reflection' to create a new Leiningen project, and edit the file src/reflection/core.clj to contain:

(ns reflection.core)

(set! *warn-on-reflection* true)

(defn foo [s]
  (doseq [i s]
    (println i)))

Using Eastwood 0.1.0, 'lein eastwood' gives me a reflection warning like the one below, but only about half the time that I run it. The other half of the time there is no reflection warning.

Reflection warning, reflection/core.clj:1:218 - call to nth can't be resolved.


 Comments   
Comment by Nicola Mometto [ 12/Jan/14 12:53 PM ]

Fixed with: https://github.com/clojure/tools.analyzer.jvm/commit/7de63ad3f81ed5a60ba0b3031383d1d63db2b51f

Still need to understand if the non-deterministic behaviour of this bug is relevant.

Comment by Andy Fingerhut [ 12/Jan/14 9:11 PM ]

Perhaps your later commits to tools.analyzer(.jvm) indicate areas of potential nondeterminism that you were trying to make deterministic?

Later testing did reveal that in the cases I was getting the reflection warnings, the tag was of type java.lang.Class, and when the tag was of type clojure.lang.Symbol there was no reflection warning. This is consistent with Compiler.java's tagOf() method ignoring any tags except those with class clojure.lang.Symbol or java.lang.String.

I did not determine why the results were nondeterministic.

Comment by Nicola Mometto [ 12/Jan/14 9:32 PM ]

No, my later commits are solely for performance enhancements, I was not able to reproduce the nondeterministic results at the repl unfortunately, so I'm unable to figure that out.





[TANAL-48] tools.analyzer fails Maven build with Jan 11 2014 1.6.0-master-SNAPSHOT Created: 11/Jan/14  Updated: 11/Jan/14  Resolved: 11/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

This appears to be due to the addition of clojure.core/record? which leads to the following warnings when building tools.analyzer with 1.6.0-master-SNAPSHOT:

% lein with-profile 1.6 do clean, test
WARNING: record? already refers to: #'clojure.core/record? in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/record?
WARNING: record? already refers to: #'clojure.core/record? in namespace: clojure.tools.analyzer, being replaced by: #'clojure.tools.analyzer.utils/record?

These cause the Maven build to fail.

Similarly for tools.analyzer.jvm



 Comments   
Comment by Nicola Mometto [ 11/Jan/14 8:04 PM ]

Fixed:
https://github.com/clojure/tools.analyzer/commit/89d98b688448fe3d02bcabea240bcb2f6d2bbd70
https://github.com/clojure/tools.analyzer.jvm/commit/e8510c21f0f1e0f8af4008b9b253783c15d9cac8





[TANAL-47] t.a.j recent commit causes a particular source file to no longer throw exception for non-qualified type tag Created: 08/Jan/14  Updated: 08/Jan/14  Resolved: 08/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File exception-no.sh    

 Description   

For all I know, this might be intended behavior, but it was a change in the output of Eastwood that looked suspicious, so wanted to report it while the changes were fresh in your mind.

The attached bash shell script, after a bit of editing for the location of github repo clones on your local file system, should run and cause an exception to be reported during analysis of pantomime project namespace pantomime.media.

If you then change the line 'git checkout ${TAJ2}' to contain TAJ3 instead of TAJ2 and run it again, there is no exception for the same pantomime source file.



 Comments   
Comment by Nicola Mometto [ 08/Jan/14 10:32 AM ]

Not intentional, fixed: https://github.com/clojure/tools.analyzer.jvm/commit/de666b9203bdc73ccc1c557ff2c4c3a733803d9e

Thanks.





[TANAL-46] Case tag handling is broken Created: 08/Jan/14  Updated: 08/Jan/14  Resolved: 08/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by Nicola Mometto [ 08/Jan/14 6:42 AM ]

https://github.com/clojure/tools.analyzer.jvm/commit/06646aa023c8db6c8e61640072d07cc648c73543





[TANAL-45] NPE in validate-loop-locals after recent changes Created: 07/Jan/14  Updated: 07/Jan/14  Resolved: 07/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

This NullPointerException is new after the commits made to tools.analyzer(.jvm) in the last day or two, and only for some namespaces in the Eastwood crucible. A few examples are given below, with latest checked in Eastwood. The first one causes the exception earliest in the source file, so might be easier to debug than the other two, which occur several hundred lines into the source file.

% lein eastwood '{:namespaces [ clojurewerkz.cassaforte.conversion ]}'
% lein eastwood '{:namespaces [ criterium.core ]}'
% lein eastwood '{:namespaces [ net.cgrand.enlive-html ]}'


 Comments   
Comment by Nicola Mometto [ 07/Jan/14 1:06 PM ]

https://github.com/clojure/tools.analyzer.jvm/commit/4f67b8c70a3754796bde91becabbb0947503f6dd





[TANAL-44] potemkin has unusual macro invocation that causes tools.analyzer.jvm to throw exception Created: 06/Jan/14  Updated: 06/Jan/14  Resolved: 06/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I am not sure whether tools.analyzer(.jvm) could/should analyze this kind of code successfully or not.

Steps to reproduce, using latest tools.analyzer(.jvm) and Eastwood as of Jan 6 2014 12:05am PST:

% git clone https://github.com/ztellman/potemkin.git
% cd potemkin

# The following gives no exception while analyzing namespace potemkin
% lein eastwood '{:namespaces [ potemkin ]}'

# The following does throw an exception while analyzing namespace potemkin
% lein eastwood '{:namespaces [ potemkin.namespaces potemkin ]}'


 Comments   
Comment by Nicola Mometto [ 06/Jan/14 9:34 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/6227561ceb1152e9984b96e60b4735e1deb1db42

Comment by Andy Fingerhut [ 06/Jan/14 10:47 AM ]

With this change, I now get an exception on many namespaces I did not before. I believe that the common denominator is that whenever a function calls itself, there is an exception similar to the following, where this particular example is from project stencil's namespace stencil.utils:

% git clone https://github.com/davidsantiago/stencil.git
% cd stencil
% lein eastwood
== Linting stencil.utils ==
Exception thrown during phase :analyze of linting namespace stencil.utils
UnsupportedOperationException count not supported on this type: Symbol
	clojure.lang.RT.countFrom (RT.java:556)
	clojure.lang.RT.count (RT.java:530)
	clojure.core/count (core.clj:832)
	clojure.core/sort-by/fn--4299 (core.clj:2769)
	clojure.lang.AFunction.compare (AFunction.java:49)
	java.util.TimSort.countRunAndMakeAscending (TimSort.java:324)
	java.util.TimSort.sort (TimSort.java:189)
	java.util.TimSort.sort (TimSort.java:173)
	java.util.Arrays.sort (Arrays.java:659)
	clojure.core/sort (core.clj:2754)
	clojure.core/sort-by (core.clj:2769)
	clojure.core/sort-by (core.clj:2767)
	clojure.tools.analyzer.utils/arglist-for-arity (utils.clj:87)
	clojure.tools.analyzer.passes.jvm.infer-tag/eval1970/fn--1972 (infer_tag.clj:221)
Comment by Nicola Mometto [ 06/Jan/14 11:08 AM ]

I should make a habit of running the eastwood crucible before closing a ticket.
Should be fixed with https://github.com/clojure/tools.analyzer.jvm/commit/339abfa780fdef0de96100eedd0fbe3101dc26ad

Comment by Andy Fingerhut [ 06/Jan/14 11:57 AM ]

Looking much better after the second fix. You are welcome to use the eastwood crucible for testing, but if you don't, I will The whole list of projects takes over 30 mins on a reasonably recent laptop, and does not give pass/fail results, but a lot of output. I typically only notice things when the saved output of a previous lint.sh run is significantly different than the latest one.





[TANAL-29] move the type-coercion logic from the emitter to a pass in the analyzer Created: 14/Dec/13  Updated: 05/Jan/14  Resolved: 05/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None





[TANAL-43] Suggestion: Throw exception for (var ...) forms unless they have exactly 1 arg Created: 03/Jan/14  Updated: 04/Jan/14  Resolved: 04/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Clojure silently allows such expressions, too, but as far as I can see it ignores all args other than the first, similarly to throw and quote.



 Comments   
Comment by Nicola Mometto [ 04/Jan/14 1:02 PM ]

Fixed also for import*, monitor-enter, monitor-exit: https://github.com/clojure/tools.analyzer.jvm/commit/9eb1dd8dfd8d8058897f446551b1aa17cbd14f46





[TANAL-41] throw called with 2 args Created: 03/Jan/14  Updated: 03/Jan/14  Resolved: 03/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-41-v1.diff    
Patch: Code

 Description   

Clojure silently ignores args to throw after the 1st, but tools.analyzer throws an exception for them. Yes, we are going meta again, in that Eastwood caught this. See the proposed patch.



 Comments   
Comment by Nicola Mometto [ 03/Jan/14 6:42 PM ]

Awesome.
Fixed: https://github.com/clojure/tools.analyzer/commit/796a89277659cb722c028d9036619f46d41ecd83





[TANAL-40] The :loop-locals keys are not updated in the uniquify-locals pass Created: 03/Jan/14  Updated: 03/Jan/14  Resolved: 03/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Jonas Enlund Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File uniquify-locals.diff    

 Description   

Currently, the :loop-locals keys in :env are not updated to the new names in the uniquify-locals pass leaving the environment inconsistent with the ast.



 Comments   
Comment by Nicola Mometto [ 03/Jan/14 3:40 PM ]

Merged: https://github.com/clojure/tools.analyzer/commit/b06fd1a17c1e8b0b0fbb108aeb63fa0623bddfbe





[TANAL-39] Add :ast key back into exception data for validate-tag exceptions Created: 02/Jan/14  Updated: 02/Jan/14  Resolved: 02/Jan/14

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-39-v1.diff    
Patch: Code

 Description   

The ast in these exceptions is used by Eastwood to provide the user with some additional context for where the bad tags occur.



 Comments   
Comment by Andy Fingerhut [ 02/Jan/14 6:09 PM ]

Patch tanal-39-v1.diff is a simple one-liner.

Comment by Nicola Mometto [ 02/Jan/14 7:13 PM ]

https://github.com/clojure/tools.analyzer.jvm/commit/1b5211bf47a9cdbca8013bb1eb8c38901206063b





[TANAL-38] tools.analyzer(.jvm) throws "Wrong number of args" exception for (= x y) Created: 31/Dec/13  Updated: 31/Dec/13  Resolved: 31/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

With the latest tools.analyzer(.jvm) and Eastwood as of Dec 30 2013 10:45PM PST, a simple namespace like the following throws an exception when attempting to analyze it:

(ns eastwood.test.testcases.f10)

(defn foo [x y]
  (= x y))

Here are some details from the exception thrown:

Exception thrown during phase :analyze of linting namespace eastwood.test.testcases.f10
Got exception with extra ex-data:
    msg='Wrong number of args to ., had: 4'
    (keys dat)=(:file :line :column :form)
    (:form dat)=
(.
 clojure.lang.Util
 clojure.core/equiv
 ^{:line 4, :column 6, :end-line 4, :end-column 7} x
 ^{:line 4, :column 8, :end-line 4, :end-column 9} y)
ExceptionInfo Wrong number of args to ., had: 4
        clojure.core/ex-info (core.clj:4327)
        clojure.tools.analyzer/eval1729/fn--1731 (analyzer.clj:690)
        clojure.lang.MultiFn.invoke (MultiFn.java:231)
        clojure.tools.analyzer.jvm/eval3179/fn--3180 (jvm.clj:51)
        clojure.lang.MultiFn.invoke (MultiFn.java:231)
        clojure.tools.analyzer/eval1570/fn--1571 (analyzer.clj:227)
        clojure.lang.MultiFn.invoke (MultiFn.java:236)
[ ... rest of stack trace deleted ... ]


 Comments   
Comment by Nicola Mometto [ 31/Dec/13 8:58 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/bb3a6bd0ae2541591eb8bf9b7686bf79aca1b143#diff-ca9fdc7850490879310292f81bfbb68fL687





[TANAL-37] Tests written using simple-check might be problematic for tools.analyzer Created: 24/Dec/13  Updated: 25/Dec/13  Resolved: 25/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

Discovered when trying to use the latest version of tools.analyzer(.jvm) and Eastwood on the automat library: https://github.com/ztellman/automat

I do not yet know whether this will be true of any project that uses simple-check for tests. I will add more to this description if I find other examples.

Below is the error message I see when running 'lein eastwood' in automat's project root dir:

== Linting automat.core-simple-check ==
Exception thrown during phase :analyze of linting namespace automat.core-simple-chec
k
Got exception with extra ex-data:
    msg='No such namespace: simple-check.core'
    (keys dat)=(:ns)
ExceptionInfo No such namespace: simple-check.core
        clojure.core/ex-info (core.clj:4327)
        clojure.tools.analyzer.passes.jvm.validate/eval1616/fn--1618 (validate.clj:3
3)

automat requires the namespaces simple-check.clojure-test, which defines a macro defspec that contains (require 'simple-check.core) here: https://github.com/reiddraper/simple-check/blob/master/src/simple_check/clojure_test.clj#L28

along with a comment mentioning that this is a cyclic dependency.

This may be beyond what is reasonable to expect tools.analyzer(.jvm) to be able to handle, but wanted to get the example to you in case not.



 Comments   
Comment by Nicola Mometto [ 25/Dec/13 8:54 AM ]

It looks like it's an instance of the Gilardi Scenario.

There's nothing that can be done to fix this in the analyzer as a "fix" for this requires breaking up do expression and evaluating them piece by piece instead of analyzing the entire do form and evaluating it all in a single step.

This is how tools.emitter.jvm does it: https://github.com/clojure/tools.emitter.jvm/blob/master/src/main/clojure/clojure/tools/emitter/jvm.clj#L19-L26

Comment by Andy Fingerhut [ 25/Dec/13 9:39 AM ]

So a question: Does such a construct become illegal in a future CinC implementation? I'm not arguing that it should become illegal, or should not, but if it should, it might be a good idea to mention it to the simple-check authors, and/or more Clojure devs.

Comment by Nicola Mometto [ 25/Dec/13 10:15 AM ]

I might have explained myself badly here.
tools.analyzer's behaviour in this case is exatly the same as clojure's one, and indeed clojure will fail on that expression too if all you do is call clojure.lang.Compiler/analyze on it.

clojure can eval that expression because it will analyze & eval all the do forms separately – this is what tools.emitter.jvm does too.

if we want to analyze this expression in eastwood we need to manually analyze do forms separately too.





[TANAL-36] Can this code avoid reflection and be analyzed by tools.analyzer? Created: 24/Dec/13  Updated: 24/Dec/13  Resolved: 24/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

This is probably another variant of TANAL-24, but I wanted to get the example to you for two reasons: (1) for me to learn how to type-hint this case correctly, if there is a way and (2) in case there was something new in this example that you haven't already seen.

(ns eastwood.test.testcases.f09
  (:use clojure.test)
  (:import [java.io BufferedOutputStream])
  (:require [clojure.java.io :as io]))

(set! *warn-on-reflection* true)

(defmacro ^:private bytestring
  [s] `(.getBytes ~s "UTF-8"))

(def ^bytes bs-crlf (bytestring "\r\n"))

(defmacro ^:private send-crlf [out] `(.write ~out bs-crlf 0 2))

(defn send-request [out]
  (send-crlf out))

As the subject says, I would like to know your thoughts on whether there is a way to type-hint this code such that the .write method invocation does not use reflection, and tools.analyzer(.jvm) can analyze it with no exception.

This test case was boiled down from some code in the ClojureWerkz Carmine library.



 Comments   
Comment by Nicola Mometto [ 24/Dec/13 2:50 PM ]

This cannot be analyzed because of the def :tag bug I talked you about, (def ^bytes ..) will attach the clojure.core/bytes function in the :tag instead of 'bytes.

A way to fix this is to change it to (def ^{:tag 'bytes} ..)
The same holds for every primitive type hint in def.

To avoid the reflection warning ^BufferedOutputStream out in either send-request or send-crlf should work





[TANAL-35] Two unit tests with incorrect parentheses Created: 23/Dec/13  Updated: 23/Dec/13  Resolved: 23/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-35-v1.diff    

 Description   

These two unit tests are of the form (is (= expr1) expr2), which always passes without testing anything because (= expr1) is true.

(is (= '(. b (a c))) (emit-form (ast (.a b c))))
  (is (= '(. b (a c))) (emit-form (ast (.a b (c)))))

Found using the newest Eastwood linter, :suspicious-test



 Comments   
Comment by Andy Fingerhut [ 23/Dec/13 6:26 PM ]

Patch tanal-35-v1.diff not only fixes the parenthesization, but also changes the second test to add parens around (c) in the expected value. Please verify that is correct before applying the patch.

Comment by Nicola Mometto [ 23/Dec/13 6:29 PM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/50c862a02662882f7fc2351659ea67753a5f49ea

We're going meta on this, thanks





[TANAL-31] analyzing extend expression with non-constant type/class throws exception Created: 15/Dec/13  Updated: 20/Dec/13  Resolved: 20/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Here is an example that is slightly cut down from code in namespace seesaw.widgets.log-window, where I first saw the issue. It could be cut down a little more. The main point is that the first arg to extend-type is not a constant, but a fn call that returns a class:

(defn- log-window-proxy [state]
  (proxy [javax.swing.JTextArea clojure.lang.IDeref] []
    ; Implement IDeref
    (deref [] state) 

    ; this is how you disable auto-scrolling :(
    (scrollRectToVisible [rect]
                         (if @(:auto-scroll? state)
                           (proxy-super scrollRectToVisible rect)))))

(defprotocol LogWindow
  (log   [this message])
  (clear [this]))

(extend-type (class (log-window-proxy nil))
  LogWindow
  (log [this message]
    (println "message" message))
  (clear [this] 
    nil))


 Comments   
Comment by Nicola Mometto [ 15/Dec/13 4:00 PM ]

Yet another #TANAL-24 related bug.

(extend-type (class 1) proto (protof [this]))

extends to

(extend (class 1) proto {:protof (fn [^{:tag '(class 1)} this])})

And clojure swallows that silently.
It's not clear to me whether extend-type is supposed to work that way, but since the docstring says explicitely "Propagates the class as a type hint on the first argument of all fns." and this is clearly not true, I will consider this as a Clojure bug and fill a ticket.

I'll keep this ticket open too until #TANAL-24 gets resolved.

Comment by Nicola Mometto [ 15/Dec/13 4:12 PM ]

Clojure ticket: http://dev.clojure.org/jira/browse/CLJ-1308





[TANAL-30] Analyzing a function call with type-hinted ret value, where ret type is a class not imported in calling namespace, throws exception Created: 15/Dec/13  Updated: 20/Dec/13  Resolved: 20/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

Shortest example I can find requires a Leiningen project with 2 namespaces.

File project.clj:

(defproject temp "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.5.1"]])

File src/temp/util.clj:

(ns temp.util
  (:import [java.awt Cursor]))

(defn foo ^Cursor []
  (Cursor. Cursor/HAND_CURSOR))

File src/temp/core.clj:

(ns temp.core
  (:require [temp.util :as u]))

(defn foo [x]
  [x (u/foo)])

Eastwood throws exception when analyzing namespace temp.core because of the call to (u/foo). I haven't tried to reproduce it with only calls to tools.analyzer functions.

This is with the latest versions of all code checked in as of the time this ticket was created.



 Comments   
Comment by Nicola Mometto [ 15/Dec/13 8:20 AM ]

This is strongly related to #TANAL-24 in that it works in clojure simply because Clojure discards the tag.
It actually doesn't really work in clojure when the :tag info is required, try:

(defn foo [] (.hashCode (u/foo)))

It'll throw the ClassNotFoundException too.

I'll let this ticket open until we close #TANAL-24, and will open a Clojure ticket fixing this behaviour since this is clearly a Clojure bug.

Thanks

Comment by Andy Fingerhut [ 15/Dec/13 11:11 AM ]

Understood. For future reference, below is a list of namespaces where tools.analyzer throws an exception of this kind, with a very similar stack trace going through the following function call:

clojure.tools.analyzer.passes.jvm.emit-form/emit-form

I have not verified if the root cause in the source code is the same for all of them. The short example in the description was cut down from a case in the Seesaw library.

seesaw.widgets.log-window
seesaw.test.core
seesaw.test.examples.hotpotatoes
clojurewerkz.titanium.query
clojurewerkz.titanium.edges-test
clojurewerkz.titanium.element-test
clojurewerkz.titanium.graph-test
clojurewerkz.titanium.integration-test
clojurewerkz.titanium.types-test
clojurewerkz.titanium.vertices-test
flatland.useful.datatypes-test
flatland.useful.experimental-test
cljs.core.async.impl.ioc-macros
cljs.core.async.macros
clojure.core.async-test
clojure.core.match.test.core
clojure.core.match.test.date
clojure.core.match.test.java
clojure.data.codec.test-base64
clojure.data.fressian-test
clojure.java.test-jdbc
clojure.test.array-test
clojure.test.core-test
clojure.test.math-test
clojure.tools.namespace.dependency

Comment by Nicola Mometto [ 15/Dec/13 4:06 PM ]

clojure issue: http://dev.clojure.org/jira/browse/CLJ-1307

Comment by Nicola Mometto [ 20/Dec/13 9:42 AM ]

I'm closing this since it's caused by a Clojure bug and on the tools.analyzer side it's related to #TANAL-24





[TANAL-34] Throw more informative exception when parameter list contains namespace-qualified symbols Created: 20/Dec/13  Updated: 20/Dec/13  Resolved: 20/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-34-v1.diff    

 Description   

Having more details in this exception would have helped in more quickly discovering the mistake in a Clojure source file being analyzed by Eastwood, where someone had a macro that used k# in most places for a parameter, but left out the # in one instance of it.



 Comments   
Comment by Nicola Mometto [ 20/Dec/13 9:33 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/31a58c280a5777acab5528ec48e10eff6ae007fa





[TANAL-33] Class/method call in macro in ns1 causes 'No such namespace' when used in ns2 that does not import the class Created: 19/Dec/13  Updated: 20/Dec/13  Resolved: 20/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I am hoping that this one is not yet another derivative of TANAL-24

I tried to create a minimal test case but it didn't work. The quickest example I can give is to pull a clone of the quartzite library and use the latest tools.analyzer(.jvm) and Eastwood to analyze it. Several namespaces throw the exception during analysis, including clojurewerkz.quartzite.test.execution-test

% git clone https://github.com/michaelklishin/quartzite.git
% cd quartzite
% lein eastwood

[ ... ]

Exception thrown during phase :analyze of linting namespace clojurewerkz.quartzite.test.execution-test
Got exception with extra ex-data:
    msg='No such namespace: JobBuilder'
    (keys dat)=(:ns)
ExceptionInfo No such namespace: JobBuilder
	clojure.core/ex-info (core.clj:4327)
	clojure.tools.analyzer.passes.jvm.validate/eval1613/fn--1615 (validate.clj:33)

The basic problem seems to be that one namespace defines a macro that invokes (Class/method args), without a package name qualifier, but imports the class so it does not need one.

A second namespace invokes the macro, but does not import the class.

However, a minimal test case based on that pattern gave no error while using Eastwood on it, so there may be more conditions required than that for the exception to occur.



 Comments   
Comment by Nicola Mometto [ 20/Dec/13 8:37 AM ]

Oddly enough this was a.. tools.reader bug
I'm pushing a new release that fixes the typo right now.





[TANAL-32] Add more ex-data to exceptions thrown when validate-tag cannot find class Created: 19/Dec/13  Updated: 19/Dec/13  Resolved: 19/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-32-v1.diff    

 Description   

This is useful for lint tools like Eastwood, or any others that wish to provide more detailed error messages that help a developer learn the reason that their code cannot be analyzed.



 Comments   
Comment by Andy Fingerhut [ 19/Dec/13 4:21 PM ]

Patch tanal-32-v1.diff is one way to achieve this. The ast node containing the failed-to-validate tag is the most useful. The :tag-kind can help the caller a bit in distinguishing the cause of the exception.

Comment by Nicola Mometto [ 19/Dec/13 4:53 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/5c87ffaec5e317deaa8457d2f6de4a321f8c226e

Thanks





[TANAL-9] AbstractMethodError exception when trying to get metadata on literal record in code Created: 28/Nov/13  Updated: 16/Dec/13  Resolved: 11/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I do not know if this is a problem with the analyzer, or something in Clojure itself, because attempting to pprint the read-in form with the problem record literal also causes it to throw an AbstractMethodError.

(ns tryanalyzer.try13)

(defrecord RecordTest [a b])

(defn mytest []
  (= (RecordTest. 1 2)
     #tryanalyzer.try13.RecordTest{:a 1, :b 2}))


 Comments   
Comment by Nicola Mometto [ 28/Nov/13 11:14 AM ]

I'm sorry, I don't understand what's throwing here.
This works fine for me

clojure.tools.analyzer.jvm> (defrecord RecordTest [a b])
clojure.tools.analyzer.jvm.RecordTest
clojure.tools.analyzer.jvm> (analyze ^:foo #clojure.tools.analyzer.jvm.RecordTest{} (empty-env))
{:tag clojure.tools.analyzer.jvm.RecordTest, :children [:meta], :meta {:tag clojure.lang.IPersistentMap, :op :const, :env {:context :expr, :ns clojure.tools.analyzer.jvm}, :type :map, :literal? т, :form {:foo т}}, :op :const, :env {:context :expr, :ns clojure.tools.analyzer.jvm}, :type :record, :literal? т, :form #clojure.tools.analyzer.jvm.RecordTest{:a ∅, :b ∅}}
Comment by Nicola Mometto [ 29/Nov/13 11:01 AM ]

Ok I understood what you mean.

This is not a bug, you need to evaluate the defrecord in order to be able to create an instance in read-time.

This is IIRC different than what Compiler.java does, as it compiled the record at analysis time instead of compile time, but it's by design that tools.analyzer/emitter separate evaluation from analysis..

Comment by Andy Fingerhut [ 29/Nov/13 5:02 PM ]

Nicola, I am pretty sure that I am getting this error even though I am evaluating the defrecord before creating the instance.

Steps to reproduce: Install latest versions of tools.analyzer and tools.analyzer.jvm in your local Maven repo with 'lein install'. Then do the following steps to install locally the latest version of Eastwood I am testing with and seeing the issue:

git clone git://github.com/jafingerhut/eastwood.git
cd eastwood
git checkout try-upgrade-to-tools.analyzer.jvm
LEIN_SNAPSHOTS_IN_RELEASE=1 lein install

Then inside of a project that contains the namespace tryanalyzer.try13 in the description, perhaps renamed to whatever namespace you like, run this command:

lein eastwood ':namespaces [tryanalyzer.try13]}'

I get the AbstractMethodError thrown. Here are some function calls in the call stack, down to the part where I am calling eval on each form after analyzing, before reading and analyzing the next form in the file:

eastwood.core/run-eastwood

eastwood.core/lint-ns

eastwood.analyze-ns/analyze-ns

eastwood.analyze-ns/analyze-file - This was copied from jvm.tools.analyzer's function analyze-file, and then modified significantly. It is the one that calls Clojure's eval function after calling eastwood.jvm/analyze (below) on each form read from the file. It is the most likely place I would guess for there to be mistakes in the way I am calling analyze, in particular with its current choice of argument for the env.

eastwood.jvm/analyze - The comment at beginning of file indicates which file from tools.analyzer.jvm this is nearly an exact copy of.

Comment by Nicola Mometto [ 29/Nov/13 6:32 PM ]

I've investigated a bit on this after being able to reproduce, it looks like it's a classloader issue but it's not clear to me whether I'm doing something I'm not supposed to do or if it's a Clojure bug.

It should be noted that the issue is not there when trying on the repl.

I'll try to find a solution but my understanding of how clojure's dynamic classloader works is really basic so this might take me a while

Comment by Nicola Mometto [ 11/Dec/13 10:13 AM ]

OK, I pushed a fix: https://github.com/clojure/tools.analyzer.jvm/commit/feecc5be7e536ca6408cd7a2a48ac178cc7e25b9

I don't like this fix at all, I have a better fix for this in my local eastwood branch but unfortunately it's not reliable (it doesn't work for e.g. (do (deftype x []) 1))

This'll have to do.

Comment by Andy Fingerhut [ 16/Dec/13 8:08 AM ]

I think this issue is in the latest release of tools.analyzer(.jvm) again. Worth reopening?

Comment by Nicola Mometto [ 16/Dec/13 8:27 AM ]

Not the same issue, we've been hitten by this http://dev.clojure.org/jira/browse/CLJ-1105 bug.

Fixed for <=clojure-1.6.0-alpha3 with https://github.com/clojure/tools.analyzer/commit/4981916afb44dcdcc9d6866f46cb55287400213b





[TANAL-2] Remove assumption about fn as top-level form from collect pass Created: 18/Nov/13  Updated: 12/Dec/13  Resolved: 12/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Currently the collect pass assumes that every expression will be wrapped in a top-level fn, this is a JVM-specific assumption and should not be the case.



 Comments   
Comment by Nicola Mometto [ 12/Dec/13 4:54 PM ]

This was addressed by https://github.com/clojure/tools.analyzer/commit/ace0bf3d38f31c6cf2b531b2d439d74218db3d3e





[TANAL-28] validate-loop-locals introduces exponential slowdown Created: 09/Dec/13  Updated: 10/Dec/13  Resolved: 09/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by Nicola Mometto [ 09/Dec/13 8:15 PM ]

https://github.com/clojure/tools.analyzer.jvm/commit/5301a03e78c91fb43b282c03af07e3b40b4c5c4f
and
https://github.com/clojure/tools.analyzer.jvm/commit/956ad56fc0d54270931beb54e8cbe5acd3567fe9

should have removed unnecessary iterations.

Comment by Andy Fingerhut [ 10/Dec/13 12:17 AM ]

One set of measurements for comparison:

I measured the total time it took Eastwood to analyze its own source files (analysis only, not counting the time to run the linters), using tools.analyzer(.jvm), before these recent commits, and after.

Before, the total analysis time was 4.6 minutes. After, it was 1.0 minutes.

Thanks!





[TANAL-27] No matching arity exception thrown when attempting to call fn with varargs Created: 08/Dec/13  Updated: 08/Dec/13  Resolved: 08/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Test case to reproduce:

(require '[clojure.tools.analyzer.jvm :as aj])
(def env (aj/empty-env))
(def form (read-string "
(defn foo [a]
  (let [err (fn [& msg] (apply str msg))]
      (err \"Invalid arg 'a'\" a)))
"))
(def an (aj/analyze form env))
;; Exception thrown by previous line


 Comments   
Comment by Nicola Mometto [ 08/Dec/13 6:38 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/54a5ba2cb58f41399900f82376e732a49e4d7662





[TANAL-26] Var not found when using 'user' as alias Created: 06/Dec/13  Updated: 06/Dec/13  Resolved: 06/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jonas Enlund Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

The following snippet of code throws "clojure.lang.ExceptionInfo: no such var: user/union"

(require '[clojure.set :as user]
         '[clojure.tools.analyzer.jvm :as jvm])

(jvm/analyze 'user/union (jvm/empty-env))

If I change the alias to something else it works as expected.



 Comments   
Comment by Nicola Mometto [ 06/Dec/13 7:37 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/cf2b0e4fa19e1ecd42d33261859348850b9f31ec





[TANAL-25] Exception for no matching method with clojure.main/repl-caught, but no exception with alias m/repl-caught Created: 04/Dec/13  Updated: 05/Dec/13  Resolved: 05/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

This does not throw any exception:

(require '[clojure.tools.analyzer.jvm :as aj])
(require '[clojure.main :as m])
(def env (aj/empty-env))
(def form (read-string "
(defn foo [e]
  (m/repl-caught e))
"))
(def an (aj/analyze form env))

This does:

(require '[clojure.tools.analyzer.jvm :as aj])
(require 'clojure.main)
(def env (aj/empty-env))
(def form (read-string "
(defn foo [e]
  (clojure.main/repl-caught e))
"))
(def an (aj/analyze form env))

The only difference is the require line for clojure.main, and the alias m or the full namespace clojure.main for repl-caught.



 Comments   
Comment by Nicola Mometto [ 05/Dec/13 8:23 AM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/7008ae75f310dfc1a6446ae1941fde088d2bb36b

Looks like if given foo.bar/baz, if foo.bar is both a class and a namespace (that is the case for clojure.main) clojure only considers it as a namespace.

Comment by Andy Fingerhut [ 05/Dec/13 8:54 AM ]

Thanks for the update. The same test case in the description still give the same exception, unless I have messed something up.

Comment by Nicola Mometto [ 05/Dec/13 9:00 AM ]

Thanks for checking it out, I forgot to include a line of diff in the commit, I pushed the additional fix: https://github.com/clojure/tools.analyzer.jvm/commit/c295e3d285a5b4ec710b649cb7c12b169d05a802





[TANAL-23] validate-loop-locals doesn't work for fn Created: 04/Dec/13  Updated: 04/Dec/13  Resolved: 04/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Comments   
Comment by Nicola Mometto [ 04/Dec/13 5:33 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/72a52fa058739b658f8838661a2f8f11e59c4f40





[TANAL-22] analyzer throws exception if fn called with wrong number of args Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

Just analyze a simple form like this:

(defn bad-num-args [x]
  (assoc))

Clojure/JVM will throw an exception if you call it, but it will compile it without any errors. Not sure if this is a sign that eastwood the lint tool should be avoiding use of the validate phase.



 Comments   
Comment by Nicola Mometto [ 03/Dec/13 7:00 PM ]

Not a tools.analyzer bug, this is by design.

Note that Clojure/JVM too will throw on exception at compile time for some expressions e.g. (Integer/foo)

If this is undesiderable for eastwood, you can wrap c.t.a.j.p.validate/validate to catch said exception and return the AST.

I'm thinking something like

(fn [ast] 
  (try (validate ast) 
    (catch ExceptionInfo e 
      (if <e is thrown by validate :invoke>
         ast
         (throw e)))))
Comment by Andy Fingerhut [ 03/Dec/13 8:00 PM ]

Does c.t.a.j.p.validate/validate make any changes to the ast, or only check a set of conditions and throw exceptions if those conditions are found to be true?

If it makes no changes to the ast, I am wondering if it would make any sense to have a way of invoking validate where it returns a sequence of maps representing all of the conditions for which it would normally throw an exception. A lint tool like eastwood could then search through that sequence and output a selected subset of them as warnings, in its own format.

Comment by Nicola Mometto [ 03/Dec/13 8:16 PM ]

It does change the AST.

For example, (-analyze 'java.lang.String (empty-env)) will return {:op :maybe-class :class 'java.lang.String ..}, validate over that AST will check whether "java.lang.String" can be resolved as a class, in that case it will transform the AST to {:op :class :class java.lang.String ..} (note that :class now is a Class, not a Symbol) otherwise it will throw a class not found exception.

I could try to decomplect the throwing from the validation behaviour of validate but I'm worried about the interaction of the non-validated nodes with the other passes.

Obviously to the specific case of :invoke it woulnd't make any difference, but some passes would probably crash on other nodes, expecting a Class on the :class field and instead finding a Symbol.





[TANAL-21] java.class.name/staticStringMember works in Clojure/JVM with or without parens around it, but analyzer throws exception with parens around it Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I found similar code in namespace clojure.data.xml, and cut it down to get a smaller example. I am not sure if this is defined behavior or not, but both foo1 and foo2 return the same string value in Clojure, with no warning or error messages.

user=> (require '[clojure.tools.analyzer.jvm :as aj])
nil
user=> (def env (aj/empty-env))
#'user/env
user=> (def form (read-string "
  #_=> (defn foo1 []
  #_=>   javax.xml.transform.OutputKeys/INDENT)
  #_=> "))
#'user/form
user=> (def an (aj/analyze form env))
#'user/an
user=> (def env (aj/empty-env))
#'user/env
user=> (def form (read-string "
  #_=> (defn foo2 []
  #_=>   (javax.xml.transform.OutputKeys/INDENT))
  #_=> "))
#'user/form
user=> (def an (aj/analyze form env))

CompilerException clojure.lang.ExceptionInfo: No matching method: INDENT for class: class javax.xml.transform.OutputKeys and arity: 0 {:method INDENT, :class javax.xml.transform.OutputKeys, :argc 0}, compiling:(form-init8051492816641918211.clj:1:9)


 Comments   
Comment by Nicola Mometto [ 03/Dec/13 4:35 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/2a6c5c08e251a085b6c26964c25ce47f72b95516





[TANAL-20] Exception that validator cannot find method browse Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I am probably going to get lazy on the description of some of these. I won't get lazy on describing how to reproduce the issues I see, though. This one is excerpted from clojure.java.browse, and I've verified that the trimmed-down function actually works with Clojure 1.5.1.

user=> (import '(java.net URI))
java.net.URI
user=> (require '[clojure.tools.analyzer.jvm :as aj])
nil
user=> (def env (aj/empty-env))
#'user/env
user=> (def form (read-string "
  #_=> (defn open-url-in-browser [url]
  #_=>   (try 
  #_=>     (-> (clojure.lang.Reflector/invokeStaticMethod \"java.awt.Desktop\" 
  #_=>                                                    \"getDesktop\" (to-array nil))
  #_=>         (.browse (URI. url)))
  #_=>     url
  #_=>     (catch ClassNotFoundException e
  #_=>       nil)))
  #_=> "))
#'user/form
user=> (def an (aj/analyze form env))

CompilerException clojure.lang.ExceptionInfo: No matching method: browse for class: class java.lang.Object and arity: 1 {:method browse, :class java.lang.Object, :argc 1}, compiling:(form-init155566724550622590.clj:1:9)


 Comments   
Comment by Andy Fingerhut [ 03/Dec/13 12:55 PM ]

If these start getting too far into undefined/undocumented territory, feel free to let me know. I'm just feeding eastwood + tools.analyzer(.jvm) a large fraction of Clojure (but not clojure.core) and contrib libraries to see what it can and cannot handle.

Comment by Nicola Mometto [ 03/Dec/13 3:11 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/compare/6e1c9e4d601896c133b4a888dd165cc132cc2ae3...master

There's no need for you to waste your time trying to debug those any further, just open a ticket and I'll take care of it

Open a ticket for everything that runs fine in clojure but tools.analyzer fails to analyze, even if the cause may be the use of a undefined behaviour in clojure, I want to tools.analyzer to be useful and if this means supporting weird edge cases used by clojure, so be it.

Hopefully those will be fixed sometime in the future but as long as they are working in clojure, I want to support those.

#TANAL-17 was an exception since the expression could compile in clojure but it would actually crash after a couple of evaluations.

Thanks again for the valuable testing/bug reporting





[TANAL-19] clojure.tools.analyzer.jvm/analyze cannot find field or no-arg method 'close' for a FileOutputStream Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

That subject line is very likely way to specific to this test case – I have seen it for other stream and reader/writer types, too, and it is probably a more general issue than that. Here is a way to reproduce it at the REPL. The code is excerpted from clojure.java.io, and works fine with Clojure/JVM since 1.3 or maybe earlier.

(import '(java.io InputStream File FileOutputStream))
(require '[clojure.tools.analyzer.jvm :as aj])
(def env (aj/empty-env))
(def form (read-string "
(defmulti do-copy
  (fn [input output opts] [(type input) (type output)]))
"))
(def an (aj/analyze form env))

(def env (aj/empty-env))
(def form (read-string "
(defmethod do-copy [InputStream File] [^InputStream input ^File output opts]
  (with-open [out (FileOutputStream. output)]
    (do-copy input out opts)))
"))
(def an (aj/analyze form env))

;; Exception output begins as follows, but flies off the screen due to
;; huge size of data attached to ex-info

;; ExceptionInfo cannot find field or no-arg method call clojure.core/close for class class java.io.FileOutputStream


 Comments   
Comment by Nicola Mometto [ 03/Dec/13 12:15 PM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/6dbcc7cac0995266f1624c00c1499ea0b2b32984

This is clojure exploiting the undocumented behaviour that makes (. class ns/method) work as (. class method) :/





[TANAL-18] clojure.tools.analyzer.jvm/analyze fails to validate defprotocol expressions containing method names having a dash in the middle Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

This might be a dup of TANAL-17, but note that there is no dash at the beginning of the method name, only in the middle. The error goes away if I change the method name to asfile, with no dash in the middle.

I get the exception shown below with 1.6.0-alpha1, alpha2, or alpha3, with these steps, using latest tools.analyzer(.jvm). I get a different exception with 1.5.1, but still an exception.

user=> (require '[clojure.tools.analyzer.jvm :as aj])
nil
user=> (def form (read-string "(defprotocol ProtocolNameNotRedefd3
  #_=>   (as-file [x]))"))
#'user/form
user=> (def env (aj/empty-env))
#'user/env
user=> (def an (aj/analyze form env))

CompilerException clojure.lang.ExceptionInfo: No matching method: as-file for class: interface user.ProtocolNameNotRedefd3 and arity: 0 {:method as-file, :class user.ProtocolNameNotRedefd3, :argc 0}, compiling:(form-init7336696426514759785.clj:1:9)


 Comments   
Comment by Nicola Mometto [ 03/Dec/13 10:14 AM ]

It was an unrelated issue: t.a.jvm.utils/members was not munging the method/field name.

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/f295f1298c17af1926f2f4262b8947c4ca10daf7





[TANAL-17] clojure.tools.analyzer.jvm/analyze fails to validate defprotocol expressions containing method names that begin with a dash Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Reproduction test case:

user=> (require '[clojure.tools.analyzer.jvm :as aj])
nil
user=> (def form (read-string "(defprotocol MyProt1 (bar [a b]))"))
#'user/form
user=> (def env (aj/empty-env))
#'user/env
user=> (def an (aj/analyze form env))
#'user/an
user=> (def form (read-string "(defprotocol MyProt2 (-bar [a b]))"))
#'user/form
user=> (def an (aj/analyze form env))

ExceptionInfo cannot find field bar for class interface user.MyProt2  clojure.core/ex-info (core.clj:4327)


 Comments   
Comment by Nicola Mometto [ 03/Dec/13 8:26 AM ]

This is actually a clojure bug, see http://dev.clojure.org/jira/browse/CLJ-1202

This works for clojure >=1.6.0-alpha1





[TANAL-16] clojure.tools.analyzer.jvm/analyze fails to validate defmulti expressions Created: 03/Dec/13  Updated: 03/Dec/13  Resolved: 03/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I haven't tracked down exactly why yet, but my guess is that it is trying to resolve the var global-hierarchy, that appears in the macroexpansion of (defmulti foo :op), in the user namespace instead of clojure.core.

Steps to reproduce:

user=> (require '[clojure.tools.analyzer.jvm :as aj])
nil
user=> (def form (read-string "(defmulti foo :op)"))
#'user/form
user=> (def env (aj/empty-env))
#'user/env
user=> (def an (aj/analyze form env))
ExceptionInfo could not resolve var: global-hierarchy  clojure.core/ex-info (core.clj:4327)


 Comments   
Comment by Nicola Mometto [ 03/Dec/13 8:01 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/fec1c8bf4d2cee0f986854b76f99147d631fe26d





[TANAL-15] analyze seems to assign incorrect name to some locals used in function bodies Created: 01/Dec/13  Updated: 01/Dec/13  Resolved: 01/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I am using latest master version of tools.analyzer(.jvm) as of Dec 1 2013 3:30pm PDT (after TANAL-14 fix went in).

Here is a reproduction test case:

(require '[clojure.tools.analyzer.jvm :as aj])
(def form (read-string "
(defn fn-with-unused-args3 [x y z]
  (let [foo (fn [y z]
              (* y z))]
    (foo x y)))
"))
(def env (aj/empty-env))
(def an (aj/analyze form env))
(def meth1 (-> an :init :methods first))
(def ret-expr-args (-> meth1 :body :ret :body :ret :args))

(map :name (:params meth1))
;;=> (x__#0 y__#0 z__#0)
(map :name ret-expr-args)
;;=> (x__#0 y__#-1)

The "y_#-1" in the last output expression should be "y_#0", I think, to match the arg y.



 Comments   
Comment by Nicola Mometto [ 01/Dec/13 7:52 PM ]

This required a long due rewrite of the uniquify pass.
The logic is now much simpler and it works as expected.

https://github.com/clojure/tools.analyzer/commit/e7e3a1a597640df62cf1e15dd0530f3fdcce2377





[TANAL-14] tools.analyzer (maybe tools analyzer jvm specific) looses the IChunk type hint in the expansion of doseq Created: 30/Nov/13  Updated: 01/Dec/13  Resolved: 01/Dec/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Kevin Downey Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-14-failing-test.diff    

 Description   

looking at the output of running the analyzer on

(doseq [i (range 10)] (println i))

the chunk local in the loop in the macro expansion of doseq his a tag of object, but if you look at the source of doseq, or manually do the macro expansion, the chunk local is type hinted as IChunk



 Comments   
Comment by Kevin Downey [ 01/Dec/13 1:21 AM ]

diff adds a failing test

Comment by Kevin Downey [ 01/Dec/13 5:46 AM ]

it looks like -validate-loop-locals :loop is incorrectly mutating the ast?

if a a symbol doesn't have :tag metadata in the body of a loop, it tags the initial binding with Object

Comment by Kevin Downey [ 01/Dec/13 5:52 AM ]

it occurs to me I could be wrong, maybe for some reason what is being passed to recur is not getting the correct tags?

Comment by Kevin Downey [ 01/Dec/13 6:10 AM ]

ok, I think what is happening is when there is a loop where the initial binding is type hinted, and you have N branches in the body, and one of the branches recurs with a literal nil in place for the type hinted binding, validate-loop-locals removes the type hint from the initial binding

Comment by Nicola Mometto [ 01/Dec/13 6:21 AM ]

I looked into this, there are a couple of issues here.

The second loop local is tagged IChunk in the loop expr and in the first two recurs, but it's nil in the third.
The validate-loop mechanism is a bit naive there and just compares the tag and casts to Object when tags don't match.

This obviously need to be fixed.

Another issue that arised is that the third&fourh args are sometime tagged as long and sometimes as int, and they both get casted to Object instead being treated as longs.
I need to fix this too.

A question that arises though is how exactly should the pass behave?
In Compiler.java recur mismatches are considered only for primitives, so this will compile (loop [^String x ""] (if (= "" x) (recur 1) (.hashCode x))) but will result in a "ClassCastException java.lang.Long cannot be cast to java.lang.String user/eval1 (NO_SOURCE_FILE:1)" at runtime.

validate-locals tries to be smarter than that and validates not only primitives but also boxed objects, so that would compile to a Object#toString

For now, I'll change validate-locals to behave like Compiler.java does, I'll probably move the "smarter" behaviour of the current loop-locals pass to an additional optional pass.

Comment by Nicola Mometto [ 01/Dec/13 1:43 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/143f19d049e6dc85ca67441c7c73c5e390aa861f

It's a compromise between the current Compiler.java approach and doing the right thing in case of boxed objects tag mismatch.

I changed your tests to return clojure.lang.IChunk instead of 'clojure.lang.IChunk





[TANAL-13] tools.analyzer* seems to cause 2nd testing expr in clojure.core.async.ioc-macros-test to macroexpand incorrectly Created: 30/Nov/13  Updated: 30/Nov/13  Resolved: 30/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

I haven't figured out why yet, but when I use clojure.core/macroexpand to expand the following expression (from inside all of the definitions present in the latest master version of namespace clojure.core.async.ioc-macros-test):

(runner ((fn [] 42)))

it includes the following subexpression:

(clojure.core/let []
  (fn* ([] 42)))

However, when I use tools.analyzer* to analyze the same expression, it seems to macroexpand into the following instead, which throws an exception because it attempts to let-bind values to keywords:

(clojure.core/let
    [:loop-locals :loop-locals
     :namespaces :namespaces
     :ns :ns
     :locals :locals
     :context :context
     :name :name
     :once :once
     :in-try :in-try]
  (fn* ([] 42)))


 Comments   
Comment by Nicola Mometto [ 30/Nov/13 12:02 PM ]

This is because `runner` uses &env.

Clojure's &env is a map of local-sym -> local-binding
tools.analyzer's &env is a much richer map, I'm afraid that code that depends on &env will never be compatible between clojure and tools.analyzer's default macroexpander.

If required however it's trivial to use a custom macroexpander with tools.analyzer that passes as &env a localsym->localbinding map.





[TANAL-12] fn* with multiple methods do not clear :in-try while analyzing their bodies Created: 30/Nov/13  Updated: 30/Nov/13  Resolved: 30/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

When a fn* body has 2 or more methods, its bodies are analyzed with the same value of :in-try in the env as the (fn* ...) expression itself, so if that happens in the body of a try expression, recur statements within the fn* bodies incorrectly throw an assertion exception. Example:

(try
  (fn foo
    ([]
       nil)
    ([x]
       (if (< x 5)
         (println x)
         (recur (inc x)))))
  (catch Exception e
    (println "Exception occurred")))

When the fn has only one method, the env correctly has no :in-try key at all, so no exception is thrown if there is a recur in the fn body.



 Comments   
Comment by Nicola Mometto [ 30/Nov/13 6:43 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/53bca4fbc26f134427f3ba2736818fe12af3cb58





[TANAL-5] replace tools.analyzer.jvm/empty-env :namespaces with a reified IPersistentMap Created: 27/Nov/13  Updated: 29/Nov/13  Resolved: 29/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Declined Votes: 0
Labels: None


 Description   

Currently it's not possible to analyze

(require '[clojure.pprint :as pp]) pp/pprint

using the same env, it is required to re-generate the environment every time an analyzed form is evaluated.

empty-env's :namespaces should not be a map, but a reify-backed map exposing clojure.lang.Namsespace fields



 Comments   
Comment by Nicola Mometto [ 29/Nov/13 9:11 AM ]

While something like http://sprunge.us/OgNP?clj could potentially work for reads, it would complicate everything a great deal for e.g assocs, it's easier to just need the regeneration of the environment after the evaluation of every form.





[TANAL-11] cloure.tools.analyzer method -parse 'def has precondition that can be violated Created: 28/Nov/13  Updated: 28/Nov/13  Resolved: 28/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Sorry, I haven't tried to cut this example down further. It is excerpted from the clojure.algo.monads contrib library.

(ns tryanalyzer.try16
  (:use [clojure.tools.macro
         :only (with-symbol-macros defsymbolmacro name-with-attributes)]))


(defmacro defmonadfn
  "Like defn, but for functions that use monad operations and are used inside
   a with-monad block."
  {:arglists '([name docstring? attr-map? args expr]
               [name docstring? attr-map? (args expr) ...])}
  [name & options]
  (let [[name options]  (name-with-attributes name options)
        fn-name (symbol (str *ns*) (format "m+%s+m" (str name)))
        make-fn-body    (fn [args expr]
                          (list (vec (concat ['m-bind 'm-result
                                              'm-zero 'm-plus] args))
                                (list `with-symbol-macros expr)))]
    (if (list? (first options))
      ; multiple arities
      (let [arglists        (map first options)
            exprs           (map second options)
            ]
        `(do
           (defsymbolmacro ~name (partial ~fn-name ~'m-bind ~'m-result 
                                                   ~'m-zero ~'m-plus))
           (defn ~fn-name ~@(map make-fn-body arglists exprs))))
      ; single arity
      (let [[args expr] options]
        `(do
           (defsymbolmacro ~name (partial ~fn-name ~'m-bind ~'m-result 
                                                   ~'m-zero ~'m-plus))
           (defn ~fn-name ~@(make-fn-body args expr)))))))

(defsymbolmacro m-bind m-bind)

(defmonadfn m-join
  "Converts a monadic value containing a monadic value into a 'simple'
   monadic value."
  [m]
  (m-bind m identity))

The latest tools.analyzer lib throws an exception when attempting to evaluate the precondition on method -parse 'def:

(or (not (namespace sym))
             (= *ns* (the-ns (namespace sym))))]}

If I try to replace (namespace sym) with (symbol (namespace sym)), the precondition no longer throws an exception, but soon after the create-var call in the method does.



 Comments   
Comment by Nicola Mometto [ 28/Nov/13 2:59 PM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/309c9f9dd90bd777cc17be2b0f8d0e28cdf670da





[TANAL-10] #'test-name var not found inside of (deftest test-name ...) Created: 28/Nov/13  Updated: 28/Nov/13  Resolved: 28/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Simplified case found while analyzing a tools.nrepl test file:

(ns tryanalyzer.try15
  (:use clojure.test))

(deftest update2
  (str #'update2))


 Comments   
Comment by Nicola Mometto [ 28/Nov/13 12:52 PM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/1b88059e0c424ce2517370169908094900ebf253

Looks like clojure.test relies on the (unspecified?) behaviour of clojure to evaluate def metadata after interning the var.





[TANAL-7] clojure.tools.analyzer.jvm.utils/maybe-class can incorrectly return array class Created: 28/Nov/13  Updated: 28/Nov/13  Resolved: 28/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-7-v1.diff    
Patch: Code

 Description   

I came across this when trying to analyze namespace clojure.core.logic.fd, which has a protocol named IIntervals and another named IInterval. The first, when used in a deftype, was turned by maybe-class into an array of IInterval, on which eval threw an exception when trying to eval the resulting deftype* data structure.

It seems safer to include in the code a list of special type names for arrays, like there are now for "short", "byte", etc.



 Comments   
Comment by Andy Fingerhut [ 28/Nov/13 12:17 AM ]

tanal-7-v1.diff is one way that seems to work.

Comment by Nicola Mometto [ 28/Nov/13 6:09 AM ]

fixed: https://github.com/clojure/tools.analyzer.jvm/commit/7f45f362168ea2ad176e5ffe066bd442db26e3f1

Thanks a lot for the help Andy!





[TANAL-8] No earmuff warnings for vars named * or ** Created: 28/Nov/13  Updated: 28/Nov/13  Resolved: 28/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-8-v1.diff    
Patch: Code

 Description   

I think it would be better not to issue warnings about vars named * or **, even if they are not dynamic.

Clojure does not warn about *, and there is a recent patch that modifies it not to warn for **, either.



 Comments   
Comment by Andy Fingerhut [ 28/Nov/13 12:28 AM ]

Patch tanal-8-v1.diff should do the job.

Comment by Nicola Mometto [ 28/Nov/13 6:08 AM ]

Fixed: https://github.com/clojure/tools.analyzer/commit/2cfa02ed7fe23f36c373c9ff0fa5eb8d77b48038





[TANAL-6] infer-tag :local method throws NPE when trying to deref @atom (which is null) Created: 27/Nov/13  Updated: 27/Nov/13  Resolved: 27/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I haven't analyzed what is going wrong here beyond what is stated in the summary line, but I have reduced the test case I found down to a single function that isn't terribly long. There isn't much that can be removed from this test case without causing the exception to no longer be thrown.

(require '[clojure.tools.analyzer.jvm :as analyze-jvm])

(def form '(defn pprint-matrix []
             (doseq [[i row] (map (fn [i] [i 1]) [5 6 7])]
               (doseq [p (take i (range))]
                 (print p))
               (print i))))

(analyze-jvm/analyze form (analyze-jvm/empty-env))


 Comments   
Comment by Andy Fingerhut [ 27/Nov/13 6:57 PM ]

Sorry, you may have already fixed this in master and I was not using the latest version when I hit this issue. Will close if I cannot reproduce with latest master of tools.analyzer and .jvm.

Comment by Nicola Mometto [ 27/Nov/13 7:04 PM ]

I can reproduce on master, I'll look into this, thanks!

Comment by Nicola Mometto [ 27/Nov/13 8:11 PM ]

This was a hard one but luckly the fix was easy.
Turns out the uniquify pass was doing some unnecessary operations that messed up some locals names, fixed with https://github.com/clojure/tools.analyzer/commit/ac5ccd7856eefef657495fe185082bf7bca9343c and https://github.com/clojure/tools.analyzer.jvm/commit/81c76490b76bfb98665f0f61d7ccda86d3b6d287





[TANAL-4] resolve-ns seems to resolve aliases incorrectly Created: 27/Nov/13  Updated: 27/Nov/13  Resolved: 27/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None

Attachments: File tanal-4-v1.diff    
Patch: Code

 Description   

I am not very familiar with the tools.analyzer code base, so double-check this analysis very carefully.

I was trying to read namespace clojure.core.match.debug using tools.analyzer, and it was throwing an exception:

ExceptionInfo var not found: pp/*print-pprint-dispatch*  clojure.core/ex-info (core.clj:4355)

Here are the first few lines of that namespace for reference:

(ns ^{:skip-wiki true}
  clojure.core.match.debug
  (:refer-clojure :exclude [compile])
  (:use [clojure.core.match.protocols]
        [clojure.core.match 
         :only [emit-matrix compile occurrences
                rows action-for-row clj-form]])
  (:require [clojure.pprint :as pp]))

(defn source-pprint [source]
  (binding [pp/*print-pprint-dispatch* pp/code-dispatch
            pp/*print-suppress-namespaces* true]
    (pp/pprint source)))

I think this happens because fn

clojure.tools.analyzer.utils/resolve-ns
is not resolving aliased namespaces correctly.



 Comments   
Comment by Andy Fingerhut [ 27/Nov/13 5:47 PM ]

Take this patch with a huge grain of salt until you've verified what it is doing yourself. As mentioned above, I am not very familiar with this code base – just trying it out analyzing various contrib libraries and seeing whether it works.

Comment by Nicola Mometto [ 27/Nov/13 5:54 PM ]

The current behaviour is clearly wrong and your patch is perfectly fine, thanks!

Fixed: https://github.com/clojure/tools.analyzer/commit/5441e5d916f664978fbd744c2f6209acb426155a





[TANAL-3] Get tools.analyzer.jvm to work on IBM JDK 1.6 Created: 24/Nov/13  Updated: 26/Nov/13  Resolved: 26/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

Currently some tests related to method validation fail on IBM JDK 1.6 for tools.analyzer.jvm, investigate and fix the cause.



 Comments   
Comment by Nicola Mometto [ 26/Nov/13 4:25 PM ]

https://github.com/clojure/tools.analyzer.jvm/commit/25e3a0fece1d96bb261e026df08d08e73766e45d





[TANAL-1] bug in validate loop locals generates invalide ast Created: 12/Nov/13  Updated: 12/Nov/13  Resolved: 12/Nov/13

Status: Closed
Project: tools.analyzer
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Kevin Downey Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None


 Description   

I am seeing valid asts being generate for some recur forms, some arguments to recur in the ast are a {:env ..} map, no :op or other information

I think the following is the source of the problem:

(defmethod -validate-loop-locals :recur
  [_ {:keys [exprs env] :as ast}]
  (if validating?
    (let [casts (:loop-locals-casts env)]
      (assoc ast
        :exprs (mapv (fn [e c]
                       (if c (assoc e :tag c) c))
                     exprs (vals casts))))
    ast))

if I change it to (return e if not c in the mapv)

(defmethod -validate-loop-locals :recur
  [_ {:keys [exprs env] :as ast}]
  (if validating?
    (let [casts (:loop-locals-casts env)]
      (assoc ast
        :exprs (mapv (fn [e c]
                       (if c (assoc e :tag c) e))
                     exprs (vals casts))))
    ast))

I get the information I expect in the ast



 Comments   
Comment by Nicola Mometto [ 12/Nov/13 5:00 PM ]

Fixed: https://github.com/clojure/tools.analyzer.jvm/commit/64f96fd5ef34d49db3a9c6cc4d009788ee3d2889





Generated at Fri Oct 31 18:03:57 CDT 2014 using JIRA 4.4#649-r158309.