<< Back to previous view

[TNS-26] Attempted call to clojure.core/alias appears to be shadowed by local binding to name 'alias' Created: 29/Sep/14  Updated: 10/Oct/14  Resolved: 10/Oct/14

Status: Resolved
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None


 Description   

I discovered this while testing a new linter for Eastwood [1], not by running code, so I don't know what the run-time effect of this issue is, but I would guess it would cause an exception to be thrown if the last line of function recover-ns is ever executed.

Either that or I am really confused by the last line of function recover-ns

[1] https://github.com/jonase/eastwood



 Comments   
Comment by Stuart Sierra [ 10/Oct/14 9:59 AM ]

Good catch, thanks! Fixed in 122e3d1d

It didn't throw an exception because the local binding of alias was a symbol. When you invoke a symbol, it behaves like keywords or get, so there was no error.





[TNS-25] A few doc string typos Created: 09/Sep/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File tns-25-v1.patch    
Patch: Code

 Description   

Attached patch corrects a few misspellings in tools.namespace doc strings



 Comments   
Comment by Stuart Sierra [ 19/Sep/14 9:06 AM ]

Applied in https://github.com/clojure/tools.namespace/commit/09e1e1fd81faed4e24399bf68bf78e87b32eb68b

Thanks.





[TNS-24] Broken tracker after mis-named namespace Created: 07/Sep/14  Updated: 11/Sep/14

Status: Open
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Unresolved Votes: 0
Labels: None


 Description   

A tools.namespace dependency tracker can get into an inconsistent state after attempting to load a file with a namespace declaration that does not match the file's name.

Steps to reproduce:

In a new Clojure project, create a file foo.clj on the classpath containing the namespace declaration (ns wrong-ns-declaration-for-foo)

Then at the REPL:

user=> (use 'clojure.tools.namespace.repl)
nil
user=> (refresh)
:reloading (wrong-ns-declaration-for-foo)
:error-while-loading wrong-ns-declaration-for-foo
#<FileNotFoundException java.io.FileNotFoundException: Could not locate wrong_ns_declaration_for_foo__init.class or wrong_ns_declaration_for_foo.clj on classpath: >

Edit the file foo.clj so that its namespace declaration is correct: (ns foo)

But at the REPL, refresh still doesn't work:

user=> (refresh)
:reloading (foo wrong-ns-declaration-for-foo)
:error-while-loading wrong-ns-declaration-for-foo
#<FileNotFoundException java.io.FileNotFoundException: Could not locate wrong_ns_declaration_for_foo__init.class or wrong_ns_declaration_for_foo.clj on classpath: >

Since tools.namespace 0.2.5, the workaround is to call clear:

user=> (clear)
{}
user=> (refresh)
:reloading (foo)
:ok


 Comments   
Comment by Andy Fingerhut [ 11/Sep/14 6:38 PM ]

Such files introduce problems for all kinds of tools, not just tools.namespace. I don't know all of the consequences, but two are listed here: https://github.com/jonase/eastwood#check-consistency-of-namespace-and-file-names

I have added warnings in Eastwood whenever it finds files like this, and doesn't do any linting checks on any files until such things are corrected.





[TNS-23] Refresh breaks when changing priority of conflicting methods in `prefer-method` Created: 30/Aug/14  Updated: 07/Sep/14  Resolved: 04/Sep/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jeff Terrell Assignee: Stuart Sierra
Resolution: Declined Votes: 0
Labels: bug
Environment:

Clojure 1.6, CTN 0.2.5.



 Description   

Try defining a record that implements the clojure.lang.IDeref protocol. Create an instance, then try to print it, and you'll get an exception like so:

java.lang.IllegalArgumentException: Multiple methods in multimethod 'print-method' match dispatch value: class <record-class-name> -> interface clojure.lang.IPersistentMap and interface clojure.lang.IDeref, and neither is preferred

The solution to this is to include this line after the defrecord form:

(prefer-method print-method clojure.lang.IDeref clojure.lang.IPersistentMap)

This is all fine so far. The problem is that if you change the preference and call (ctn/refresh), you get an exception:

Error refreshing environment: java.lang.IllegalStateException: Preference conflict in multimethod 'print-method': interface clojure.lang.IPersistentMap is already preferred to interface clojure.lang.IDeref, compiling<source-file.clj>:164:22)



 Comments   
Comment by Stuart Sierra [ 04/Sep/14 4:18 PM ]

This is because prefer-method is a global side-effect at the top-level of a namespace.

The only way to work around this would be to call remove-method before reloading the namespace. Knowing to do that would require deep analysis far outside the scope of tools.namespace, but it can be done in application-specific code in a wrapper around refresh.

Comment by Jeff Terrell [ 05/Sep/14 2:05 PM ]

Fair enough. Thanks for looking into it.

Is this limitation worth documenting somewhere? Perhaps either in the API docs or in the c.t.n README?

Comment by Stuart Sierra [ 05/Sep/14 2:32 PM ]

Will add a note in README

Comment by Stuart Sierra [ 07/Sep/14 10:37 AM ]

Added warning to README in [commit 57e5658c8d9154711979019d85b279ad5f6898c9](https://github.com/clojure/tools.namespace/commit/57e5658c8d9154711979019d85b279ad5f6898c9).





[TNS-22] Function ns-file-name makes incorrect call to clojure.string/replace when File/separator is "\\" (e.g. Windows) Created: 24/Aug/14  Updated: 07/Sep/14  Resolved: 07/Sep/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File tns-22-v1.patch    
Patch: Code

 Description   

On Windows, where File/separator is a string consisting of a single backslash character:

user=> (require '[clojure.string :as str])
nil

;; This works as expected, because str/replace with second arg being a string handles arbitrary strings as the third arg
user=> (str/replace "foo.bar" "." File/separator)
"foo\\bar"

;; This gives an exception, because str/replace with second arg being a regex treats backslash characters in third arg specially.
user=> (str/replace "foo.bar" #"\." File/separator)
StringIndexOutOfBoundsException String index out of range: 1  java.lang.String.charAt (String.java:658)

;; This works as expected, but is more complex than the first alternative above
user=> (str/replace "foo.bar" #"\." (str/re-quote-replacement File/separator))
"foo\\bar"

I'd recommend using str/replace calls like the first one above when the things you want to replace are constant strings.



 Comments   
Comment by Andy Fingerhut [ 24/Aug/14 7:01 PM ]

Patch tns-22-v1.patch changes the only 2 calls to clojure.string/replace I found in tools.namespace that could have their second arg be a constant string rather than a regex.

Comment by Stuart Sierra [ 07/Sep/14 11:32 AM ]

Thanks.

Patch applied in https://github.com/clojure/tools.namespace/commit/a44f709e9e0a35a0eac6ec8eb6a2845c54efb97f

Comment by Stuart Sierra [ 07/Sep/14 2:39 PM ]

In release 0.2.6





[TNS-21] tools.namespace ignores ns form dependencies inside vectors Created: 23/Aug/14  Updated: 04/Sep/14  Resolved: 04/Sep/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Declined Votes: 0
Labels: None


 Description   

Cross reference: TLOG-13

For example, tools.logging has this ns form (metadata omitted):

(ns clojure.tools.logging
  [:use
   [clojure.string :only [trim-newline]]
   [clojure.pprint :only [code-dispatch pprint with-pprint-dispatch]]]
  [:require [clojure.tools.logging.impl :as impl]])

tools.namespace ignores these dependencies, only processing those that are in list subforms of the ns form.

Not sure if the best way to handle this is to update tools.logging or tools.namespace. tools.logging seems pretty unusual in using vectors like this, but the Clojure compiler seems to accept it just fine.



 Comments   
Comment by Stuart Sierra [ 24/Aug/14 1:29 PM ]

The ns macro docstring is quite clear that references inside it should be lists. I've seen a lot of weird ns forms, and this is the first time I've seen vectors used this way.

I would say I'm surprised that vectors work here, except nothing weird about the ns macro surprises me any more.





[TNS-20] tracker's unload order sometimes incorrect Created: 23/Aug/14  Updated: 02/Oct/14

Status: Open
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File tns-20-v1.patch     Text File tns-20-v2.patch    

 Description   

The attached patch contains a new test namespace clojure.tools.namespace.load-unload-order-test that demonstrates the incorrect unload order, if you leave out the proposed fix in file track.clj



 Comments   
Comment by Andy Fingerhut [ 23/Aug/14 3:02 PM ]

Patch tns-20-v1.patch contains a test that demonstrates the bug, and contains a proposed fix. Please scrutinize the proposed fix carefully, as I haven't yet convinced myself 100% that it is the right fix.

Also adds one more dependency check in dependency_test.clj that I discovered while reviewing those tests, to model my dependency checking tests on.

Comment by Andy Fingerhut [ 24/Aug/14 4:52 PM ]

I've got some code that checks that the load and unload orders in a tools.namespace tracker are consistent with its dependencies. It is similar to the checks done now in dependencies_test.clj, except it assumes that the dependencies in the tracker are correct and verifies the load and unload orders against those.

If you would be interested in a patch that added these checks to tools.namespace, perhaps along with a debug/consistency-check flag that when true causes these checks to be run every time a tracker is updated, let me know.

Comment by Stuart Sierra [ 07/Sep/14 11:11 AM ]

Thanks for this, Andy. I will need to study it further to convince myself this is the correct behavior.

To help, can you describe what the observed problem would be from a user's point of view?

We cannot assume that the dependencies in the tracker are always correct. I have demonstrable cases where they are wrong. Usually that takes the form of nonexistent namespaces left in the dependency graph from files that were deleted or had an invalid ns declaration.

Comment by Andy Fingerhut [ 07/Sep/14 8:10 PM ]

I have not used tools.namespace for the reload workflow that it was originally developed for, so I can't say with certainty what the effect on a user would be, but I can make some guesses that you can probably confirm more quickly than I can.

The test case in the patch is one where a few Clojure source files are added to a tracker using function c.t.n.dir/scan-all. No other changes are made to the tracker. At that point, the dependencies are completely correct, and the load order calculated from those dependencies honors them, but the unload order does not.

If that tracker were then used in a call to c.t.n.reload/track-reload, track-reload would call remove-lib on a library B before calling remove-lib on a library A, even though A requires or uses B.

I guess your question confuses me a bit. Do you believe tools.namespace should only create trackers that have load and unload orders that honor the dependencies? Or is that a wrong assumption I was making from reading the implementation?

Comment by Andy Fingerhut [ 08/Sep/14 5:52 PM ]

Attaching slightly cleaned up patch tns-20-v2.patch. Identical to tns-20-v1.patch except it avoids copying a function into the new test namespace by adding a dependency on the test namespace where it is defined.

Also updates the name of a deftest I had copied but not renamed in v1 of the patch.

Comment by Stuart Sierra [ 19/Sep/14 4:19 PM ]

I think I'm starting to get a handle on this. There's a lot going on here, and it's been more than two years since I wrote most of this code, so bear with me.

At track.clj line 78 we compute the dependency order for unloading namespaces based on the dependency graph (the local deps) as it existed before the most recent set of changes. I believe that this is correct, or at least the correct intention. If we change a file such that its dependencies are different, the order in which we unload namespaces should reflect the namespaces in memory, as they were before we changed the file.

When using c.t.n.dir/scan to detect and reload changed files this works correctly, at least most of the time.

When adding files to a new tracker for the first time, the old dependency graph is empty, so the unload order is undefined. This is arguably incorrect but effectively meaningless, since those namespaces have not yet been loaded.

In release 0.2.6, there was a bad commit which mistakenly changed c.t.n.dir/scan and scan-all to remove the files which have changed before adding them again. As a result, the dependencies of changed files were removed from the tracker's dependency graph before the unload order could be calculated, so unload order was always undefined. I have reverted that change — thanks for drawing my attention to it!

Now the unload order should be "correct" after scan or scan-all, except for the first time files are added to the tracker, when unload order is undefined.

The tracker doesn't currently have a way to distinguish between changed files which need unload+load and new files which only need load. Even if there were a way to distinguish between new and changed files, we can't be certain that a namespace has not been loaded by other means (e.g. require at the REPL) so it's safest to unload everything before reloading.

In general, unload order shouldn't matter at all. Clojure doesn't care when namespaces are removed: the Vars are still referenced from wherever they are used.

Comment by Andy Fingerhut [ 20/Sep/14 1:08 PM ]

OK, makes sense. Is it already documented anywhere that the unload order is independent of the dependencies after scan-all (and perhaps other calls)? I can create a separate ticket for that if you think it is worth adding such documentation.

It sounds like for the application I had in mind, the reverse of the load order is always defined, and is a correct unload order. You are welcome to close this ticket.

Comment by Stuart Sierra [ 26/Sep/14 9:44 AM ]

After release 0.2.7, which fixed the regression in 0.2.6, :unload order should be correct in all cases except the first time files are added to a new tracker, even with scan-all. I would appreciate it if you can confirm this in your application.

It may be possible to fix the new-tracker case too. For example, commit c0b6b93d, currently on the branch TNS-20-fix-initial-unload-order. This works for the tests in your patch.

I'm not sure if the same change is needed at track.clj line 104 as well.

Comment by Andy Fingerhut [ 02/Oct/14 5:12 PM ]

My application right now is very simple compared to the component workflow – simply use dir/scan-all to get the namespaces and their dependencies, and then print them in a particular order consistent with a correct unload order. I'm sorry, but I don't have the interest right now in testing whether the unload order is correct after doing additional operations on the tracker, since it isn't needed in my application.

I do have a sanity check in my application that confirms the load order is consistent with the dependencies in my application, and will file a bug if I ever see that trigger anything, but I don't expect there is a bug there.

I have switched to using the reverse of the load order in my application for what I believe should always be a correct unload order.

I may look into the branch change you made, but it may fall off my radar unless my needs change.





[TNS-19] deps-from-ns-decl can return nil Created: 06/Jun/14  Updated: 19/Sep/14  Resolved: 11/Jul/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Ambrose Bonnaire-Sergeant Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None


 Description   

I expect this to be #{}.

(ns-parse/deps-from-ns-decl '(ns cljs.core.typed "Internal functions for CLJS" (:refer-clojure :exclude [IFn]) (:require-macros [clojure.core.typed.bootstrap-cljs :as boot])))
;=> nil


 Comments   
Comment by Stuart Sierra [ 13/Jun/14 5:04 PM ]

Fixed in commit e9394727 on Git master. Please verify.

Comment by Stuart Sierra [ 11/Jul/14 4:42 PM ]

Merged on Git master; will be in 0.2.5 release.

Comment by Stuart Sierra [ 19/Sep/14 9:08 AM ]

Included in release 0.2.6





[TNS-18] Use linear time algorithm to calculate transitive dependencies, instead of current exponential time algorithm Created: 03/Jun/14  Updated: 19/Sep/14  Resolved: 11/Jul/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File demo-exponential-algo.patch     Text File linear-time-no-debug.patch    
Patch: Code

 Description   

The function clojure.tools.namespace.dependency/transitive takes exponential time for cases where there are an exponential number of paths in the DAG (directed acyclic graph) of dependencies. I discovered this when using tools.namespace on the namespaces of project core.typed, where the total time for finding a topological order was several minutes on a reasonably fast machine.

It is easy to calculate transitive dependents/dependencies in linear time in the worst case (linear in the number of dependency arcs in the DAG).



 Comments   
Comment by Andy Fingerhut [ 03/Jun/14 3:18 PM ]

Patch demo-exponential-algo.patch only adds some test cases that demonstrate the exponential running time of the current algorithm for calculating transitive dependencies.

Comment by Andy Fingerhut [ 03/Jun/14 3:20 PM ]

Patch linear-time-no-debug.patch is one way to implement transitive dependencies in linear time. It also introduces additional protocol functions, which may not be desired. I am definitely open to suggestions on what kind of change you would like to see here (assuming you want any changes at all).

Comment by Stuart Sierra [ 06/Jun/14 10:27 AM ]

This is a valuable improvement — thanks, Andy!

I don't want to add more methods to the protocol, but I don't see any way to avoid it right now. Ideally, transitive-dependencies would change to take a set, but that breaks backwards-compatibility.

I'll give it a week or so to think about naming:

  • transitive-dependencies-of-node-set is too long
  • transitive-dependencies-all ?
  • transitive-dependencies-set ?
  • all-transitive-dependencies ?
Comment by Stuart Sierra [ 11/Jul/14 4:42 PM ]

Merged on master as of commit 3f946380. Will be in release 0.2.5.

New functions are named transitive-dependencies-set and transitive-dependents-set.

Comment by Stuart Sierra [ 19/Sep/14 9:08 AM ]

Included in release 0.2.6





[TNS-17] doesn't track namespace when ns decl isn't first in file Created: 04/Feb/14  Updated: 19/Sep/14  Resolved: 11/Jul/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Trevor Wennblom Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: bug, enhancement


 Description   

This was first filed for ns-tracker https://github.com/weavejester/ns-tracker/issues/15 but I understand it's more specific to tools.namespace.

(I realize this is likely a very low priority, but it'd still be nice to see fixed if possible. Thanks!)


It appears ns-tracker tools.namespace will cease following a namespace when there's non-commented content before the namespace declaration.

works:

;-begin
(ns myproject.core)
(println "hi")
;-end

untracked — (any of the four possibilities when uncommented, for example):

;-begin
1
;;; (println "before ns")
;;; (println "loading" *file*)
;;; :abc
(ns myproject.core)
(println "hi")
;-end


 Comments   
Comment by Stuart Sierra [ 13/Jun/14 4:43 PM ]

What's the use case for having a file in which the ns declaration isn't the first non-comment form?

Comment by Trevor Wennblom [ 13/Jun/14 5:53 PM ]

In the case that triggered the issue for me, it can be useful to see the load-order of files based on the dependencies given in ns for debugging. That is, to print a message before and after the completion of loading the namespace dependencies.

I'm not sure how difficult or involved the issue would be to solve.

Comment by Andy Fingerhut [ 13/Jun/14 6:42 PM ]

Would the :verbose option to require perhaps give you the load-order info you want? Try something like this at a REPL, with a namespace you care about:

(require '[clojure.core.typed :as ct] :verbose)

In general, I think Stuart's concern would be questions like: How many forms would you expect tools.namespace to skip over before giving up? If it is a fixed number of forms before it gives up, why that number and not some other?

If it is based upon some other condition to give up looking for an ns form, what would that condition be?

Or would you expect it to be able to find the ns form in the body of a 'when' or 'if' form, and not at the top level?

The criteria can easily start to sound kind of arbitrary, and/or complex to implement.

Comment by Stuart Sierra [ 11/Jul/14 5:04 PM ]

Fixed on master as of commit 3c08b722. This turned out to be an easy change.

I wouldn't recommend putting anything before the ns declaration, but it's no longer required to be first. A file could also have multiple ns declarations, although clojure.tools.namespace.track will only look at the first one.

Comment by Stuart Sierra [ 19/Sep/14 9:08 AM ]

Included in release 0.2.6





[TNS-16] Some tests depend upon Clojure hash for ordering Created: 30/Jan/14  Updated: 19/Sep/14  Resolved: 31/Jan/14

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: File tns-16-v1.diff    

 Description   

In particular there are several that do a topological sort of a dependency graph, and compare the returned node sequence against a single node order. In general there are many correct results for a topological sort, and the implementation's return order depends upon Clojure's hash function, recently changed.



 Comments   
Comment by Andy Fingerhut [ 30/Jan/14 9:40 AM ]

Patch tns-16-v1.diff is one way to make the tests independent of Clojure's hash function. It implements a topo-check function that determines whether a particular node sequence order is in a valid dependency order, i.e. is a subsequence of at least one topological sorting of the nodes.

Comment by Stuart Sierra [ 31/Jan/14 9:09 AM ]

Given that the current tests only use very small, hand-drawn graphs, I'm going to go with the brute-force approach for now.

But thanks for this patch. I'd like to hold on to it for possible future use with generated graphs and/or test.check.

Comment by Stuart Sierra [ 19/Sep/14 9:08 AM ]

Included in release 0.2.6





[TNS-15] Can't find files when their absolute path contains a space Created: 12/Dec/13  Updated: 10/Jan/14  Resolved: 13/Dec/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ryan Senior Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: File directories-with-a-space.diff    
Patch: Code

 Description   

The URLClassloader getURLs call is returning URLs that contain %20 instead of spaces. The current code calls .getPath, which leaves the %20 in there and ultimately can't find the directory. Changing the .getPath call to .toURI calls a different java.io.File constructor and fixed the problem for me.



 Comments   
Comment by Stuart Sierra [ 13/Dec/13 9:22 AM ]

Applied in commit 8f116ba8c6944acc347cba97db0244e247021016

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-14] Wasted call to str in read-ns-decl Created: 02/Dec/13  Updated: 10/Jan/14  Resolved: 02/Dec/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Declined Votes: 0
Labels: None


 Description   

The function read-ns-decl (both copies in each of two namespaces) contain the expression:

(doto (read rdr) str)

This is equivalent to (read rdr), plus a wasted call to str whose return value is discarded.



 Comments   
Comment by Stuart Sierra [ 02/Dec/13 12:07 PM ]

The call to str was added to force the Clojure reader to throw exceptions immediately when it encounters a syntax error. See TNS-1

Comment by Andy Fingerhut [ 02/Dec/13 4:20 PM ]

Sorry for the noise. I was trying out pre-release version of Eastwood linter on as many contrib libraries as possible, and it found that odd-looking code. It didn't occur to me to look through the commit history to see when it became like it was, and why. Subtle.

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-13] Refresh function not working properly Created: 17/Oct/13  Updated: 10/Jan/14  Resolved: 21/Oct/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Pierre Allix Assignee: Stuart Sierra
Resolution: Declined Votes: 0
Labels: None
Environment:

Linux, lein 2.2.0, Clojure 1.5.1



 Description   

It seems the clojure.tools.namespace.repl/refresh functions has a problem reloading my namespaces.

user=> (refresh)
:reloading (tikkba.core lacij.model.history tikkba.utils.dom tikkba.apps.svgbrowser lacij.model.graph-history tikkba.dom lacij.utils.core lacij.view.core lacij.view.rectnodeselection lacij.view.utils.style lacij.view.utils.text lacij.view.rectnodeview lacij.view.circlenodeview lacij.view.nodeview lacij.geom.distance lacij.view.straightedgeview tikkba.utils.xml lacij.view.nodelabelview lacij.model.edge lacij.view.graphview lacij.view.segmentededgeview lacij.model.node tikkba.swing lacij.model.graph lacij.view.edgelabelview lacij.edit.graph lacij.examples.swing carneades.engine.uuid carneades.engine.utils carneades.engine.statement carneades.engine.dublin-core carneades.engine.unify carneades.engine.argument carneades.engine.argument-generator carneades.config.config carneades.engine.theory.zip carneades.engine.theory.namespace carneades.engine.theory carneades.project.admin carneades.database.db carneades.database.argument-graph carneades.web.pack lacij.examples.simple carneades.engine.argument-graph carneades.engine.translation carneades.engine.theory.translation carneades.database.export carneades.database.import carneades.policy-analysis.web.core carneades.engine.search carneades.engine.argument-evaluation carneades.engine.dung carneades.engine.caes carneades.database.evaluation carneades.engine.aspic carneades.engine.policy carneades.policy-analysis.web.logic.questions carneades.engine.dialog carneades.policy-analysis.web.controllers.reconstruction lacij.opt.annealing carneades.xml.graphml.export carneades.engine.theory.namespace-test lacij.geom.intersect lacij.layouts.core lacij.layouts.randomlayout lacij.layouts.utils.position carneades.maps.format-statement carneades.engine.triplestore carneades.web.license-analysis.model.triplestore carneades.maps.lacij-params lacij.layouts.naivelayout lacij.layouts.hierarchicallayout lacij.layouts.radiallayout carneades.maps.subset-ag lacij.layouts.layout carneades.maps.lacij-export carneades.engine.ask carneades.engine.sandbox carneades.engine.argument-builtins carneades.engine.argument-construction carneades.engine.shell carneades.policy-analysis.web.logic.askengine carneades.maps.lacij carneades.web.license-analysis.model.analysis carneades.web.license-analysis.model.debug-analysis carneades.policy-analysis.web.views.pages carneades.database.case carneades.policy-analysis.web.controllers.policy-simulation carneades.policy-analysis.web.routes carneades.web.outline carneades.web.license-analysis.routes.service carneades.web.vote carneades.policy-analysis.web.jetty carneades.xml.caf.export carneades.web.project carneades.web.info carneades.web.service carneades.analysis.web.routes-dev carneades.analysis.web.routes-selfexe tikkba.utils.test.dom lacij.edit.dynamic lacij.examples.listener carneades.engine.dnf carneades.engine.test-dnf carneades.analysis.web.routes-war carneades.policy-analysis.web.main lacij.examples.styles lacij.examples.textwrapping lacij.examples.undoredo carneades.owl.owl carneades.owl.import-axioms lacij.examples.radiallayout2 carneades.engine.argument-edit carneades.project.admin-test carneades.web.liverpool-schemes carneades.engine.owl.rule lacij.examples.circle lacij.examples.def carneades.maps.graphviz lacij.examples.hierarchicallayout2 lacij.examples.layout carneades.engine.test-evaluation carneades.engine.test-theory lacij.examples.radiallayout tikkba.examples.dynamic carneades.analysis.web.system carneades.owl.import-language carneades.owl.import carneades.engine.triplestore-test carneades.web.test.service carneades.xml.lkif.lkif tikkba.examples.output-string lacij.examples.dynamic lacij.examples.hierarchicallayout tikkba.examples.writefile tikkba.examples.image carneades.owl.import-language-test lacij.examples.display-xml lacij.examples.bug tikkba.examples.analemma carneades.engine.test-scheme lacij.test.core carneades.engine.test-unify carneades.database.test.admin carneades.engine.test-statement lacij.examples.radialloop carneades.xml.lkif.import lacij.examples.custom tikkba.examples.core tikkba.examples.svgattr lacij.examples.graphdeps user tikkba.test.core carneades.engine.owl.reasoner carneades.policy-analysis.web.logic.server-properties carneades.xml.lkif.export carneades.xml.lkif.validator)
:error-while-loading carneades.engine.argument-generator
#<Exception java.lang.Exception: No namespace: carneades.engine.argument>

user=> (require 'carneades.engine.argument)
nil
user=> (require 'carneades.engine.argument-generator)

Exception No namespace: carneades.engine.argument clojure.core/refer (core.clj:3832)
user=>

Just (require 'carneades.engine.argument-generator) from a newly started REPL works so there is a problem with refresh.

I use checkouts directory in Lein and various use and requires forms.



 Comments   
Comment by Stuart Sierra [ 17/Oct/13 11:02 AM ]

Do you have any AOT-compiled namespaces in your project?

`refresh` does not work with AOT-compiled namespaces.

Comment by Pierre Allix [ 21/Oct/13 4:20 AM ]

I had one aot namespace, which I removed. I still get one strange error:

user=> (compile 'carneades.web.service)
carneades.web.service
user=> (refresh)
:reloading (tikkba.core lacij.model.history tikkba.utils.dom tikkba.apps.svgbrowser lacij.model.graph-history tikkba.dom lacij.utils.core lacij.view.core lacij.view.rectnodeselection lacij.view.utils.style lacij.view.utils.text lacij.view.rectnodeview lacij.view.circlenodeview lacij.view.nodeview lacij.geom.distance lacij.view.straightedgeview tikkba.utils.xml lacij.view.nodelabelview lacij.model.edge lacij.view.graphview ...)
:error-while-loading carneades.web.service
#<ExceptionInInitializerError java.lang.ExceptionInInitializerError>
user=> (pst)
IllegalStateException Alias dc already exists in namespace eu.markosproject.licensing.oss-licensing-theory, aliasing carneades.engine.dublin-core
clojure.lang.Namespace.addAlias (Namespace.java:224)
clojure.core/alias (core.clj:3870)
clojure.core/load-lib (core.clj:5387)
clojure.core/apply (core.clj:619)
clojure.core/load-libs (core.clj:5413)
clojure.core/apply (core.clj:619)
clojure.core/require (core.clj:5496)
eu.markosproject.licensing.oss-licensing-theory/eval19048/loading-4910auto---19049 (oss_licensing_theory.clj:4)
eu.markosproject.licensing.oss-licensing-theory/eval19048 (oss_licensing_theory.clj:4)
clojure.lang.Compiler.eval (Compiler.java:6619)
clojure.lang.Compiler.eval (Compiler.java:6608)
clojure.lang.Compiler.load (Compiler.java:7064)
nil

The namespace eu.markosproject.licensing.oss-licensing-theory is loaded dynamically with load-file. Could this be the source of the problem?

Comment by Stuart Sierra [ 21/Oct/13 3:46 PM ]

JIRA is not a good venue for debugging problems with one specific application. If you have an isolated test case, please create a new ticket with instructions to reproduce the failure. Until then, please direct all questions to the Clojure mailing list: https://groups.google.com/forum/#!forum/clojure

To answer the latest question: yes, clojure.tools.namespace.repl/refresh is incompatible with other code which generates or loads namespaces on-the-fly.

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-12] Duplicate jar-file? definition in find.clj Created: 16/Sep/13  Updated: 10/Jan/14  Resolved: 17/Sep/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Alex Coventry Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: patch

Attachments: Text File 0001-Remove-duplicate-of-jar-file-fn.patch    
Patch: Code

 Description   

There are two copies of the jar-file? function in find.clj.



 Comments   
Comment by Stuart Sierra [ 17/Sep/13 7:24 AM ]

Fixed in commit 0e4fe44be8ded594d0374d4cc21d6f42a697c18e

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-11] c.t.n.p/deps-from-ns-decl does not like (:use [some-root-ns module1 module2]) Created: 02/Jul/13  Updated: 10/Jan/14  Resolved: 02/Jul/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Thomas Christensen Assignee: Stuart Sierra
Resolution: Duplicate Votes: 0
Labels: None
Environment:

Clojure 1.5.1 ; Clojure.tools.namespace 0.2.3



 Description   

I am experiencing issues using the c.t.n.r/refresh on my project.

I believe that I have tracked down the issue to the way deps-from-ns-decl parses :use; that is: it is different from Clojure's.

I have several places used ns-forms of the shape:

(ns some-root-ns.module1
       (:use [some-root-ns module2 module3])

Where Clojure would load some-root-ns.module2 and some-root-ns.module3.

However, this is what deps-from-ms-decl gets:

(clojure.tools.namespace.parse/deps-from-ns-decl '(ns some-root-ns.module1
       (:use [some-root-ns module2 module3])))
;; => #{some-root-ns}

Note that while it is of course easily fixed by using this ns-form:

(clojure.tools.namespace.parse/deps-from-ns-decl '(ns some-root-ns.module1
       (:use [some-root-ns.module2])
       (:use [some-root-ns.module3])))
;; => #{some-root-ns.module3 some-root-ns.module2}

It would probably be desirable to be equivalent with Clojure's behavior.



 Comments   
Comment by Stuart Sierra [ 02/Jul/13 7:44 AM ]

c.t.namespace.parse does not currently support all the varied argument forms that Clojure's use and require will accept.

In general, c.t.namespace.parse adheres to the docstrings of the use and require functions, which state that prefixed namespaces are written as a list, not a vector. The use and require functions do not enforce this, leading to widespread variation in usage.

This is a duplicate of TNS-9.

Until it is resolved, you can write your prefixed namespaces as a list, like:

(ns some-root-ns.module1
  (:use (some-root-ns module2 module3)))

In general, I recommend against using prefix lists in ns declarations because they are harder to identify using text-based search tools.

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-10] Stack overflow on cyclic dependency Created: 15/Jun/13  Updated: 10/Jan/14  Resolved: 17/Jun/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ambrose Bonnaire-Sergeant Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None


 Description   

If a cyclic dependency is introduced between a namespace and itself, it results in a StackOverflow when calculating transitive-dependencies.

(transitive-dependencies (-> (graph)
                             (depend 'two 'two))
                         'two)
; => StackOverflow

I believe clojure.tools.namespace.dependency/transitive should maintain a set of seen namespaces and check if we have already seen the namespace at each step.



 Comments   
Comment by Stuart Sierra [ 15/Jun/13 10:33 AM ]

Already fixed in 0.2.4-SNAPSHOT by forbidding cyclic dependencies: https://github.com/clojure/tools.namespace/commit/85e73af04fdb497db1600ef3bf21aef5ed676619

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-9] Warn or error when parsing illformed prefix list Created: 14/Jun/13  Updated: 10/Jan/14  Resolved: 19/Jul/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Ambrose Bonnaire-Sergeant Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: File parse-fail.diff    

 Description   

Currently a ns entry (:require [foo [bar]]) where the prefix list is a vector is silently ignored.

Preferred behaviour is a warning or an error.



 Comments   
Comment by Ambrose Bonnaire-Sergeant [ 15/Jun/13 11:27 PM ]

The patch throws an exception on several ill-formed expressions.

Comment by Stuart Sierra [ 19/Jul/13 8:30 AM ]

Fixed in 0.2.4-SNAPSHOT

Commits e37221fecaf71ac0ca7e3022e3ee2cadbfbc1a45 and 930833dce8949a0154382038ac5fde85c5fb0abd

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-8] move-ns changes the timestamps on files whose content doesn't change Created: 14/Jun/13  Updated: 10/Jan/14  Resolved: 19/Jul/13

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Simon Katz Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: enhancement


 Description   

The summary says it all.
It's a minor annoyance; fixing it would be trivial.



 Comments   
Comment by Stuart Sierra [ 19/Jul/13 8:29 AM ]

Fixed in 0.2.4-SNAPSHOT

Commit 9b43f4a07e4d7264368d64a0dbbd3a2bc3b04097

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-7] Stack overflow on refresh after circular dependency detected Created: 17/Apr/13  Updated: 17/Apr/13

Status: Open
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Unresolved Votes: 1
Labels: None
Environment:

tools.namespace 0.2.3



 Description   
user> (refresh)
Exception Circular dependency between com.example.foo and com.example.bar  clojure.tools.namespace.dependency.MapDependencyGraph (dependency.clj:74)
user> (refresh)
StackOverflowError   clojure.lang.PersistentHashMap$BitmapIndexedNode.index (PersistentHashMap.java:576)
user> (clojure.repl/pst)
StackOverflowError 
	clojure.lang.PersistentHashMap$BitmapIndexedNode.index (PersistentHashMap.java:576)
	clojure.lang.PersistentHashMap$BitmapIndexedNode.find (PersistentHashMap.java:676)
	clojure.lang.PersistentHashMap$ArrayNode.find (PersistentHashMap.java:396)
	clojure.lang.PersistentHashMap.valAt (PersistentHashMap.java:152)
	clojure.lang.PersistentHashMap.valAt (PersistentHashMap.java:156)
	clojure.lang.RT.get (RT.java:645)
	clojure.tools.namespace.dependency/transitive (dependency.clj:52)
	clojure.tools.namespace.dependency/transitive/fn--7043 (dependency.clj:51)
	clojure.core.protocols/fn--6022 (protocols.clj:79)
	clojure.core.protocols/fn--5979/G--5974--5992 (protocols.clj:13)
	clojure.core/reduce (core.clj:6177)
	clojure.tools.namespace.dependency/transitive (dependency.clj:52)
nil





[TNS-6] Attempt to reload deleted file Created: 14/Dec/12  Updated: 14/Dec/12

Status: Open
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Unresolved Votes: 1
Labels: None


 Description   

I can't identify the exact circumstances, but I have seen events where a source code file has been deleted but clojure.tools.namespace.repl/refresh still tries to reload it. Because the file doesn't exist, there's an exception when you try to load it, so you're stuck.






[TNS-5] Allow any valid .clj* source file to be parsed/analysed Created: 01/Nov/12  Updated: 07/Sep/14

Status: Open
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Max Penet Assignee: Stuart Sierra
Resolution: Unresolved Votes: 1
Labels: enhancement

Attachments: Text File e3cd6d1fa6e0c900bc1086e4a93bbc9cb343a820.patch    

 Description   

This broadens the allowed file types to anything ending with #"\.clj.?$", meaning this would work for clj, cljs, cljc, cljx and possibly other Clojure implementations with their own extension in the future.

This allows libraries such as codox (and possibly autodoc) to work with ClojureScript and others implementations without any modification.

Note: My CA is on the way, I sent it a week ago, I am not sure if it arrived yet (it was sent from Switzerland with normal mail, with an ETA of 1 week).



 Comments   
Comment by Max Penet [ 02/Nov/12 6:51 AM ]

CA received it seems (I am listed on http://clojure.org/contributing ).

Comment by Stuart Sierra [ 02/Nov/12 3:23 PM ]

I'm not sure about this. If you're only using c.t.n.find in isolation, it's fine. But if you're using code-reloading and c.t.n.repl, it could incorrectly try to reload .cljs files in JVM Clojure.

We really need [Feature Expressions]http://dev.clojure.org/display/design/Feature+Expressions or something like it to get away from multiple file extensions.

Until then, I think it has to be optional. I don't know how best to achieve this. The APIs in c.t.n.repl and c.t.n.dir are not amenable to extension. I'll think about it.

Comment by Max Penet [ 06/Nov/12 6:11 PM ]

True, I didn't realize that.

Maybe using a dynamic var to hold the regex pattern (or a predicate?) could be a reasonable solution in the meantime, this would allow to rebind it in the case of codox & similar libs.
I don't really "like" to use dynamic vars (or regexes!), but in this case it might make sense.
Not to mention it would also allow more flexibility on projects where you don't want to have your clj codoxed, but only your cljs for instance.

Comment by Max Penet [ 24/Nov/12 7:47 AM ]

Any thoughts on my last comment/edit? I don't think there is a single doc lib that works with cljs at the moment, it is a bit painful to be honest.

Comment by Stuart Sierra [ 24/Nov/12 4:25 PM ]

Yes, I think a dynamic var would be OK. However, I would like to know of a real use case, not just a potential one.

Comment by Max Penet [ 13/Dec/12 2:40 AM ]

The idea was to be able to use codox on cljs files, I tried locally but there are other problems with this approach to be able to get to cljs vars metadata. So in the end I think you were right, maybe it's better to wait for feature expressions for this.





[TNS-4] Eliminate several uses of reflection in tools.namespace Created: 28/Oct/12  Updated: 10/Jan/14  Resolved: 28/Oct/12

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File tns-4-eliminate-reflection-v1.txt    

 Description   

There are several reflection warnings when compiling tools.namespace, all of which can be eliminated with fairly straightforward additions of type hints.



 Comments   
Comment by Andy Fingerhut [ 28/Oct/12 8:08 PM ]

tns-4-eliminate-reflection-v1.txt dated Oct 28 2012 adds the necessary type hints to eliminate reflection warnings in tools.namespace.

Comment by Stuart Sierra [ 28/Oct/12 9:14 PM ]

Patch applied. Thanks!

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-3] refresh-dirs not used Created: 08/Oct/12  Updated: 10/Jan/14  Resolved: 08/Oct/12

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None


 Description   

Submitted by Mika Raento as a GitHub pull request: https://github.com/clojure/tools.namespace/pull/1



 Comments   
Comment by Stuart Sierra [ 08/Oct/12 4:59 PM ]

Fixed in commit bc9d5c1a6f191070a425b04feda9e2d3c2eb6928

https://github.com/clojure/tools.namespace/commit/bc9d5c1a6f191070a425b04feda9e2d3c2eb6928

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-2] Add function to parse dependencies from namespace declarations Created: 16/May/12  Updated: 10/Jan/14  Resolved: 16/May/12

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File TNS-2-0001.patch    
Patch: Code

 Comments   
Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





[TNS-1] Workaround to Clojure 1.2 reader bug Created: 14/Dec/11  Updated: 10/Jan/14  Resolved: 24/Apr/12

Status: Closed
Project: tools.namespace
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Sam Ritchie Assignee: Stuart Sierra
Resolution: Completed Votes: 2
Labels: None
Environment:

Mac OS X, Clojure 1.2.1, Leiningen 1.6.2


Attachments: Text File 0001-Workaround-to-Clojure-1.2-reader-bug.patch    
Patch: Code and Test

 Description   

The clojure 1.2 reader will allow invalid forms like {:key} to be read in, and only throw an exception on printing. Currently clojure.tools.namespace calls (read rdr) within a try form; the bug means that this particular type of error is never caught. This patch forces the reader to try and resolve with str, allowing clojure.tools.namespace to catch and bury the error.

I was running into this with moustache templates from lein-newnew on the classpath; these contain namespace headers that look like (ns name.core). This would cause (clojure.tools.namespace/find-namespaces-on-classpath) to fail when printing its results but not when actually running.



 Comments   
Comment by Sam Ritchie [ 14/Dec/11 4:55 PM ]

Funny, jira picked up the moustache markup. a bad namespace looks like (ns { { name } } . core).

Comment by Sam Ritchie [ 11/Jan/12 3:59 PM ]

Ping – Stuart, any thoughts on this?

Comment by Stuart Sierra [ 11/Jan/12 6:09 PM ]

Why should tools.ns do this? If the syntax is wrong, it's wrong.

Comment by Sam Ritchie [ 11/Jan/12 6:43 PM ]

Because without this patch, it's impossible to catch and bury errors from invalid reader syntax. I believe this comes from a bug in the reader that was fixed with 1.2.

Comment by Stuart Sierra [ 27/Jan/12 9:44 AM ]

Declined. It is not the responsibility of this library to catch errors in old versions of Clojure.

Comment by Stuart Sierra [ 24/Apr/12 2:01 PM ]

Reopening because this is still a visible issue for some libraries. I still don't like it, but I'm going to include it.

Comment by Stuart Sierra [ 24/Apr/12 2:04 PM ]

Patch applied.

Comment by Sam Ritchie [ 24/Apr/12 2:07 PM ]

Great, thanks!

Comment by Stuart Sierra [ 10/Jan/14 11:26 AM ]

Mark old resolved issues as 'closed'





Generated at Thu Oct 23 01:42:39 CDT 2014 using JIRA 4.4#649-r158309.