<< Back to previous view

[NREPL-28] Clarify semantics for String encoding Created: 21/Aug/12  Updated: 11/Oct/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Comments   
Comment by Chas Emerick [ 21/Aug/12 8:27 PM ]

Initial discussion held in NREPL-11.





[NREPL-53] Middleware linearization is nondeterministically wrong Created: 02/May/14  Updated: 19/Nov/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Gary Fredericks Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None
Environment:

clojure 1.6.0


Attachments: Text File NREPL-53.patch    

 Description   

I've had trouble getting my middleware ordered correctly, and have wittled it down to the following example which seems to only fail in some processes. I.e., it is deterministic within a particular jvm, but across jvms the result can change. My guess is that this has to do with hashing vars.

This code prints true (signifying a correct result) roughly 1/3 of the time and false otherwise.

(ns chas.core
  (:require [clojure.tools.nrepl.middleware :as mid]
            [clojure.tools.nrepl.middleware.pr-values]
            [clojure.tools.nrepl.middleware.session]))

(def wrap-foo identity)
(mid/set-descriptor! #'wrap-foo
                     {:expects #{#'clojure.tools.nrepl.middleware.session/session "eval"}})

(def wrap-bar identity)
(mid/set-descriptor! #'wrap-bar
                     {:requires #{#'clojure.tools.nrepl.middleware.session/session},
                      :expects #{"describe" #'clojure.tools.nrepl.middleware.pr-values/pr-values},
                      :handles {"stacktrace" {}}})

(defn -main
  []
  (println
   ;; should return true, since #'wrap-foo belongs later in the stack than
   ;; #'clojure.tools.nrepl.middleware.session/session due to its :expects
   ;; clause
   (->> [#'wrap-foo
         #'wrap-bar
         #'clojure.tools.nrepl.middleware.session/session]
        (mid/linearize-middleware-stack)
        (filter #{#'clojure.tools.nrepl.middleware.session/session #'wrap-foo})
        (= [#'clojure.tools.nrepl.middleware.session/session #'wrap-foo]))))

Leiningen makes it difficult to use forked versions of nrepl, so as a hackier workaround until this issue is resolved I've created this Leiningen plugin.



 Comments   
Comment by Gary Fredericks [ 02/May/14 10:35 PM ]

Interesting followup observation I made with Alan Malloy in IRC: if we remove "eval" from the :expects clause, it seems to return consistently true. On the other hand if we instead add #'clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval to the middleware list, suddenly it returns consistently false.

Comment by Gary Trakhman [ 04/Jun/14 3:35 PM ]

I should add my own experience report with this, hope to dig in to it someday soon: https://groups.google.com/d/msg/clojure/nUBBbYZHuTE/ScLBH-A2HkoJ

In my case, it was incorrectness, not indeterminism.

Comment by Chas Emerick [ 19/Aug/14 1:59 PM ]

Half of the middleware linearization stuff is too clever, the other half is really dumb. :-/

I'd love a patch for this. Absent one, I'll have to dig into it, but I'm not going to hold up 0.2.4 any further for that opportunity to come around.

Comment by Gary Fredericks [ 19/Aug/14 2:12 PM ]

Is this a fair summary of the requirements?

1) Should be deterministic
2) Should only return a result that respects the :expects and :requires clauses – if this is impossible it should throw an exception explaining why

Comment by Chas Emerick [ 19/Aug/14 2:58 PM ]

Yes, that's fair. I might add (3) try to not break anything significant that might rely upon undocumented behaviour, but I doubt that's helpful.

Comment by Gary Fredericks [ 25/Sep/14 9:29 PM ]

I've written generative tests for this, but test.check requires clojure 1.4 (while nrepl tests run on 1.2).

If I wrap them in code that NOOPs for old clojures, I imagine they will they get run in CI? Is that a reasonable approach?

Comment by Colin Jones [ 26/Sep/14 7:50 AM ]

That approach sounds great to me.

Comment by Gary Fredericks [ 26/Sep/14 10:06 AM ]

Given I've coded it that way, is there an easy way for me to run the tests without tweaking the pom.xml to specify 1.4?

Comment by Gary Fredericks [ 30/Sep/14 10:11 PM ]

Okay I think I finally tracked this down.

Basically the conj-sorted function was not a valid way of doing a topological sort.

I replaced it with an actual topological sort, that still pushes independent middlewares toward the end.

The generative tests are now passing, as are the rest of the tests.

At some point I'll circle back to my original problem and see if that's solved.

I still don't fully understand all of the linearization code, so it's possible that the problem is more subtle than this.

The fix is on a github branch here. I might want to wait on another release of test.check before making a patch, since it should have some new features that make the generative tests better (in particular I'd be able to remove the dependency on my test.chuck utility library, which obviously shouldn't be in the tools.nrepl codebase).

Comment by Gary Fredericks [ 04/Oct/14 10:07 AM ]

Attached a patch that contains the aforementioned topological sort, with a single regression test (and no generative tests).

Things to note:

  • I created a wrapper for clojure.core/ex-info so that in the case of a dependency cycle we can include with the exception message an example cycle, but don't have to assume clojure 1.4. If assuming clojure 1.4 is okay I can take that out and use clojure.core/ex-info directly.
  • I tried adding a test to verify that a dependency cycle creates an exception, but the test failed, and when I dug into it I discovered that the extend-deps function was doing strange things I did not expect, which means either it is buggy or I do not understand what it is supposed to do. See this commit for the failing test.
  • The topological sort now explicitly pushes middlewares with no dependencies to the end of the stack, instead of the more implicit fashion in the older code.
  • This patch does fix the problem I described in the description




[NREPL-3] Adopt default port Created: 30/Sep/11  Updated: 13/Jun/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 2
Labels: None


 Description   

7888 is "free", at least in IANA.

Most users want to put an nREPL port on their app, and that should always be on a typical port. Auto-selection of a port is only desirable in tooling scenarios, and tooling authors can pass an argument to start-server.

This will change the behaviour of (start-server).



 Comments   
Comment by Bozhidar Batsov [ 13/Jun/14 9:55 AM ]

Sounds like a good idea.





[NREPL-4] Provide sane multiplexing of output in the face of multithreaded, asynchronous operation Created: 30/Sep/11  Updated: 12/Nov/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Basically, (send-off some-agent println) & co. should get what's printed to out in that agent's thread back to the nREPL client — not silently let it dump out to System/out.

Portal ostensibly does this well. Examine their approach, see if it is compatible with nREPL's objectives.

Ill-formed brain dump:

  • multiplex new out's to System/out
    • (still won't solve clojure.test/test-out content will disappearing into the ether when it's loaded when out is bound to an nREPL out; maybe we should ensure out is bound to System/out while code is being loaded?)
  • optionally multiplex System/out and System/err
  • optionally join multiplexed S/out and S/err, receive :stdout, :stderr msgs


 Comments   
Comment by Federico Ragona [ 12/Nov/14 1:59 PM ]

Seems to be fixed, I was unable to reproduce the issue using

REPL-y 0.3.5, nREPL 0.2.6
Clojure 1.6.0

Should this issue be closed?





[NREPL-15] Allow clients to specify an ID for newly-retained sessions Created: 29/Mar/12  Updated: 12/Oct/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

...particularly important where less sophisticated clients and polling-oriented transports are involved (e.g. HTTP).






[NREPL-24] :session key is overloaded Created: 10/Aug/12  Updated: 03/Oct/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Hugo Duncan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The session middleware takes the :session key from the message, and uses it as an id to lookup session data, which it then places on the same key.

Using a separate key for the session id and the session data would be less confusing, and would allow easier checking of the availability of the session data in the message map.



 Comments   
Comment by Chas Emerick [ 10/Aug/12 11:26 PM ]

The session id will need to continue to come in in the :session slot, as I don't want to break clients (and, there's very, very few middlewares out there yet).

Suggestions for names for the actual session atom? :the-session? :-/





[NREPL-29] Provide a mechanism for overriding an operation Created: 09/Sep/12  Updated: 18/Apr/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.0-beta9, 0.2.0-beta10
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Hugo Duncan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When specifying middleware, it would be much easier for the user to be able to override a default middleware without having to specify a handler.

For example, if there is a default middleware providing the "complete" operation, the user should be able to just specify their preferred completion middleware, without having to specify all middleware as a handler.

One way to do this might be to check metadata for the provided operations of the specified metadata, and ensure that either the default middleware for that operation is removed, or that the specified middleware takes precendence (which may be simpler when a middleware provides multiple operations).



 Comments   
Comment by Chas Emerick [ 14/Sep/12 7:44 AM ]

Agreed.

Just making sure that the order in which additional middlewares are provided is taken as a default stack order will suffice for most use cases. Transforming middlewares (either in full or in part) would need another metadata slot, :replace perhaps, though it seems like that would be much more difficult to get right.





[NREPL-33] Consider making session and eval functionality more accessible Created: 08/Oct/12  Updated: 18/Nov/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.0-beta9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Hugo Duncan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

ritz re-uses some of nrepl's private functions to avoid duplication. The uses are listed below.

Would it be possible to make these functions public? More subjectively, it might also be worth considering factoring out the session functionality into it's own namespace (including msg and possibly queue-eval), so the functionality is not split across the session middleware and the interruptible-eval middleware.

The debug nrepl server:
https://github.com/pallet/ritz/blob/develop/nrepl/src/ritz/nrepl.clj#L189

This uses clojure.tools.nrepl.middleware.session/create-session and clojure.tools.nrepl.middleware.session/session-out

ritz provides an eval op that tracks source forms:
https://github.com/pallet/ritz/blob/develop/nrepl-middleware/src/ritz/nrepl/middleware/tracking_eval.clj

This uses clojure.tools.nrepl.middleware.interruptible-eval/queue-eval and clojure.tools.nrepl.middleware.interruptible-eval/configure-executor






[NREPL-59] Tracking source form positions in eval Created: 19/Jun/14  Updated: 25/Nov/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Bozhidar Batsov Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: enhancement


 Description   

Dev tools writers using nREPL's eval op face the following problem - using the `eval` op you cannot pass information regarding the filename and the file position of the form you're evaluating and therefore afterwards you can't use find its definition. Because of this most CIDER users reload their source buffers all the time with `load-file`. Another implication of this problem is that exception backtraces have meaningless information about the position of the problem (usually something like `NO_FILE:1`). I've noticed that LightTable has a custom middleware for this, ritz used to have one as well, probably others too. Now I'm contemplating doing something similar for CIDER, but it seems to me this is basic functionality that should be supported out-of-the-box.

I'd like to propose that the default eval middleware be augmented to support some form of source form tracking to spare tool writers from having to reinvent the wheel.



 Comments   
Comment by Chas Emerick [ 19/Aug/14 1:50 PM ]

I agree that this should be baked in. `:file`, `:line`, and `:column` are some optional slots eval could accept (but should not require), with the obvious effect on the reader passed along to the compiler. This may be trickier than it sounds, especially if you're aiming to keep things at parity for both Clojure and ClojureScript (as code to be eval'd is read well before it hits interruptible_eval, in piggieback).

In any case, patch quite welcome.

Comment by Bozhidar Batsov [ 22/Aug/14 2:57 PM ]

`:filepath` should be another optional slot. I don't do much ClojureScript (and I don't know much about piggieback) so getting something working for Clojure would be a good enough start for me. I'll try to cook some patch soon.

Comment by Bozhidar Batsov [ 03/Sep/14 9:58 AM ]

Chas, can you give me some pointers about how to approach the task. I don't think we can use `clojure.main/repl` for this and I'm not sure what makes sense and what doesn't. One implementation of the idea can be found here https://github.com/pallet/ritz/blob/develop/nrepl-middleware/src/ritz/nrepl/middleware/tracking_eval.clj but I'm not sure whether it's good or bad.

Seems to me there should be a simpler way to provide a context for the evaluation, but I'm far from an expert in the area.

Comment by Chas Emerick [ 03/Sep/14 12:45 PM ]

Now I remember what I was planning the last time I thought about it: the easiest way AFAICT would be to use load-file, and simply pad out the top-level form being evaluated. So if you had this file:

(ns foo.bar
  (:require clojure.set))

(defn x [] 5)

    (defn y [] 6)

and the user loads `y`, you'd send this in a load-file message with appropriate file path information:

(ns foo.bar)




    (defn y [] 6)

load-file turns this into a call like (clojure.lang.Compiler/load "(ns foo.bar)\n\n\n\n\n (defn y [] 6)" "foo/bar.clj" "bar.clj". That doesn't change *ns*, and gives you proper metadata.

I've only skimmed Ritz's implementation, but it does a lot of clever things, and is tied to Clojure. Perhaps there's reasons why my (perhaps hacky?) approach isn't suitable in certain situations, but it should be pretty future-proof, and should work transparently with ClojureScript/piggieback. I'd rather dummy up "files" to be loaded than overriding LineNumberingPushbackReader method impls, etc.

Comment by Bozhidar Batsov [ 04/Sep/14 8:07 AM ]

Your idea sounds great to me. I guess the only question is whether the clients should take care of this or nREPL when passed extra arguments to the eval op.

Comment by Chas Emerick [ 04/Sep/14 10:08 AM ]

All of the source-file-aware stuff is in load-file, so I'm inclined to put the burden on the client. Even if an op accepted the necessary info, the client would still need to provide it (target namespace + line and column information); going from there to a dummied-up ns form and the corresponding whitespace is so straightforward, I'd rather not complicate the op interface(s).

Comment by Bozhidar Batsov [ 04/Sep/14 10:15 AM ]

Roger that! I've just implemented this in cider and it works great. I'll be closing this. Thanks for all your help!

Comment by Bozhidar Batsov [ 04/Sep/14 10:27 AM ]

P.S. You should probably mention this technique in the README for posterity's sake.

Comment by Bozhidar Batsov [ 04/Sep/14 10:27 AM ]

P.S. You should probably mention this technique in the README for posterity's sake.

Comment by Chas Emerick [ 04/Sep/14 10:33 AM ]

True. Really, the repo should have a separate file for implementors, as there's a lot of stuff in the source and tribal knowledge that isn't documented anywhere....

Comment by Bozhidar Batsov [ 09/Sep/14 9:17 AM ]

Btw, looking at this ticket https://github.com/clojure-emacs/cider/issues/781 I'm not sure we can use just (ns ns-name) at the start of the dummy files as it seems this overrides the namespace definition. Perhaps we should always insert the complete ns declaration from the source file in question? Or is there some other smart trick I'm not aware of that'll help avoid this ns clobbering.

Comment by Chas Emerick [ 09/Sep/14 2:00 PM ]

load-file only changes *ns* for the duration of the loading process. If it clobbered *ns* otherwise, then loading entire files would change it in the REPL (which doesn't happen)…

Comment by Bozhidar Batsov [ 10/Sep/14 9:28 AM ]

Are we talking about the same thing? My concern is that swapping:

(ns foo
   (:require [bar :as b]))

with

(ns foo)

changes the ns definition (not the ns itself) in a manner in which you're no longer referring to bar as b anymore.

Comment by Vitalie Spinu [ 16/Sep/14 4:36 AM ]

Another solution is to allow the `eval` middleware to accept a token that will be appended to the temp file that eval uses for evaluation. Then the client can split the file path and retrive the metadata back. You can even allow for a full path of a temp file as an argument. This should be trivial to implement.

Comment by Bozhidar Batsov [ 16/Sep/14 4:45 AM ]

The problem with the solution you propose is that *eval* is implemented in terms of Clojure's built *clojure.main/repl* and we obviously can't change it to suit our needs. Don't know about you, but I definitely don't want *eval* to have to be implemented in terms of some custom evaluation logic developed specially for nREPL.

The current solution works well enough and requires zero middleware changes. The only question is does the temp file need the complete ns declaration of the original source file or not.

Comment by Vitalie Spinu [ 16/Sep/14 4:56 AM ]

I have just looked at the code and it looks like you are right. I am now confused. Where do "tmp/form-init34343" files come from? I thought they are handled in nREPL.

Comment by Bozhidar Batsov [ 16/Sep/14 5:00 AM ]

It comes from Clojure itself (or rather its built-in REPL evaluation). These tmp filenames appeared in Clojure 1.6. In older Clojure releases the var metadata had just *NO_FILE* for *:file*.

Comment by Chas Emerick [ 16/Sep/14 8:03 AM ]

ns declarations are strictly additive AFAICT:

user=> (ns foo (:require [clojure.set :as set]))
nil
foo=> set/intersection
#<set$intersection clojure.set$intersection@3e4db6a>
foo=> (ns foo)
nil
foo=> set/intersection
#<set$intersection clojure.set$intersection@3e4db6a>

And using load-file, which is effectively what the load-file op duplicates:

chas@t440p:~$ cat > foo.clj
(ns foo (:require [clojure.set :as set]))
chas@t440p:~$ cat > foo2.clj
(ns foo)
chas@t440p:~$ lein repl
nREPL server started on port 40749 on host 127.0.0.1 - nrepl://127.0.0.1:40749

user=> (load-file "foo.clj")
nil
user=> (in-ns 'foo)
#<Namespace foo>
foo=> set/intersection
#<set$intersection clojure.set$intersection@2fcecc1>
foo=> (in-ns 'user)
#<Namespace user>
user=> (load-file "foo2.clj")
nil
user=> (in-ns 'foo)
#<Namespace foo>
foo=> set/intersection
#<set$intersection clojure.set$intersection@2fcecc1>

The load-file op appears to work properly outside of this use case as well, since I can (from cider) remove a :require clause from a file, reload it, and its alias remains in place.

If you seem to need to include the full ns declaration, then something else is amiss somewhere...

Comment by Bozhidar Batsov [ 16/Sep/14 9:51 AM ]

I couldn't reproduce the problem some people are reporting, so I'm inclined to believe you. I've just merged a patch that uses the full ns declarations all the time, but I may revert it in light of your comment. I wonder if this behaviour changes in different Clojure versions.

Comment by Chas Emerick [ 16/Sep/14 9:58 AM ]

Using the full ns declaration isn't necessarily bad, but there is the edge case of people who add :reload and/or :reload-all options to their :require, etc clauses. In such cases, sending a single top-level expr to be loaded will then cause other namespaces to be reloaded, perhaps with side effects (if those other namespaces have impolite top-level forms themselves).

Re: version differences, I'm pretty sure that's not an issue either:

chas@t440p:~/dev/pdfts$ java -cp ~/Downloads/clojure-1.0.0.jar clojure.main
Clojure 1.0.0-
user=> *clojure-version*
{:major 1, :minor 0, :incremental 0, :qualifier ""}
user=> (load-file "foo.clj")
nil
user=> (in-ns 'foo)
#<Namespace foo>
foo=> set/intersection
#<set$intersection__5909 clojure.set$intersection__5909@5b0abc94>
foo=> (in-ns 'user)
#<Namespace user>
user=> (load-file "foo2.clj")
nil
user=> (in-ns 'foo)
#<Namespace foo>
foo=> set/intersection
#<set$intersection__5909 clojure.set$intersection__5909@5b0abc94>

I could believe that there's some bug/oddity with the implementation of the load-file op in this case, but the underlying mechanism is stable and behaves non-destructively.

Comment by Bozhidar Batsov [ 22/Sep/14 12:22 AM ]

So, we've finally tracked down the problem - load-file returns for "ns" either "user" or ":main" for some reason, regardless of the namespace of the file you're loading. This sounds like a bug to me; I'll find another ticket.

Comment by Bozhidar Batsov [ 30/Sep/14 11:20 AM ]

We found one more related problem. Seems that load-file in piggieback discards the return value of the evaluation. More details can be found here. I think the behaviour for Clojure and ClojureScript load-file should be unified.





[NREPL-67] nrepl-server reuseAddr Created: 19/Sep/14  Updated: 19/Sep/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.5
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Sun Ning Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

We are using nrepl-server widely in our application. A pain with nrepl-server is it's not configured to reuseAddr, so we often failed to restart server due to port conflicting.

So I hope you can enable reuseAddr by default for the server socket. Thanks!






[NREPL-68] load-file op responses always have "user" or ":main" as the value of their ns slot Created: 22/Sep/14  Updated: 23/Sep/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.5
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Bozhidar Batsov Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Regardless of the ns of the file being loaded by load-file, its response always has "user" or ":main" as the value of its ns slot. Seems much more logical to me to return the ns of the file that was loaded.

On a related note - the return value for this op does not appear in the op reference.



 Comments   
Comment by Chas Emerick [ 22/Sep/14 4:58 AM ]

load-file reuses the default handler for eval, so the returned namespace is going to be *ns*. I just confirmed this by fiddling with one of nREPL's load-file tests.

Perhaps cider is not sending the session ID on load-file messages?

Comment by Bozhidar Batsov [ 22/Sep/14 5:05 AM ]

Seems to me we're sending it. The relevant code is here.

Comment by Kevin Downey [ 23/Sep/14 7:21 PM ]

the compiler binds and pops the value of ns when loading, so by the time eval captures the value of ns the compiler has already loaded the file and popped the value.





[NREPL-69] Interrupt of load-file generates java.lang.ThreadDeath exception Created: 02/Nov/14  Updated: 02/Nov/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.6
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Vitalie Spinu Assignee: Chas Emerick
Resolution: Unresolved Votes: 1
Labels: errormsgs, interop, stacktrace
Environment:

Clojure 1.6.0, (both CIDER and lein's REPL-y 0.3.5). No problem with Clojure 1.5.1.



 Description   

At lein prompt (load-file "src/test/core.clj") where core.clj contains (Thread/sleep 10000) then C-c to interrupt.

{{
CompilerException java.lang.ThreadDeath, compiling/home/vitoshka/tmp/cidertest/src/blabla/core.clj:1:17)
clojure.lang.Compiler.load (Compiler.java:7142)
clojure.lang.Compiler.loadFile (Compiler.java:7086)
clojure.lang.RT$3.invoke (RT.java:318)
user/eval3418 (form-init8959987907704530559.clj:1)
clojure.lang.Compiler.eval (Compiler.java:6703)
clojure.lang.Compiler.eval (Compiler.java:6666)
clojure.core/eval (core.clj:2927)
clojure.main/repl/read-eval-print-6625/fn-6628 (main.clj:239)
clojure.main/repl/read-eval-print--6625 (main.clj:239)
clojure.main/repl/fn--6634 (main.clj:257)
clojure.main/repl (main.clj:257)
clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn--594 (interruptible_eval.clj:67)
Caused by:
ThreadDeath
java.lang.Thread.stop (Thread.java:836)
clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn--636 (interruptible_eval.clj:204)
clojure.tools.nrepl.middleware/wrap-conj-descriptor/fn--408 (middleware.clj:17)
clojure.tools.nrepl.middleware.pr-values/pr-values/fn--573 (pr_values.clj:17)
clojure.tools.nrepl.middleware/wrap-conj-descriptor/fn--408 (middleware.clj:17)
clojure.tools.nrepl.middleware.load-file/wrap-load-file/fn--751 (load_file.clj:77)
clojure.tools.nrepl.middleware/wrap-conj-descriptor/fn--408 (middleware.clj:17)
clojure.tools.nrepl.middleware.session/add-stdin/fn--710 (session.clj:235)
clojure.tools.nrepl.middleware/wrap-conj-descriptor/fn--408 (middleware.clj:17)
cider.nrepl.middleware.test/wrap-test/fn--3003 (test.clj:199)
clojure.tools.nrepl.middleware/wrap-conj-descriptor/fn--408 (middleware.clj:17)
cider.nrepl.middleware.stacktrace/wrap-stacktrace/fn--2909 (stacktrace.clj:146)
}}

In CIDER, eval the buffer with C-c C-k which uses load-file op and then C-c C-b to interrupt. The relevant CIDER issue is here.






[NREPL-36] Too many DynamicClassLoaders created Created: 10/Dec/12  Updated: 18/Apr/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.0-RC1, 0.2.0-beta9, 0.2.0-beta10
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Colin Jones Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Not sure whether this ticket belongs here or in the Clojure-proper JIRA, so feel free to close if this is an inappropriate location. clojure.main/repl creates a new DynamicClassLoader on every execution, so it looks like the stack of classloaders grows without bounds. Seems a bit similar to http://dev.clojure.org/jira/browse/NREPL-31 in that clojure.main/repl has another assumption about when clojure.main/repl will run.

See https://groups.google.com/forum/?fromgroups=#!topic/clojure/firG9zTVecU%5B1-25%5D for the original report.



 Comments   
Comment by Chas Emerick [ 18/Apr/14 10:17 AM ]

Yes, this should be fixed upstream; a new DynamicClassLoader should only be set as the thread-context classloader if one is not already in place.

My first impulse upon seeing this was that this was "last straw" territory w.r.t. using clojure.main/repl (as recorded in the thread linked above), but the work necessary to stop depending upon it would be considerable (not because repl does much, but because interruptible-eval/evaluate is structured to cater to it). Some years on, and it's clear that this fundamentally a minor problem (insofar as hardly anyone has complained AFAIK), so a fix that requires e.g. Clojure 1.7.0 would be fine.





[NREPL-43] Document the availability/usage of *e, *1, *2, ... in nREPL Created: 04/Jul/13  Updated: 24/Aug/13

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Jakub Holy Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: documentation


 Description   

I have only by chance discovered that nREPL binds the lat error/outputs to the vars *e, *1 etc. This should be documented clearly somewhere, possibly in https://github.com/clojure/tools.nrepl/blob/master/README.md

When I have forgotten the names of the vars while remembering that something like them exists, I tried to google them out but failed. So better documentation would help.

Thank you!



 Comments   
Comment by Chas Emerick [ 04/Jul/13 7:07 AM ]

The role of those vars is actually the same across all Clojure REPLs. In this department, nREPL is simply following Clojure's lead.

That said, yes, there is room to specify fully what nREPL's behaviour is, beyond the implied equivalence (at a minimum) to Clojure's included console REPL.

Comment by Jakub Holy [ 22/Jul/13 7:37 AM ]

Thank you for the clarification!

> The role of those vars is actually the same across all Clojure REPLs.

Do you know if this behavior of all Clojure REPLs is documented anywhere? And yes, it would be nice if the nREPL documentation linked to this doc and/or it printed a short summary and/or link when starting (in addition to the currently provided info about (source) etc.)

Thanks!

Comment by Chas Emerick [ 22/Jul/13 7:52 AM ]

REPL-bound vars are documented in a variety of places, though nowhere "official" AFAIK. We talked about it in Chapter 10 of Clojure Programming FWIW (I'm certain other books and online resources cover these vars as well, but the CP citation is the only one I have close at hand.)

FYI, the "currently-provided info" you mention is emitted by Leiningen/Reply, not nREPL.

Comment by Jakub Holy [ 24/Aug/13 4:50 AM ]

Thanks a lot, Chas, that was helpful. I have submitted a patch to Leiningen to include the info it its REPL' welcome message: https://github.com/technomancy/leiningen/pull/1310

Comment by Jakub Holy [ 24/Aug/13 5:38 AM ]

I have published a blog post about this, Clojure REPL stores the latest results in *1, *2, *3, exception in *e, to make it more googlable (is that even a word? ).

The top hit for "Clojure REPL" seems to be http://clojure.org/repl_and_main, so it perhaps should be documented there or it should link to a more detailed documentation. Not sure how to make that happen I have also checked http://clojure-doc.org/ but there doesn't seem to be a suitable place to add this info either. Perhaps it should be mentioned in the docstring of clojure.main/repl and the Clojure page should link to it?

Comment by Jakub Holy [ 24/Aug/13 5:52 AM ]

I have created an issue under Clojure itself, #CLJ-1247, so this can likely be closed.





[NREPL-46] nREPL crashes when required more than one time with :reload-all Created: 21/Dec/13  Updated: 25/Aug/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.2
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Alex Fowler Assignee: Chas Emerick
Resolution: Unresolved Votes: 1
Labels: bug
Environment:

Irrelevant, but the provided test case project is for CounterClockWise on Eclipse. Although it should run just fine with vanilla lein.


Attachments: Zip Archive nrepl-test.zip    

 Description   

When the namespace "clojure.tools.nrepl.server" is required more than once with :reload-all option, nREPL crashes. Accrding to current info it occures because some protocol instances get re-evaluated and are no longer the same JVM classes as they were before reload-all.

Steps to reproduce the bug in CCW:
1) Import the project into the workspace
2) Go into the core.clj
3) Click Clojure -> Load file in REPL
4) After the evaluation is complete, try evaluating (+ 1 2) in the repl. You should see no response at all.
5) Try writing something with letters, like "nrepl". The autocompletion will try and freeze and fail with the above exception.
6) Play around more to get a full hang or kill the REPL/Java to revive Eclipse.

Upon trying CCW autocompetion, the following exception occures, what might give some hint as to why:

Exception in thread "nREPL-worker-2" java.lang.IllegalArgumentException: No implementation of method: :send of protocol: #'clojure.tools.nrepl.transport/Transport found for class: clojure.tools.nrepl.middleware.pr_values$pr_values$fn$reify__1283
at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:541)
at clojure.tools.nrepl.transport$eval7291$fn_7292$G7280_7299.invoke(transport.clj:16)
at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn_7726$fn_7739.invoke(interruptible_eval.clj:75)
at clojure.main$repl$fn__6597.invoke(main.clj:279)
at clojure.main$repl.doInvoke(main.clj:277)
at clojure.lang.RestFn.invoke(RestFn.java:1096)
at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__7726.invoke(interruptible_eval.clj:56)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:617)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1788)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:41)
at clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn_1345$fn_1348.invoke(interruptible_eval.clj:171)
at clojure.core$comp$fn__4154.invoke(core.clj:2330)
at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__7760.invoke(interruptible_eval.clj:138)
at clojure.lang.AFn.run(AFn.java:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)



 Comments   
Comment by Chas Emerick [ 19/Aug/14 8:56 PM ]

Confirmed. Using lein repl (though you're right that the particulars of the tooling shouldn't matter):

user=> 3
3
user=> (require '[clojure.tools.nrepl.server] :reload-all)
nil
user=> 2

user=> 1

Not sure if it's the protocol or the types that are implicated (or both), but resolving this is going to be unpleasant.

Comment by Chas Emerick [ 19/Aug/14 8:57 PM ]

Couple of questions:

  • Does the fact that this occurs matter to anyone in a meaningful way?
  • Would an acceptable solution be that reloading of the affected namespaces be no-ops?
Comment by Alex Fowler [ 25/Aug/14 9:34 AM ]

Well, in my case I have a project which is required to often reload various namespaces because they are updated in the background. If it happens that any of the namespaces refer to nrepl, the thing crashes.

The error happens due to implementation problems and is not "a correct error" from the POV of Clojure language so it is better to be fixed. However, of course, in the real world it could be rather hard to do so..

Chas, could you please explain more about the no-ops approach? Do you mean it is possible to somehow force loading nrepl just once?

Comment by Chas Emerick [ 25/Aug/14 10:20 AM ]

Yes, there are ways to prevent the re-definition of types and protocols, but I'd like to avoid doing so. It's a hack, and would be an impediment to developing nREPL itself.

Actually, you should be able to accomplish the same thing more easily in your project: rather than require nREPL namespaces in the ns form, use a top-level require form, wrapped in a suitable conditional to determine if nREPL has been loaded yet:

(when-not (resolve 'clojure.tools.nrepl/version)
  (require 'clojure.tools.nrepl))




[NREPL-48] *1/*2/*3/*e nil in cloned session Created: 02/Jan/14  Updated: 02/Jan/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Tim Pope Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Summary says it all. This isn't blocking me; just something I stumbled onto.






[NREPL-54] No version of nrepl when connect to the lein-droid REPL Created: 04/May/14  Updated: 31/May/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Eugene Apollonsky Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None
Environment:

tools.nrepl/0.2.3
lein-droid/0.2.3



 Description   

1. Connect Android device.
2. Start REPL.
lein droid run.
3. Forward port.
lein do droid forward-port, droid repl.
4. Send command {'op': 'describe', 'verbose?': 1}

Result:
...
'versions': { 'clojure': { 'incremental': 0, 'major': 1, 'minor': 6, 'qualifier': 'RC1'},
'nrepl': { }}}



 Comments   
Comment by Chas Emerick [ 05/May/14 11:51 AM ]

The current nREPL version is driven by reading the resource at /clojure/tools/nrepl/version.txt. If it's not found (perhaps because of an android packaging / classloader detail I'm not familiar with?), then no nREPL version will be returned.

FWIW, the version indicator is meant to be strictly advisory, and not really depended upon for e.g. exposing tooling capabilities. What are you needing the version indicator for?

Comment by Eugene Apollonsky [ 05/May/14 1:34 PM ]

Vim plugin "vim-fireplace" has nREPL version check (>0.2.0). OK, I think it can be fixed in the plugin.

Comment by Chas Emerick [ 05/May/14 3:29 PM ]

Yeah, that's likely there to guard against nREPL "classic" (which met its demise ~ 0.0.5). But, that's many years old at this point.

A more interesting question is: why isn't the version file being found by nREPL at load-time? I presume it's an Android-specific idiosyncrasy, but I'm curious nonetheless.

Comment by Eugene Apollonsky [ 31/May/14 4:39 PM ]

lein-droid developer commented this issue. It is Android-specific.
https://github.com/clojure-android/lein-droid/issues/92





[NREPL-55] Support custom value rendering middleware Created: 13/May/14  Updated: 13/May/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Gregory Look Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently, nREPL's interruptible-eval middleware hardcodes a dependency on pr-values. It would be nicer if the user could either specify alternate rendering middleware or if nREPL's built-in middleware accepted an arbitrary function to render values with. Specifically, this would enable pretty-printing REPL output.

I've set up a middleware to demonstrate this, in combination with the Puget printing library:
https://github.com/greglook/whidbey

Currently it is possible to replace the default middleware, but it involves some ugly runtime metadata manipulation which reaches into the nREPL internals. Addressing this would be another step towards simplifying pretty-printing/color integration in the REPL.






[NREPL-64] Add current ns to describe session's response Created: 24/Aug/14  Updated: 08/Sep/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.4
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Bozhidar Batsov Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: enhancement


 Description   

In CIDER the REPL buffer gets created on receiving a describe-session response from the nREPL session that will back the REPL buffer. There's a problem with determining the default ns for the REPL buffer as many people use nREPL servers started with lein (lein repl :headless) and expect the `:init-ns` option for their project to the honored, but there's no easy way for clients to know what the default namespace is. It'd be great if this information was returned by `describe-session`.

Perhaps clients should be able to create a session with a particular default namespace to avoid the need for code like https://github.com/technomancy/leiningen/blob/554861505c23763d6583fe3b1f236a08ad02a4ca/src/leiningen/repl.clj#L103. I noticed that current the default ns is hardcoded as `user`.



 Comments   
Comment by Chas Emerick [ 25/Aug/14 8:59 PM ]

As you say, the default nREPL namespace is always user, so that's easy to know.

Leiningen's :init-ns support is implemented sanely AFAICT, given what it's aiming to accomplish. There's lots of vars in the session that one might want to customize on a per-project basis, and that tools might want to know the value of. I don't see any reason to special-case *ns*. This is one of the rare times where I'd say that a tool's best option is to use eval.

Comment by Bozhidar Batsov [ 27/Aug/14 10:09 AM ]

While I'd normally agree, the problem is that I'd have to rework cider's init in a bit awkward manner:

  • run describe session to obtain the REPL capabilities and version info
  • its callback should then do eval for ns
  • its callback should then spawn the REPL buffer

While I can certainly do this, chaining callbacks just to get some bootstrap info seems suboptimal to me. Ideally all bootstrap information
should be obtainable with a single request.
You can see the code in question here https://github.com/clojure-emacs/cider/blob/master/nrepl-client.el#L434

I can't imagine any other var value that a similar tool would like to know while bootstrapping. Obviously once you're up and running using eval is the natural solution.

Comment by Chas Emerick [ 05/Sep/14 8:05 AM ]

Not being thrilled with the idea of special-casing *ns*, I thought we might instead add a way for any middleware to contribute to the result of describe via a function in an optional descriptor slot. Each of them can be called here, and their results merged into the describe middleware's response. That function will need to take the requesting message (this will give the session middleware what it needs to provide *ns*, for example).

Happy to take a patch before I get around to this myself.

Comment by Bozhidar Batsov [ 05/Sep/14 8:25 AM ]

Sounds good to me. I guess the functions in question should be passed fully-qualified and have zero arity, right?

Comment by Chas Emerick [ 05/Sep/14 8:28 AM ]

I don't think they need to be named at all; e.g. session's :describe fn would just go here. They have to accept at least the request message as received/provided by the describe middleware.

Comment by Bozhidar Batsov [ 08/Sep/14 5:07 AM ]

Ah, I misinterpreted your previous comment. I thought you meant that we'd pass extra arguments to the describe request to get more things in the response. I think that now I understand the idea. I might take a stab at implementing this.





[NREPL-44] Expose JMX MBean to provide list of available nREPL endpoints Created: 06/Aug/13  Updated: 19/Aug/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

In a development environment, one often has many nREPL servers running, sometimes more than one per project. This means that it's sometimes difficult to keep track of which ports are associated with which projects / processes, and existing per-project REPL tracking mechanisms (e.g. Leiningen/reply's repl-port file) are generally not up the job.

It would be great if nREPL exposed a JMX MBean that provided information on each active nREPL server, what URL(s) they'll respond to, and perhaps other details (what middleware is installed on each, etc). Clients could then use JMX to discover all of the nREPL servers on a given host, and present that information in tool-appropriate 'connect' dialogs, etc.



 Comments   
Comment by Alan Effrig [ 08/Aug/13 7:50 PM ]

How do you envision clients discover which processes have nREPL servers? How would it work in practice from the client's point of view? Would the client be responsible for knowing the JMX port & associated configuration? Do you envision this feature applying mainly to clients on the local machine or to any number of hosts?

When I've worked with JMX, part of the battle is knowing the proper port and associated JMX configuration for each process so a client knows how to connect, especially with a variety of processes on a given host. It is even more problematic in the cloud with dozens of servers. And then there is the complication that locally one can sometimes use the Attach API, a mechanism that differs from how JMX is accessed remotely. At the risk of thinking too much, is there any other standard discovery mechanism that might be appropriate to share the JMX port & configuration with potential clients? Would some sort of multicast/cluster mechanism work here?





[NREPL-70] Add monroe to the list of nrepl clients Created: 04/Nov/14  Updated: 26/Nov/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Sanel Zukan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: documentation

Attachments: Text File NREPL-70-Add-monroe-to-the-list-of-nrepl-clients.patch    

 Description   

Hi,

Can you add monroe (https://github.com/sanel/monroe) to the list of nrepl clients in README.md, please?

Thanks!



 Comments   
Comment by Andrew Rosa [ 26/Nov/14 7:32 AM ]

Made a small patch to fix the documentation. I've also updated the CIDER description so be able to differentiate both solutions.





Generated at Thu Nov 27 07:15:19 CST 2014 using JIRA 4.4#649-r158309.