<< Back to previous view

[NREPL-59] Tracking source form positions in eval Created: 19/Jun/14  Updated: 19/Jun/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Bozhidar Batsov Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: enhancement


 Description   

Dev tools writers using nREPL's eval op face the following problem - using the `eval` op you cannot pass information regarding the filename and the file position of the form you're evaluating and therefore afterwards you can't use find its definition. Because of this most CIDER users reload their source buffers all the time with `load-file`. Another implication of this problem is that exception backtraces have meaningless information about the position of the problem (usually something like `NO_FILE:1`). I've noticed that LightTable has a custom middleware for this, ritz used to have one as well, probably others too. Now I'm contemplating doing something similar for CIDER, but it seems to me this is basic functionality that should be supported out-of-the-box.

I'd like to propose that the default eval middleware be augmented to support some form of source form tracking to spare tool writers from having to reinvent the wheel.






[NREPL-58] Inconsistency between session and eval responses Created: 04/Jun/14  Updated: 13/Jun/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.4
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jeff Valk Assignee: Chas Emerick
Resolution: Unresolved Votes: 1
Labels: bug, patch,

Attachments: Text File 0001-eval-session-consistency.patch    
Patch: Code

 Description   

At present, the interruptible-eval handler sends the :value or :eval-error status message before it updates the session to reflect the new binding result. This leaves the session values, particularly *e, *1, etc inconsistent with the state implied by those response messages.

As a practical example, if we want to wrap the "eval" op to check for an :eval-error and add error detail to the response from *e, (get @session #'*e) needs to hold the exception that resulted in the :eval-error. At present, however, it still holds the previous exception, leaving us no way to retrieve the current one in band.

The attached (minimal) patch makes ensures the session is updated before "eval" responses are sent.



 Comments   
Comment by Jeff Valk [ 04/Jun/14 1:00 PM ]

In the event that the attached patch seems a reasonable fix, I should probably add that my CA will be in the mail shortly.

Comment by Bozhidar Batsov [ 13/Jun/14 8:05 AM ]

I'm just here to pledge my support for Jeff's patch. Btw, Jeff, few days later and you could have signed the CA electronically.

Comment by Jeff Valk [ 13/Jun/14 9:32 AM ]

The irony did not escape me. Anyway, my CA (in dead tree form) is now on file.





[NREPL-56] Test suite failing on JDK 1.8 Created: 23/May/14  Updated: 23/May/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: 0.2.4

Type: Defect Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

e.g. http://build.clojure.org/job/tools.nrepl-test-matrix/154/CLOJURE_VERSION=1.6.0,jdk=Oracle%20JDK%201.8/console






[NREPL-53] Middleware linearization is nondeterministically wrong Created: 02/May/14  Updated: 04/Jun/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.3
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Gary Fredericks Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None
Environment:

clojure 1.6.0



 Description   

I've had trouble getting my middleware ordered correctly, and have wittled it down to the following example which seems to only fail in some processes. I.e., it is deterministic within a particular jvm, but across jvms the result can change. My guess is that this has to do with hashing vars.

This code prints true (signifying a correct result) roughly 1/3 of the time and false otherwise.

(ns chas.core
  (:require [clojure.tools.nrepl.middleware :as mid]
            [clojure.tools.nrepl.middleware.pr-values]
            [clojure.tools.nrepl.middleware.session]))

(def wrap-foo identity)
(mid/set-descriptor! #'wrap-foo
                     {:expects #{#'clojure.tools.nrepl.middleware.session/session "eval"}})

(def wrap-bar identity)
(mid/set-descriptor! #'wrap-bar
                     {:requires #{#'clojure.tools.nrepl.middleware.session/session},
                      :expects #{"describe" #'clojure.tools.nrepl.middleware.pr-values/pr-values},
                      :handles {"stacktrace" {}}})

(defn -main
  []
  (println
   ;; should return true, since #'wrap-foo belongs later in the stack than
   ;; #'clojure.tools.nrepl.middleware.session/session due to its :expects
   ;; clause
   (->> [#'wrap-foo
         #'wrap-bar
         #'clojure.tools.nrepl.middleware.session/session]
        (mid/linearize-middleware-stack)
        (filter #{#'clojure.tools.nrepl.middleware.session/session #'wrap-foo})
        (= [#'clojure.tools.nrepl.middleware.session/session #'wrap-foo]))))


 Comments   
Comment by Gary Fredericks [ 02/May/14 10:35 PM ]

Interesting followup observation I made with Alan Malloy in IRC: if we remove "eval" from the :expects clause, it seems to return consistently true. On the other hand if we instead add #'clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval to the middleware list, suddenly it returns consistently false.

Comment by Gary Trakhman [ 04/Jun/14 3:35 PM ]

I should add my own experience report with this, hope to dig in to it someday soon: https://groups.google.com/d/msg/clojure/nUBBbYZHuTE/ScLBH-A2HkoJ

In my case, it was incorrectness, not indeterminism.





[NREPL-44] Expose JMX MBean to provide list of available nREPL endpoints Created: 06/Aug/13  Updated: 08/Aug/13

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

In a development environment, one often has many nREPL servers running, sometimes more than one per project. This means that it's sometimes difficult to keep track of which ports are associated with which projects / processes, and existing per-project REPL tracking mechanisms (e.g. Leiningen/reply's repl-port file) are generally not up the job.

It would be great if nREPL exposed a JMX MBean that provided information on each active nREPL server, what URL(s) they'll respond to, and perhaps other details (what middleware is installed on each, etc). Clients could then use JMX to discover all of the nREPL servers on a given host, and present that information in tool-appropriate 'connect' dialogs, etc.



 Comments   
Comment by Alan Effrig [ 08/Aug/13 7:50 PM ]

How do you envision clients discover which processes have nREPL servers? How would it work in practice from the client's point of view? Would the client be responsible for knowing the JMX port & associated configuration? Do you envision this feature applying mainly to clients on the local machine or to any number of hosts?

When I've worked with JMX, part of the battle is knowing the proper port and associated JMX configuration for each process so a client knows how to connect, especially with a variety of processes on a given host. It is even more problematic in the cloud with dozens of servers. And then there is the complication that locally one can sometimes use the Attach API, a mechanism that differs from how JMX is accessed remotely. At the risk of thinking too much, is there any other standard discovery mechanism that might be appropriate to share the JMX port & configuration with potential clients? Would some sort of multicast/cluster mechanism work here?





[NREPL-40] Thread leak in clojure.tools.nrepl.transport$fn_transport? Created: 11/Mar/13  Updated: 18/Apr/14

Status: In Progress
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.1, 0.2.2
Fix Version/s: 0.2.4

Type: Defect Priority: Major
Reporter: David Lao Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows 7 x64, Oracle JDK 1.7.0.11 x64, clojure 1.4.0



 Description   

When trying out remote eval using your sample in the README, ie

(with-open [conn (repl/connect :port 59258)]
(-> (repl/client conn 1000) ; message receive timeout required
(repl/message {:op "eval" :code "(+ 2 3)"})
repl/response-values))

I'm noticing that hosting process leaking a thread each time the remote eval is called. Jconsole shows a clojure-agent-send-off-pool-xxx thread got spawn as result of the call. The stack appears to be pointing to the "while true" loop inside fn_transport.

java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:878)
clojure.tools.nrepl.transport$fn_transport$fn__3912.invoke(transport.clj:44)
clojure.core$binding_conveyor_fn$fn__3989.invoke(core.clj:1819)
clojure.lang.AFn.call(AFn.java:18)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
java.util.concurrent.FutureTask.run(FutureTask.java:166)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
java.lang.Thread.run(Thread.java:722)

What do you recommend as way to free the thread? I have server code that calls nrepl on behave of client connections and the number of call can pile up fairly quickly.



 Comments   
Comment by David Lao [ 12/Mar/13 3:37 PM ]

Here is my workaround.

— a/src/main/clojure/clojure/tools/nrepl/transport.clj
+++ b/src/main/clojure/clojure/tools/nrepl/transport.clj
@@ -36,12 +36,12 @@
to the 2 or 3 functions provided."
([read write] (fn-transport read write nil))
([read write close]

  • (let [read-queue (SynchronousQueue.)]
  • (future (try
  • (while true
  • (.put read-queue (read)))
  • (catch Throwable t
  • (.put read-queue t))))
    + (let [read-queue (SynchronousQueue.)
    + transport-thread (future (try
    + (while true
    + (.put read-queue (read)))
    + (catch Throwable t
    + (.put read-queue t))))]
    (FnTransport.
    (let [failure (atom nil)]
    #(if @failure
    @@ -51,7 +51,7 @@
    (do (reset! failure msg) (throw msg))
    msg))))
    write
  • close))))
    + (fn [] (close)(future-cancel transport-thread))))))
Comment by Chas Emerick [ 06/Aug/13 5:47 AM ]

Very nice catch. I'd love to have a patch from you; do you have a CA filed? (see http://clojure.org/contributing)

Comment by Chas Emerick [ 18/Apr/14 10:07 AM ]

Ping: did you send in a CA? Would love to apply a patch from you for this.





[NREPL-33] Consider making session and eval functionality more accessible Created: 08/Oct/12  Updated: 18/Nov/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.0-beta9
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Hugo Duncan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

ritz re-uses some of nrepl's private functions to avoid duplication. The uses are listed below.

Would it be possible to make these functions public? More subjectively, it might also be worth considering factoring out the session functionality into it's own namespace (including msg and possibly queue-eval), so the functionality is not split across the session middleware and the interruptible-eval middleware.

The debug nrepl server:
https://github.com/pallet/ritz/blob/develop/nrepl/src/ritz/nrepl.clj#L189

This uses clojure.tools.nrepl.middleware.session/create-session and clojure.tools.nrepl.middleware.session/session-out

ritz provides an eval op that tracks source forms:
https://github.com/pallet/ritz/blob/develop/nrepl-middleware/src/ritz/nrepl/middleware/tracking_eval.clj

This uses clojure.tools.nrepl.middleware.interruptible-eval/queue-eval and clojure.tools.nrepl.middleware.interruptible-eval/configure-executor






[NREPL-29] Provide a mechanism for overriding an operation Created: 09/Sep/12  Updated: 18/Apr/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: 0.2.0-beta9, 0.2.0-beta10
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Hugo Duncan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

When specifying middleware, it would be much easier for the user to be able to override a default middleware without having to specify a handler.

For example, if there is a default middleware providing the "complete" operation, the user should be able to just specify their preferred completion middleware, without having to specify all middleware as a handler.

One way to do this might be to check metadata for the provided operations of the specified metadata, and ensure that either the default middleware for that operation is removed, or that the specified middleware takes precendence (which may be simpler when a middleware provides multiple operations).



 Comments   
Comment by Chas Emerick [ 14/Sep/12 7:44 AM ]

Agreed.

Just making sure that the order in which additional middlewares are provided is taken as a default stack order will suffice for most use cases. Transforming middlewares (either in full or in part) would need another metadata slot, :replace perhaps, though it seems like that would be much more difficult to get right.





[NREPL-24] :session key is overloaded Created: 10/Aug/12  Updated: 03/Oct/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Hugo Duncan Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The session middleware takes the :session key from the message, and uses it as an id to lookup session data, which it then places on the same key.

Using a separate key for the session id and the session data would be less confusing, and would allow easier checking of the availability of the session data in the message map.



 Comments   
Comment by Chas Emerick [ 10/Aug/12 11:26 PM ]

The session id will need to continue to come in in the :session slot, as I don't want to break clients (and, there's very, very few middlewares out there yet).

Suggestions for names for the actual session atom? :the-session? :-/





[NREPL-15] Allow clients to specify an ID for newly-retained sessions Created: 29/Mar/12  Updated: 12/Oct/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

...particularly important where less sophisticated clients and polling-oriented transports are involved (e.g. HTTP).






[NREPL-4] Provide sane multiplexing of output in the face of multithreaded, asynchronous operation Created: 30/Sep/11  Updated: 03/Oct/12

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Basically, (send-off some-agent println) & co. should get what's printed to out in that agent's thread back to the nREPL client — not silently let it dump out to System/out.

Portal ostensibly does this well. Examine their approach, see if it is compatible with nREPL's objectives.

Ill-formed brain dump:

  • multiplex new out's to System/out
    • (still won't solve clojure.test/test-out content will disappearing into the ether when it's loaded when out is bound to an nREPL out; maybe we should ensure out is bound to System/out while code is being loaded?)
  • optionally multiplex System/out and System/err
  • optionally join multiplexed S/out and S/err, receive :stdout, :stderr msgs





[NREPL-3] Adopt default port Created: 30/Sep/11  Updated: 13/Jun/14

Status: Open
Project: tools.nrepl
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Chas Emerick Assignee: Chas Emerick
Resolution: Unresolved Votes: 1
Labels: None


 Description   

7888 is "free", at least in IANA.

Most users want to put an nREPL port on their app, and that should always be on a typical port. Auto-selection of a port is only desirable in tooling scenarios, and tooling authors can pass an argument to start-server.

This will change the behaviour of (start-server).



 Comments   
Comment by Bozhidar Batsov [ 13/Jun/14 9:55 AM ]

Sounds like a good idea.





Generated at Tue Jul 29 15:55:41 CDT 2014 using JIRA 4.4#649-r158309.