<< Back to previous view

[ASYNC-196] Add ability to transduce values collected via clojure.core.async/into Created: 25/May/17  Updated: 25/May/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File core.async-into-xform-1.patch     Text File core.async-into-xform-2.patch     Text File core.async-into-xform-3.patch    
Patch: Code and Test
Approval: Triaged

 Description   

This is a trivial change that allows a transducer to be provided as the first argument to clojure.core.async/into. This feels like a common use case ... transforming values and conj'ing them into a collection, as with (transduce xform conj [] ch).



 Comments   
Comment by Howard Lewis Ship [ 25/May/17 6:47 PM ]

This fixes two typos I, of course, didn't notice until after submitting.

Comment by Howard Lewis Ship [ 25/May/17 7:20 PM ]

Patch 3 doesn't include the change to CONTRIBUTING.md





[ASYNC-159] (CLJS) promise-chan in ClojureScript is broken Created: 13/Feb/16  Updated: 25/May/17  Resolved: 25/May/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Rangel Spasov Assignee: Unassigned
Resolution: Completed Votes: 4
Labels: promise-chan
Environment:

Google Chrome 48.0.2564.109 (64-bit) Mac OS X El Capitan 10.11


Attachments: Text File 0001-fixes-http-dev.clojure.org-jira-browse-ASYNC-159.patch     Text File 0002-fixed-tests-to-take-ASYNC-159-into-account.patch    
Approval: Triaged

 Description   

promise-chan in ClojureScript does not seem to work. In Clojure on the JVM, the code below produces the expected result.

Example:

(ns hush-vendor-cljs.example
(:require-macros [cljs.core.async.macros :refer [go]])
(:require [cljs.core.async :refer [promise-chan <! >!]]))

(def p (promise-chan))

(def go-1 (go (let [r (<! p)] (println "got value on promise-chan::" r))))

(def go-2 (go (let [r (<! p)] (println "got value on promise-chan::" r))))

(go (>! p 1))
;This prints only once in ClojureScript, should be twice. It works on JVM Clojure (prints twice).
;=> got value on promise-chan:: 1



 Comments   
Comment by Alex Miller [ 24/Mar/16 11:02 AM ]

Can you update with what version of core.async you're using?

Comment by Johannes Gustafsson [ 24/Mar/16 12:51 PM ]

I can confirm the bug using:

Mac OS X El Capitan
Chrome 49.0.2623.87 (64-bit)
core.async 0.2.374

I created an empty figwheel project using lein new figwheel.

In the REPL, I evaluate each expression one by one. When I evaluate the last expression, only one row is printed. If I evaluate the last expressione one more time, then the second print shows up.

Comment by Johannes Gustafsson [ 24/Mar/16 12:54 PM ]

I get the same result in Firefox

Comment by Rangel Spasov [ 24/Mar/16 1:54 PM ]

Alex - This is under 0.2.374. Sorry for missing the version, this is my first bug report .

Comment by Marcin Kulik [ 18/Apr/16 6:40 AM ]

I am experiencing the same issue. This prints only once:

(let [p (promise-chan)]
  (go
    (<! (timeout 500))
    (println (<! p)))
  (go
    (<! (timeout 500))
    (println (<! p)))
  (go
    (<! (timeout 1000))
    (>! p :yup)))

While this prints twice as expected:

(let [p (promise-chan)]
  (go
    (<! (timeout 1500))
    (println (<! p)))
  (go
    (<! (timeout 1500))
    (println (<! p)))
  (go
    (<! (timeout 1000))
    (>! p :yup)))

I am under 0.2.374.

Comment by Jacek Tomaszewski [ 11/Oct/16 7:51 AM ]

The issue is still present in 0.2.391.

I needed promise-chan so badly that I wrote a fixed version. It uses an extra channel for consumers to wait upon it until promise is resolved.
Feel free to use it.
Disclaimer: tested only under ClojureScript.

; (:require [cljs.core.async :as async :refer [<! >!]]
;           [cljs.core.async.impl.protocols :as impl]


(deftype FixedPromiseChannel [c p]
  impl/WritePort
  (put! [this val ^not-native handler]
    (let [ret (impl/put! p val handler)]
      (impl/close! c)
      ret))
  impl/ReadPort
  (take! [this ^not-native handler]
    (if (impl/closed? c)
      (impl/take! p handler)
      (async/take! c #(async/take! p (impl/commit handler)))))
  impl/Channel
  (closed? [_] (impl/closed? p))
  (close! [this]
    (impl/close! p)
    (impl/close! c)))

(defn fixed-promise-chan []
  (let [c (async/chan)
        p (async/promise-chan)]
    (FixedPromiseChannel. c p)))

And then just use fixed-promise-chan instead of async/promise-chan as drop-in replacement.

Comment by Nick Alexander [ 11/Oct/16 2:52 PM ]

This bit me as well; I worked around it by using a native JS Promise.

Comment by Bogdan Bugarschi [ 03/Apr/17 2:22 PM ]

I think i might have a solution for this. Looks like the put in MMC was not writing all values in the buffer only the first one.
Solution was pritty simple check https://github.com/boogie666/core.async/blob/master/src/main/clojure/cljs/core/async/impl/channels.cljs#L62

The recur is less nested. Look like that does the trick.

It makes more sense to dispatch only on the active take handler and not stop at the last active one, but keep going while there are values in the buffer and takers.

Comment by Bogdan Bugarschi [ 03/Apr/17 2:39 PM ]

i've added a patch

Comment by Bogdan Bugarschi [ 03/Apr/17 2:55 PM ]

Added a patch for the tests as well. looks like promise-chan tests work when they should not. now they fail with out previous patch

Comment by David Nolen [ 07/Apr/17 11:44 AM ]

Bogdan that also looks closer to what Clojure does. Have you submitted your CA?

Comment by Bogdan Bugarschi [ 07/Apr/17 11:47 AM ]

"Clojure CA between Rich Hickey and Bogdan Bugarschi is Signed and Filed!"

Comment by Bogdan Bugarschi [ 10/May/17 6:51 AM ]

@David any news on this?

Comment by David Nolen [ 12/May/17 4:14 PM ]

Bogdan I finally got a chance to look at this, I think you should follow the Clojure implementation more closely to avoid unintended behavior. In Clojure, we collect all takers and then call dispatch on all of them in one go. If you make this change then I think we are good. Let me know if this needs more clarification.

Comment by David Nolen [ 19/May/17 2:04 PM ]

I believe I've resolved the issue on master as of this commit https://github.com/clojure/core.async/commit/52f6bc19b599623a247539311e81336c5979b19f. Please test master and report back here. Thanks everyone.

Comment by Bogdan Bugarschi [ 23/May/17 11:22 AM ]

Fix looks good as far as i can tell, but i think you should update the tests to account for this bug as well. There already is a test for that case, it's just not correct. it's "put on promise-chan fulfills all pending takers" from tests.cljs. basically the go block around the test needs to moved inside (see my 0002 patch for the tests).

Comment by David Nolen [ 24/May/17 10:49 AM ]

Bogdan, thanks for the feedback. I'll apply your test patch.

Comment by Bogdan Bugarschi [ 24/May/17 4:39 PM ]

yey thx

Comment by David Nolen [ 25/May/17 4:41 PM ]

fixed in master https://github.com/clojure/core.async/commit/5459fe8fb34e2fee9db0ed3ba9f12a4183fe9342





[ASYNC-192] Metadata does not persist across go-block boundaries in Clojurescript core.async Created: 22/May/17  Updated: 24/May/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Víctor M. Valenzuela Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Clojurescript 1.9.542



 Description   

In clojurescript's version of core.async, the following should print hello, but it doesn't:

(go
  (println
    (meta
      (<!
        (let [c (chan)]
          (go
            (>! c ^:hello [42]))
          c)))))

No such difference is mentioned in https://github.com/clojure/clojurescript/wiki/Differences-from-Clojure.



 Comments   
Comment by Víctor M. Valenzuela [ 24/May/17 6:28 PM ]

A more consise repro: (go (js/alert (meta ^{:id 1} [])))

This is even worse than originally reported - metadata disappears even within the same go block that creates it.





[ASYNC-195] `recur` in alt go throws in ClojureScript, work in Clojure. Created: 22/May/17  Updated: 23/May/17  Resolved: 23/May/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Isaac Zeng Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

Mac OS & JDK8



 Description   

try belowing code:
```clojure
(go-loop []
(a/alt!
ch
([x] (when x
(println "got" x)
(recur)))
(timeout 3000)
(do
(println "timeout")
(recur))))
```



 Comments   
Comment by Isaac Zeng [ 22/May/17 11:10 PM ]

Sorry, How can I close it? it duplicate with https://dev.clojure.org/jira/browse/ASYNC-194

I don't know why it creates two same issues?

Comment by Isaac Zeng [ 23/May/17 1:17 AM ]

nevermind, wrong usage.





[ASYNC-194] `recur` in alt go throws in ClojureScript, work in Clojure. Created: 22/May/17  Updated: 23/May/17  Resolved: 23/May/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Isaac Zeng Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

Mac OS & JDK8



 Description   

try belowing code:
```clojure
(go-loop []
(a/alt!
ch
([x] (when x
(println "got" x)
(recur)))
(timeout 3000)
(do
(println "timeout")
(recur))))
```



 Comments   
Comment by Isaac Zeng [ 23/May/17 1:18 AM ]

nevermind, wrong usage

Comment by Alex Miller [ 23/May/17 8:10 AM ]

per request





[ASYNC-193] Opaque string representation of channels in Clojurescript core.async Created: 22/May/17  Updated: 22/May/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Víctor M. Valenzuela Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Clojurescript 1.9.542
core.async 0.3.442


Attachments: PNG File Screen Shot 2017-05-23 at 02.24.18.png    

 Description   

In clojurescript, (str (chan)) returns the string {{"[object Object]"}}. This is different from JVM Clojure behavior per se, and more practically, it can result in hard-to-debug stack traces, such as the attached one.

The attached backtrace doesn't reach my application code, and does not show any cljs.core.async code either. Therefore it's hard to figure out that I was mishandling a channel. A nicer string representation of channels would have saved me 20 minutes of head-scratching!



 Comments   
Comment by Alex Miller [ 22/May/17 9:38 PM ]

It's never been worked on, but we have talked about having the ability to name or add meta to channels.





[ASYNC-191] storage-buffer and storage-chan Created: 19/May/17  Updated: 19/May/17  Resolved: 19/May/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Vitalie Spinu Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: enhancement


 Description   

Would it be possible to add a buffer that `constantly` emits the last placed value? It's effectively a storage cell which behaves like already implemented `promise-buffers`, but allows for new values to be put on the chan.

The motivation is as follows. If some sub-system computes statistics and places them on a chan, receivers could pick up the most recent value without bothering to store it themselves. In my application this would completely eliminate all needs for mutable storage.

Sorry if this feature already exists or is trivial to implement. I couldn't find anything relevant on this.



 Comments   
Comment by Alex Miller [ 19/May/17 8:06 AM ]

I don't think we're going to add this but I suspect you could make a custom buffer pretty easily that did this.

Comment by Vitalie Spinu [ 19/May/17 9:18 AM ]

I currently do it by relying on implementation details in clojure.core.async.impl which is not ideal. Or did you have something else in mind?
AFAICT promise-chan can be easily mimicked by constant go-loop, but mimicking storage buffer is a non-trivial business.

Any specific reason for the decline? Is it a bad idea to use channels in this way?





[ASYNC-58] mult channel deadlocks when untapping a consuming channel whilst messages are being queued/blocked Created: 20/Feb/14  Updated: 10/May/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mathieu Gauthron Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: deadlock, mult, untap
Environment:

Mac 10.7.5; java version "1.7.0_40"; [org.clojure/clojure "1.5.1"]; [org.clojure/core.async "0.1.267.0-0d7780-alpha"]; Tested with cider and emacs 24.3


Approval: Triaged

 Description   

I have two (or more) listeners tapped onto a mult channel. I want to use them all then have one (or more) of them to leave at will without blocking the other consumer(s) or the publisher. Initially they work fine until one of them wants to stop listening. I thought the listener which drops out needs to (be a good citizen and) untap its channel from mult (otherwise a deadlock is systematic). However if messages are put into the mult before the leaving listener has had a chance to untap its channel, it creates a deadlock on the main thread (which is putting more messages simultaneously). I do not find a way to guarantee that I can untap the channel in time to avoid this race condition.

Once I have reproduced the deadlock, the repl is frozen until I interrupt with ctrl-c.
I have also tried to close the tapped channel before untapping it but the result was the same.

In the following snippet, the last (println "I'm done. You will never see this") is never reached. The publisher and the remaining consumer (consumer 1) are deadlocked even though consumer 2 was trying to leave in good terms.

(require '[clojure.core.async :refer (chan go <! >!! mult tap untap)])
(let [to-mult (chan 1)
      m (mult to-mult)]

  ;;consumer 1
  (let [c (chan 1)]
    (tap m c)
    (go (loop []
          (when-let [v (<! c)]
            (println "1 Got! " v)
            (recur))
          (println "1 Exiting!"))))

  ;;consumer 2
  (let [c (chan 1)]
    (tap m c)
    (go (loop []
          (when-let [v (<! c)]
            (when (= v 42)  ;; exit when value is not 42
              (println "2 Got! " v)
              (recur)))
          (println "2 about to leave!")
          (Thread/sleep 5000) ;; wait a bit to exacerbate the race condition
          (untap m c) ;; before unsubscribing this reader
          (println "2 Exiting."))))

   (println "about to put a few messages that work")
   (doseq [a (range 10)]
     (>!! to-mult 42))
   (println "about to put a message that will force the exit of 2")
   (>!! to-mult 43)
   (println "about to put a few more messages before reader 2 is unsubscribed to show the deadlock")
   (doseq [a (range 10)]
     (println "putting msg" a)
     (>!! to-mult 42))
   (println "I'm done. You will never see this"))
about to put a few messages that work
2 Got!  42
1 Got!  42
2 Got!  42
1 Got!  42
1 Got!  42
2 Got!  42
1 Got!  42
1 Got!  42
2 Got!  42
2 Got!  42
2 Got!  42
2 Got!  1 Got!  42
422 Got!  42

1 Got!  42
1 Got!  42
2 Got!  42
1 Got!  42
about to put a message that will force the exit of 2
1 Got!  42
2 Got!  about to put a few more messages before reader 2 is unsubscribed to show the deadlock
42
putting msg 1 Got!  0
2 about to leave!
43
1 Got!  42
putting msg 1
putting msg 2
putting msg 3
1 Got!  42
2 Exiting.


 Comments   
Comment by Ghadi Shayban [ 22/Apr/14 10:18 AM ]

Mathieu, this is probably expected. It's important to note that to guarantee correct ordering/flow when using a mult, you should enforce it on the source/producer side of the mult, and not asynchronously on the tap side.

Mult will deref a stable set taps just before distributing a value to them, and does not adjust dynamically during value distribution except when a tap has been closed [1]. If you would like to stably untap without closing the tap you can/should let the 'producer' do it in an ordered fashion in between values on the input channel.

Knowing that a put occurred to a closed channel is new on release 0.1.278.

In general, walking away on the consuming side of a channel is tricky. Depending on the semantics of your processes, if the producer side of a channel isn't aware that a close! can happen from the consumer side, you might have to launch a draining operation.

(defn drain [c] (go (when (some? (<! c)) (recur))))

Golang disallows closing a read-only channel FWIW [2]

Better documentation is probably warranted.

[1] https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L680-L682
[2] http://golang.org/ref/spec#Close

Comment by Sam Umbach [ 10/May/17 11:57 AM ]

See also ASYNC-66.





[ASYNC-66] Add drain! to consume and discard a channel Created: 09/May/14  Updated: 10/May/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Ghadi Shayban Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None

Approval: Vetted

 Description   

Do we want this for consumer walkaway scenarios or something baked into the channel impl?

(defn drain!
 "Fully consumes and discards a channel. Returns a channel that
  closes when ch is fully drained"
 [ch]
 (go-loop []
   (if (some? (<! ch))
     (recur))))


 Comments   
Comment by Sam Umbach [ 10/May/17 11:55 AM ]

When a channel's buffer is full, close! and/or untap on that channel is not sufficient to unblock other taps on the same mult channel. This is a specific case when drain! is necessary; be sure to drain after untap/close to avoid a race condition.





[ASYNC-190] (CLJS) Some non-tail-recursive loops compile without error Created: 27/Mar/17  Updated: 28/Mar/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Whittington Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Clojurescript



 Description   

The following code compiles without error using core.async 0.3.442:

(go-loop []
  (when true
    (recur))
  1)

The same loop structure without core.async causes the compiler to error ("ERROR: Can't recur here at..."):

(loop []
  (when true
    (recur))
  1)

Interestingly, some non-tail-recursive loops correctly fail to compile. For example the following fails with "ERROR: No implementation of method: :emit-instruction...":

(go-loop []
  (do
    (recur)
    1))


 Comments   
Comment by David Whittington [ 27/Mar/17 1:31 PM ]

Created a repository demonstrating this issue: https://github.com/djwhitt/async-190





[ASYNC-189] Tap return value not documented Created: 24/Mar/17  Updated: 24/Mar/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: justin taft Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

(tap mult ch) returns ch. This is not documented.
https://clojuredocs.org/clojure.core.async/tap






[ASYNC-74] Provide a chan? function to determine if a given var is a channel Created: 23/Jun/14  Updated: 12/Mar/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Reno Reckling Assignee: Unassigned
Resolution: Unresolved Votes: 7
Labels: None

Attachments: File drupp-async-74.diff    

 Description   

When using channels, for example for streaming http bodies, it would be really helpful if there would be a chan? function to check whether a given var is a channel or not.

Aleph already does it like that in it's http-server/client implementation using lamina channels and it feels quite consistent.

It would of course also be more consistent with the rest of clojure to provide a type checking function for basic objects.



 Comments   
Comment by David Rupp [ 30/Jun/14 9:58 AM ]

Implement chan? predicate.

Comment by Timothy Baldridge [ 30/Jun/14 10:50 AM ]

I'm in favor of this, but last time I asked Rich about it his quote was "do you want a predicate for every single interface?".

Due to the implementation of core.async you'd probably need two additional predicates. One for read-port? and write-port?. You can use (satisfies? clojure.core.async.impl.protocols/ReadPort ...) but that's an internal implementation, so I'd rather have a new predicate than to tell people to touch the innards of core.async. But this call is up to Rich.

Comment by Reno Reckling [ 30/Jun/14 11:07 AM ]

I understand that reasoning. But then we would have to step up on documentation and provide a way to easily determine which interfaces are implemented by the return values of for example (chan). For me, going into the innards of core.async and trying to determine which interfaces I want, especially on very basic things like a channel, is not a user friendly approach at all.

Comment by Thomas Heller [ 12/Mar/17 5:10 AM ]

I would like to see the mentioned chan? read-port? write-port? predicates as they would make writing some clojure.spec/fdefs simpler (and portable to cljs.core.async).





[ASYNC-114] Allow a second arity on channel ex-handler which takes val as well as throwable Created: 27/Jan/15  Updated: 23/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Derek Troy-West Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None

Attachments: Text File 0001-Pass-the-val-along-with-the-exception-to-the-channel.patch    

 Description   

When a transducer is applied to a channel and an exception occurs during transformation the ex-handler will be called with
the Throwable as an argument.

Can we also send the val which caused the transformation exception to the ex-handler? When dealing with transducer errors it might be useful to have the full picture.

If that's agreeable, and the solution is a second-arity on ex-handler that takes val and throwable, and the call applied is:

clojure.core.async.impl.channels becomes:

(defn chan
      ...
             (try
               (add! buf val)
               (catch Throwable t
                 (handle buf exh t val)))))))))


 Comments   
Comment by Nivedita Priyadarshini [ 19/Feb/15 5:56 AM ]

Found the same need when using core.async/pipeline in my code. Wrote a patch of sorts (demonstrative) which allows you to have val in the exh for clj. Happy to write a full patch if this looks okay.

Comment by Derek Troy-West [ 23/Feb/17 2:41 PM ]

Perhaps leave the ex-handler as-is, but pass an ex-info rather than simply the caught Throwable. Value that caused the exception provided in the ex-info map, Throwable as cause.





[ASYNC-188] Properly pop `let` locals binding Created: 23/Feb/17  Updated: 23/Feb/17  Resolved: 23/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File 0001-ASYNC-188-properly-pop-locals-binding.patch    

 Description   

ASYNC-187 incorrectly popped the locals stack, this patch fixes it



 Comments   
Comment by Alex Miller [ 23/Feb/17 12:56 PM ]

Applied for next release.





[ASYNC-185] `thread` prevents clearing of body locals Created: 17/Feb/17  Updated: 23/Feb/17  Resolved: 23/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: memory

Attachments: Text File 0001-ASYNC-185-use-fn-once-for-thread-thunk.patch    
Patch: Code
Approval: Ok

 Description   

`thread` wraps its body in a `(fn [] ~@body)`, it should wrap it in `(^:once fn* [] ~@body)` instead to allow clearing of closed over locals

Patch: 0001-ASYNC-185-use-fn-once-for-thread-thunk.patch



 Comments   
Comment by Alex Miller [ 23/Feb/17 9:16 AM ]

Applied for next release.





[ASYNC-186] NPE when `go` closes over a local variable bound to nil Created: 23/Feb/17  Updated: 23/Feb/17  Resolved: 23/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: regression

Attachments: Text File 0001-ASYNC-186-Apparently-Compiler.java-can-return-true-f.patch    
Patch: Code and Test
Approval: Ok

 Description   

This throws a NPE after ASYNC-138

(defn foo [x]
  (let [y nil]
    (a/go)))

Analysis: Compiler.java exhibits a quircky behaviour where for a local bound to nil, it'll return `true` for `lb.hasJavaClass()` but `lb.getJavaClass()` will return `nil`, which causes a NPE when we invoke `lb.getJavaClass().getName()`

user> (defmacro foo []
        (doseq [[l ^clojure.lang.Compiler$LocalBinding lb] &env]
          (println (.hasJavaClass lb) (.getJavaClass lb))))
#'user/foo
user> (let [a nil] (foo))
true nil

Approach:
Don't assume `.getJavaClass` will return non-nil when `hasJavaClass` returns true

Patch: 0001-ASYNC-186-Apparently-Compiler.java-can-return-true-f.patch



 Comments   
Comment by Alex Miller [ 23/Feb/17 9:11 AM ]

Applied for next build.





[ASYNC-187] Tag metadata is lost in local closed over by a loop Created: 23/Feb/17  Updated: 23/Feb/17  Resolved: 23/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: typehints

Attachments: Text File 0001-ASYNC-187-preserve-type-info-of-locals.patch    
Patch: Code
Approval: Ok

 Description   

This seems related to ASYNC-155, a small repro:

(defn foo []
  (a/go
    (let [^String a (identity "foo")]
      (loop []
        (.substring a 1 2)
        (a/<! nil)
        (recur)))))
Reflection warning, NO_SOURCE_PATH:23:17 - call to method substring can't be resolved (target class is unknown).

This bug affects all version of core.async, but it's getting triggered more now that the patch for ASYNC-138 applies a lexical transformation that makes code look like this, thus hitting this bug more often

Approach: The approach is the same used for ASYNC-155, applied to :let binding

Patch: 0001-ASYNC-187-preserve-type-info-of-locals.patch



 Comments   
Comment by Alex Miller [ 23/Feb/17 12:55 PM ]

Applied for next release.





[ASYNC-138] Go blocks leak memory Created: 10/Aug/15  Updated: 22/Feb/17  Resolved: 22/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Brian Lubeski Assignee: Unassigned
Resolution: Completed Votes: 14
Labels: go, memory
Environment:

clojure 1.7.0
core.async 0.1.346.0-17112a-alpha
Java HotSpot(TM) Client VM 1.8.0_31-b13


Attachments: Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2-nested.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v3.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v4.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v5.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v6.patch    
Patch: Code and Test
Approval: Vetted

 Description   

The following example, after running for a few minutes, generates an OutOfMemoryError.

(let [c (chan)]
  (go (while (<! c)))
  (let [vs (range)]
    (go
      (doseq [v vs]
        (>! c v)))))

By contrast, the following example will run indefinitely without causing an OutOfMemoryError.

(let [c (chan)]
  (go (while (<! c)))
  (go
    (let [vs (range)] 
      (doseq [v vs]
        (>! c v)))))

The only significant difference I see between the two examples is that the (range) is created outside the go block in the first example but is created inside the go block in the second example. It appears that the go block in the first example is referencing vs in such a way as to prevent if from being garbage-collected.

This behavior is also be the cause of ASYNC-32.

Approach:
The attached patch applies a transformation from

(let [a large-seq] (loop [..] .. a ..))
where the loop exemplifies the state machine loop created by `go`, into
(let [a large-seq a' (^:once fn* [] a)] (loop [..] .. (let [a (a')] ..) ..))

which allows the closed over locals to cross into the state machine loop without getting held. While this might look unsafe, because of how the `go` loop state machine works, we're guaranteed that `(a')` will only be invoked on the first iteration of the loop.
The patch looks way more complex than it actually is, because it needs to deal with: renaming locals so that shadowing of special symbols works properly, preserving locals type hints and handle nested go blocks, where the value of `&env` will be provided by tools.analyzer rather than from Compiler.java

There are two different type propagations that we need to do, one in case of non primitive objects and one in the case of primitive values:

  • if we're propagating type info for a non primitive object, the transformation is as follow:
    (let [a ^Klass my-obj] (loop [..] .. a ..))
    ->
    (let [a ^Klass my-obj a' (^:once fn* [] a)] (loop [..] .. (let [^Klass a (a')] ..) ..))
  • if we're propagating type info for a primitive local, a type hint won't do – we actually need to unbox the value out of the thunk, so the transformation will be:
(let [a (int x)] (loop [..] .. a ..))
->
(let [a (int x) a' (^:once fn* [] a)] (loop [..] .. (let [a (clojure.core/int (a'))] ..) ..))

Patch: 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v6.patch



 Comments   
Comment by Nicola Mometto [ 15/Dec/15 9:34 AM ]

attached is a WIP patch to fix this issue, would appreciate some testing on this

Comment by Nicola Mometto [ 15/Dec/15 5:52 PM ]

CLJ-1744 contributes to this bug

Comment by Nicola Mometto [ 15/Dec/15 6:23 PM ]

Updated patch to preserve type info

Comment by Nicola Mometto [ 17/Dec/15 10:16 AM ]

Note that the current patch breaks nested `go` blocks at compile time since the nested `go` block will be macroexpanded by tools.analyzer.jvm rather than Compiler.java and this &env will be different.

Is this something we want to support? There seem to be a lot of other cases that break when nesting `go` blocks.

If this is something that we do want to support 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2-nested.patch fixes this

Comment by Jan Rychter [ 21/Dec/15 6:18 AM ]

This just bit me badly when doing data processing work — I never expected that (async/to-chan (line-seq rdr)) can be the culprit of my heap getting exhausted in 30 seconds. Took quite some time to narrow it down. Thanks to Nicola for letting me know about this bug on Slack.

Comment by Alex Miller [ 13/Feb/17 3:31 PM ]

Nicola, not sure how to read your comments above re the patches. Most of the other cases of nested go blocks failing that I see in the tracker are related to the exception handling bugs in ASYNC-169. Do you have any hesitancy over your patch for the nested stuff?

Comment by Nicola Mometto [ 13/Feb/17 3:40 PM ]

Alex, I think the only reason why I proposed the -nested patch as an alternative rather than as a main patch is that it adds complexity to the patch to handle an unlikely scenario. I think we should go with the -nested patch, but if understanding the patch for screeners is more important than handling weird edge cases, the other patch is easier

Comment by Alex Miller [ 13/Feb/17 3:55 PM ]

Tests don't pass with -nested patch.

Caused by: java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to java.util.Map$Entry
	at clojure.lang.APersistentMap$ValSeq.first(APersistentMap.java:232)
	at clojure.lang.RT.first(RT.java:653)
	at clojure.core$first__4110.invoke(core.clj:55)
	at clojure.core.async.impl.ioc_macros$state_machine.invoke(ioc_macros.clj:1107)
	at clojure.core.async.ioc_macros_test$runner.doInvoke(ioc_macros_test.clj:24)
Comment by Nicola Mometto [ 13/Feb/17 5:04 PM ]

updated patch making all the tests run

Comment by Nicola Mometto [ 14/Feb/17 4:49 AM ]

updated patch with an additional testcase making sure nested go blocks work properly after the transformation, also bumping tools.analyzer.jvm to the last version as that includes a fix for nested t.a calls

Comment by Alex Miller [ 14/Feb/17 10:40 AM ]

Applied.

Comment by Alex Miller [ 15/Feb/17 3:36 PM ]

I am reopening this and reverting the change for the moment. This example fails when using Clojure 1.9.0-alpha12+ (anything after the commit for CLJ-1224).

(require '[clojure.core.async :refer (go)])
(defprotocol P
  (x [p]))

(defrecord R [z]
  P
  (x [_]
    (go
      (loop []
        (recur)))))

CompilerException java.lang.UnsupportedOperationException: Can't type hint a primitive local

I believe this is due to the code in the patch related to applying meta to the local binding, specifically this comes into interaction with the code now generated in hasheq after CLJ-1224.

Comment by Nicola Mometto [ 15/Feb/17 4:58 PM ]

Updated the patch to handle with primitive type hints.
In case of primitive type hints, rather than emitting

(let [^String a (a')] ..)
we emit
(let [a (clojure.core/int (a'))] ..)
as hinting is not valid and proper unboxing is required

Comment by Alex Miller [ 15/Feb/17 8:12 PM ]

Did you mean to have ^String and clojure.core/int in that example?

Comment by Nicola Mometto [ 16/Feb/17 3:12 AM ]

yes, to showcase the two different transformations we do in case of primitive or non primitive type propagation. I'll expand on that in the ticket description so it's clearer what i meant

Comment by Alex Miller [ 17/Feb/17 9:10 AM ]

Applied v5 version.





[ASYNC-169] Try/catch block loops eternally inside go block Created: 16/May/16  Updated: 22/Feb/17  Resolved: 22/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Frank Burnham Assignee: Kevin Downey
Resolution: Completed Votes: 9
Labels: try-catch
Environment:

Clojure 1.7.0; core.async 0.2.374; java 1.8


Attachments: Text File 0001-overhaul-exceptions-in-clojure-go-macro-ASYNC-169.patch     Text File 0005-Turn-tag-class-in-to-symbol-ASYNC-169.patch     Text File 0006-Turn-tag-class-in-to-string-ASYNC-169.patch     Text File 0010-overhaul-exceptions-in-clojure-go-macro-ASYNC-169.patch     Text File ASYNC-169-4.patch    
Patch: Code and Test
Approval: Ok

 Description   
(a/<!!
    (a/go-loop []
      (try
        (a/<! (throw (Exception. "Ex")))
        (catch clojure.lang.ExceptionInfo ei
          :retry))))

The above example appears to loop forever when I expected it to return nil.

(a/<!!
    (a/go-loop []
      (try
        (throw (Exception. "Ex"))
        (catch clojure.lang.ExceptionInfo ei
          :retry))))

This example returns nil.

Related tickets:

Patch: 0010-overhaul-exceptions-in-clojure-go-macro-ASYNC-169.patch



 Comments   
Comment by Kevin Downey [ 16/May/16 3:50 PM ]

it looks like maybe "process-exception" in clojure.core.async.impl.ioc-macros is missing an :else (throw exception) clause at the end.

Comment by Frank Burnham [ 17/May/16 7:12 AM ]
user> (a/<!!
       (a/go-loop []
                  (try
                     (a/<! (throw (Exception. "Ex")))
                   (catch clojure.lang.ExceptionInfo ei
                          :retry) 
                   (catch Throwable t :drop))))
nil
user>

That example is also odd to me. I expected to see :drop returned.

Comment by Kevin Downey [ 17/May/16 12:07 PM ]

looking at the macro expansion for the two catches, the emitted code is only setting up an exception frame for the ExceptionInfo case, and not for the Throwable case.

I suspect the exception handling code is all sawdust and broken promises.

Comment by Kevin Downey [ 17/May/16 12:12 PM ]

this looks like it is ignoring everything except the first catch https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/ioc_macros.clj#L709

Comment by Kevin Downey [ 17/May/16 7:33 PM ]

this patch seems to make everything work, needs more tests of course

Comment by Kevin Downey [ 17/May/16 7:55 PM ]

looks like this might be the same issue as http://dev.clojure.org/jira/browse/ASYNC-100, which also has a patch attached

Comment by Alexey Aristov [ 10/Jul/16 11:36 AM ]

are there any plans/estimations regarding this bug? we were recently hit by it, and was a very special pleasure to analyze the issue.

Comment by Kevin Downey [ 20/Sep/16 11:38 AM ]

new patch, includes tests

Comment by Roman Pearah [ 09/Nov/16 9:33 PM ]

Wow, this is huge. My try block was just replaying over and over for no discernible reason, all without anything new coming off the channel (or anything prior to the try block executing). Stumped me for a while until I found this. Switching to catching Throwable instead of Exception worked as a quick fix.

Comment by Alex Miller [ 13/Feb/17 2:17 PM ]

Kevin Downey Hey, I've been looking at this, had a couple questions.

1) Does something similar need to be done for cljs?

2) There are a couple minor whitespace issues in the tests - you'll see the warning if you 'git apply' the patch - just need to remove trailing whitespace in a couple places.

3) I see these new reflection warnings from the new tests. Can these be avoided by typehinting to the type of the exception being caught inside the ?

Reflection warning, clojure/core/async/ioc_macros_test.clj:71:18 - reference to field getMessage can't be resolved.
Reflection warning, clojure/core/async/ioc_macros_test.clj:69:51 - reference to field getMessage can't be resolved.
Reflection warning, clojure/core/async/ioc_macros_test.clj:67:53 - reference to field getMessage can't be resolved.
Comment by Alex Miller [ 13/Feb/17 2:34 PM ]

The example in ASYNC-173 is another cases that causes a reflection warning on .printStackTrace.

Comment by Kevin Downey [ 13/Feb/17 3:54 PM ]

Latest patch fixes reflection on caught exceptions

Comment by Alex Miller [ 13/Feb/17 3:56 PM ]

thanks - for future reference, please add a -2 or something on the end so the patches aren't the same name (too easy to apply the wrong one). No need to do anything here though.

Comment by Kevin Downey [ 13/Feb/17 3:57 PM ]

Latest patch fixes trailing whitespace errors

Comment by Kevin Downey [ 13/Feb/17 4:00 PM ]

As for cljs, I am not sure, a quick glance has not filled me with confidence. I suppose it should be put through its paces with a similar set of tests. Off the bat it looks like the code only looks for one catch clause, which could be a problem, but maybe something up the line is turning multi-clause catches in to single clause catches for it.

Comment by Alex Miller [ 13/Feb/17 4:51 PM ]

For the examples up in the description:

  • the first example invokes <! with no channel, so seems broken to start (with patch applied, it throws)
  • the second example after patch is applied, throws as well.

I did not expect either example to throw.

Comment by Kevin Downey [ 13/Feb/17 5:26 PM ]

I don't follow

When I run the example I get a nil result (taking from a closed channel with nothing on it because the go-loop threw an exception). I get no new exception bound to *e, because the exception happened in the go loop thread, not the repl thread. And I get a print out of the exception thrown on the core.async thread because of core.async's default exception handling.

The exception bubbles out of the go block because the catch is for ExceptionInfo, but Exception is being thrown.

Is some part of that incorrect, or does it not match what you get when you run the example?

user=> (require '[clojure.core.async :as a])
nil
user=> (a/<!!
  #_=>     (a/go-loop []
  #_=>       (try
  #_=>         (a/<! (throw (Exception. "Ex")))
  #_=>         (catch clojure.lang.ExceptionInfo ei
  #_=>           :retry))))
Exception in thread "async-dispatch-1" java.lang.Error: java.lang.Exception: Ex
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1148)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Ex
        at user$eval8675$fn__8693$state_machine__6725__auto____8694$fn__8697.invoke(form-init491495474467240539.clj:2)
        at user$eval8675$fn__8693$state_machine__6725__auto____8694.invoke(form-init491495474467240539.clj:2)
        at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:972)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:976)
        at user$eval8675$fn__8693.invoke(form-init491495474467240539.clj:2)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        ... 2 more
nil
user=> *1
nil
user=> *e
nil
user=>
Comment by Alex Miller [ 14/Feb/17 10:01 AM ]

Got it. This example from ASYNC-180 doesn't seem to be doing the right thing now:

(def t3
  (go
    (try
      1
      (finally
        (<! (timeout 100))
        2))))

(println (<!! t3))
;; expect: 1
;; got: IllegalArgumentException No implementation of method: :take! of protocol: #'clojure.core.async.impl.protocols/ReadPort found for class: clojure.lang.Var$Unbound
Comment by Alex Miller [ 14/Feb/17 10:08 AM ]

Never mind, I must have had something weird referred in the midst of my example set.

Comment by Alex Miller [ 14/Feb/17 10:11 AM ]

Applied

Comment by Kevin Downey [ 14/Feb/17 1:41 PM ]

great news, thanks!

Comment by Kevin Downey [ 16/Feb/17 2:12 PM ]

I re-opened this issue due to a bug discussed on slack

(async/go
  (try
    (catch Exception e
      (async/<! 1)
      e)))

doesn't compile

Comment by Kevin Downey [ 16/Feb/17 2:14 PM ]

The latest patch (0003-Don-t-clobber-existing-metadata-ASYNC-169.patch) fixes the issue in my previous comment, and should apply cleanly to master.

Comment by Alex Miller [ 17/Feb/17 9:04 AM ]

When I apply the patch, tests don't run:

Exception in thread "main" clojure.lang.ExceptionInfo: Could not resolve var: ne {:var ne, :file "clojure/core/async/ioc_macros_test.clj", :column 9, :line 94}, compiling:(clojure/core/async/ioc_macros_test.clj:91:6)
Comment by Alex Miller [ 17/Feb/17 9:18 AM ]

Updated last patch to fix typo

Comment by Alex Miller [ 17/Feb/17 9:24 AM ]

I'm seeing a reflection warning here that does not seem expected (example from ASYNC-173):

(require '[clojure.core.async :as a :refer [chan go go-loop <! >! <!! >!! take! put! timeout]])
(defn endless-fun [f]
  (let [in-ch (a/to-chan (range 1 10))]
    (->>
     (a/go
       (try
         (loop []
           (when-let [value (a/<! in-ch)]
             (f value)
             (recur)))
         (catch Exception e
           (.printStackTrace e)
           (throw e))))
     (a/<!!))))

Reflection warning, /private/var/folders/7r/_1fj0f517rgcxwx79mn79mfc0000gn/T/form-init4526179122097363504.clj:11:12 - reference to field printStackTrace can't be resolved.
Comment by Kevin Downey [ 17/Feb/17 10:42 AM ]

0005-Turn-tag-class-in-to-symbol-ASYNC-169.patch applies cleanly to master(v0.2.395-19-g6e36258), and the tests pass so no typos. As I mentioned in slack, in the previous patch I changed how the tag metadata is attached, and ended up attaching a class instead of a symbol

Comment by Kevin Downey [ 17/Feb/17 11:29 AM ]

0006-Turn-tag-class-in-to-string-ASYNC-169.patch replaces the 0005 patch with improvements suggested by Bronsa in slack

Comment by Kevin Downey [ 17/Feb/17 12:39 PM ]

as discussed in slack, 0010-overhaul-exceptions-in-clojure-go-macro-ASYNC-169.patch combines all previous changes and applies cleanly against master (v0.2.395-22-g3e37e0b, all 169 changes reverted)





[ASYNC-128] (CLJS) or evaluation not stopped when exp nests take Created: 09/Jun/15  Updated: 14/Feb/17

Status: Reopened
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

CLJS



 Description   
(go (or true
        (= (<! (do (js/console.log "this should not happen") (chan)))
           :doesnt-matter)))


 Comments   
Comment by Alex Miller [ 13/Feb/17 4:17 PM ]

That <! is missing a channel so this seems to be broken code to start with - marking not reproducible till example fixed.

Comment by Leon Grapenthin [ 14/Feb/17 11:59 AM ]

Hi Alex, the invalid arg to <! was intended.

The point is that it should never be evaluated according to defined behavior of `or` and thus is correct.

You can also substitute line 2 with
(= (<! (do (js/console.log "this should not happen") (chan)))
or invoke a function there that logs and returns a channel.

IMO that should not make a difference to label this a bug.

The bug still exists in 0.2.395

Comment by Kevin Downey [ 14/Feb/17 4:25 PM ]

For some weird reason (maybe as an optimization of some sort?), the or macro in cljs some times expands as calls to if (like clojure's does) and some times expands to `(js* "({}) || ({})" a b)`. the ioc machinery has no special handling for `js*` so it does its normal ANF like transform:

(js* "(~{}) || (~{})" A B)

to

(let [x A
      y B]
  (js* "(~{}) || (~{})" x y))

so of course you lose short circuiting.

General support in the ioc machinery for js* would require parsing the rewriting the javascript, which seems unlikely to happen. Specific cases of js* could be handled by matching the exact string of javascript, but that may be kind of brittle.

The best thing to do is likely to change the clojurescript definition of or to always emit ifs (like clojure's) and move an optimizations in to the compiler.

Comment by Kevin Downey [ 14/Feb/17 4:43 PM ]

also `or` and `and` in clojurescript both run analysis as part of the macro expansion, which can lead to macros be expanded multiple times





[ASYNC-143] Assert that fixed buffers must have size > 0 Created: 18/Aug/15  Updated: 14/Feb/17  Resolved: 14/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 5
Labels: None

Attachments: Text File async-143.patch    
Patch: Code
Approval: Triaged

 Description   

(chan 0) is invalid (and is not the same thing as an unbuffered (chan)).

Approach: Add an assertion to throw on (chan 0). I found based on some failing tests that one place this can come up is in to-chan, which creates a buffer sized to the collection (0). The patch also changes to-chan to return an empty, closed chan if the collection is of size 0.

Patch: async-143.patch



 Comments   
Comment by Alexander Yakushev [ 16/Aug/16 3:24 AM ]

Bump! I was quite surprised today by this.

Besides, the wording could be changed to something like "If a transducer is supplied a non-zero length buffer must be specified."

Comment by Alex Miller [ 14/Feb/17 11:01 AM ]

Applied





[ASYNC-144] pipeline-async docstring correction Created: 27/Aug/15  Updated: 14/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Derek Troy-West Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: docstring, pipeline


 Description   

Currently says:

"af must close! the channel before returning."

more correctly:

"af is responsible for closing the channel.", or similar.

af is asynchronous so should return immediately, the thread or go-block it spawns should close the channel later.






[ASYNC-111] maps are represented different inside and outside of go blocks Created: 27/Dec/14  Updated: 14/Feb/17  Resolved: 14/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Sven Richter Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None
Environment:

clojure 1.6.0, clojurescript "0.0-2371", W7, Oracle JDK 8



 Description   

When initializing a state atom with a set "conjuring" a map behaves differently depending of the scope where it happens:

(def state (atom {:transformations []}))
(reset! state {:transformations #{:foo "bar"}})

; throws: Uncaught Error: compare on non-nil objects of different types
(...(go ;some async stuff
(swap! state conj {:name "baz"}))...)

; works
(...(go ;some async stuff)
(swap! state conj {:name "baz"}))

I am not sure if this is a defect at all as maps are not comparable as Tom in the thread stated. However, the fact that it works in one case and does not in the other is weird.
You can find the complete thread here: https://groups.google.com/forum/#!topic/clojurescript/lmvltpa3k8s



 Comments   
Comment by Alex Miller [ 14/Feb/17 10:56 AM ]

Can't repro, don't see a bug.





[ASYNC-4] Record literals become ordinary maps Created: 15/Jul/13  Updated: 14/Feb/17  Resolved: 14/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None
Environment:

CLJS


Approval: Triaged

 Description   
(require '[clojure.core.async :refer (<!! go)])
(defrecord Foo [x])
(def f (Foo. 4))
(<!! (go f))
;; => #user.Foo{:x 4}
;; OK

(<!! (go #clojure.core.async.Foo{:x 4}))
;; CLJ: => #user.Foo{:x 4}   ;; expected
;; CLJS => {:x 4}            ;; wrong

Approach: Query the analyzer to know if we have a record or not.

(Copied from GitHub issue #13 - https://github.com/clojure/core.async/issues/13)



 Comments   
Comment by Ghadi Shayban [ 03/Aug/13 2:08 PM ]

0c6e663493 contains a fix on the Clojure side, would appreciate help porting to cljs.

Comment by David Nolen [ 08/Aug/13 8:35 AM ]

I think on the ClojureScript we'll have to query the analyzer to know if we have a record or not.

Comment by Alex Miller [ 14/Feb/17 10:49 AM ]

Not reproducible





[ASYNC-145] mix throws assertion error when many channels are added to a mix. Created: 08/Sep/15  Updated: 14/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Angus Fletcher Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: mix
Environment:

Found in core.async version 0.1.346.0-17112a-alpha


Attachments: Text File 0001-Change-channel-to-sliding-buffer-090815.patch     Text File 0001-Change-channel-to-sliding-buffer.patch    
Patch: Code

 Description   

When a large number of channels is added to a mix simultaneously, this error is thrown:

java.lang.AssertionError: Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.

We can reproduce the issue with the following in a REPL:

user> (require '[clojure.core.async :as a])
nil
user> (defn mixtest []
(let [out (a/chan 1024)
mix (a/mix out)]
(dotimes [i 2048]
(let [chan (a/chan)]
(a/admix mix chan)
(a/put! chan i)))))

(mixtest)
#'user/mixtestAssertionError Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.
(< (.size puts) impl/MAX-QUEUE-SIZE) clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:150)
user>

This is a consequence of the use of an unbuffered channel in the mix implementation. Since the channel's function is just to update the mix's internal data structure, using a windowed buffered channel appears to clear up this issue. Patch 0001-Change-channel-to-sliding-buffer.patch contains the proposed solution.



 Comments   
Comment by Angus Fletcher [ 08/Sep/15 2:32 PM ]

Update patch to include cljs fix as well.





[ASYNC-113] alt!! should not evaluate the :default expr if a non-default clause is selected Created: 13/Jan/15  Updated: 14/Feb/17  Resolved: 14/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Caleb Spare Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: alt
Environment:

Clojure 1.5.0, core.async 0.1.346.0-17112a-alpha


Approval: Triaged

 Description   

alt!! will always evaluate the expr given in the :default clause. For instance, consider this code:

(require '[clojure.core.async :as async])
(let [c1 (async/chan 1)
      c2 (async/chan 1)]
  (async/alt!!
    [[c1 "a"]] (println "first")
    [[c2 "a"]] (println "second")
    :default (println "default")))

It will print either "first" or "second", but also (always) "default". Reading the documentation, technically it doesn't say whether the non-selected clauses are evaluated (only that the value of the selected expr is returned) but it's certainly not the behavior I would expect.

I haven't checked whether alt! does the same thing as well.



 Comments   
Comment by Daniel Compton [ 27/Jul/15 7:37 PM ]

It looks to me like :default is expected to be a value, not a function, and println is being run as part of evaluating the alt!! expression, not as the return value.

Comment by Alex Miller [ 14/Feb/17 10:46 AM ]

Agreed - re Daniel's comment.





[ASYNC-32] onto-chan retains head of input sequence causing OutOfMemoryError Created: 06/Nov/13  Updated: 14/Feb/17  Resolved: 14/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Brian Lubeski Assignee: Unassigned
Resolution: Duplicate Votes: 3
Labels: memory
Environment:

org.clojure/core.async 0.1.242.0-44b1e3-alpha


Attachments: File patch.clj    
Patch: Code
Approval: Triaged

 Description   
(require '[clojure.core.async :as a])
(let [c (a/chan)]
  (a/onto-chan c (iterate inc 0))
  (while true
    (a/<!! c)))

onto-chan is holding on to the head of the input sequence as it is unfolded, resulting in an (eventual) OutOfMemoryError.

I've attached a diff showing changes I made to onto-chan that fixed the problem for me.



 Comments   
Comment by Colin Taylor [ 07/Oct/14 7:13 PM ]

Just to note, to-chan uses onto-chan so is similarly affected.
We ran into this, Brian's patch worked fine.

Comment by Alex Miller [ 10/Aug/15 3:22 PM ]

This seems like a viable change, but the patch needs a better test (even if not in the patch) and to be properly formatted for git apply (per http://dev.clojure.org/display/community/Developing+Patches).

Comment by Brian Lubeski [ 11/Aug/15 12:00 AM ]

This issue might be caused by ASYNC-138.

Comment by Nicola Mometto [ 15/Dec/15 6:12 PM ]

The fix for ASYNC-138 should fix this issue aswell and is more general

Comment by Alex Miller [ 14/Feb/17 10:35 AM ]

Duplicate of ASYNC-138





[ASYNC-134] async/alts!! with an empty vector throws ArrayIndexOutOfBoundsException Created: 27/Jul/15  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

I'm not sure if this is intended, but this code throws an exception.

(async/alts!! [])
;; => ArrayIndexOutOfBoundsException 1  clojure.core.async/random-array (async.clj:182)

Is this desired behaviour, or is there another way we could represent that this collection is empty? I'm thinking of the pattern where we take from any channel, and recur with that channel removed from the collection until the collection is empty. Perhaps this usage should be discouraged. If it should be discouraged, should it be documented in the docstring?



 Comments   
Comment by Alex Miller [ 28/Jul/15 9:08 AM ]

What would you expect this to do?

Comment by Daniel Compton [ 28/Jul/15 5:20 PM ]

You could return [nil nil]? That's not great either though. Documentation is probably a better fix.

Comment by Alex Miller [ 13/Feb/17 3:52 PM ]

Declining any change for now, may reconsider in future

Comment by Daniel Compton [ 13/Feb/17 4:27 PM ]

Would it be possible to add something to the docstring explaining this behaviour?

Comment by Daniel Compton [ 13/Feb/17 4:41 PM ]

"Declining any change" presumably includes docstrings too, so will leave this.





[ASYNC-184] (CLJS) Port ASYNC-169 fixes to cljs Created: 13/Feb/17  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: try-catch

Approval: Vetted

 Description   

ASYNC-169 addresses many try/catch/finally in go block issues on the Clojure side and should be ported to cljs side as well.






[ASYNC-183] Completion arity of transducer is called twice Created: 20/Jan/17  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jozef Wagner Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: chan, transducers


 Description   

completion arity (1) of a transducer function is called twice in a channel

(let [xf (fn [rf]
           (fn
             ([] (println "INIT") (rf))
             ([result] (println "COMPLETING") (rf result))
             ([result input] (println "STEP") (rf result input))))
      c (chan 10 xf)]
  (close! c)
  (println "RESULT" (<!! c)))

will print

COMPLETING
COMPLETING
RESULT nil

According to https://clojure.org/reference/transducers, it is probably a bug:

> A completing process must call the completion operation on the final accumulated value exactly once

Looks like completing fn is called at there places in core.async: https://github.com/clojure/core.async/blob/master/src/main/clojure/cljs/core/async/impl/channels.cljs#L122 https://github.com/clojure/core.async/blob/master/src/main/clojure/cljs/core/async/impl/channels.cljs#L146






[ASYNC-129] Channels with transducer using reduced don't work as intended Created: 09/Jun/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Ghadi Shayban
Resolution: Unresolved Votes: 0
Labels: chan, transducers
Environment:

CLJ 1.7.0-RC1, (CLJS not tested)



 Description   
(def ch (chan 1 (take 2)))

(put! ch 1)
(put! ch 2)
(put! ch 3)

(take! ch identity)

ClassCastException clojure.lang.PersistentVector cannot be cast to java.util.concurrent.locks.Lock clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:55)

This does not happen if a take is made before the first put so that the buffer size fits. The problem doesn't seem to be related only to the buffer size though.

(def ch (chan 1 (take-while true?)))

(put! ch true)
(put! ch false)
(put! ch false)

(take! ch identity)

ClassCastException clojure.lang.PersistentVector cannot be cast to java.util.concurrent.locks.Lock clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:55)






[ASYNC-153] Return value of pipe is not documented Created: 29/Nov/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Viktor Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring, pipe

Approval: Triaged

 Description   

pipe explicitly returns its "to" channel, but it's not documented.

If that's how it should work, this should be documented.






[ASYNC-69] How to better communicate "mix" lifecycle wrt coordination Created: 09/May/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Timothy Baldridge Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: mix

Approval: Vetted

 Description   

Confusion regarding 'mix' lifecycle. Users have a false expectation that a mix's output should magically close when its source channels close, not realizing its a dynamic coordination process. Actually ensuring correctness when using mult/mix/pub isn't so straightforward, people need to control consistency from the producer end, and not ad hoc from the consumer side (e.g. late taps of a mult).






[ASYNC-156] (CLJS) go block containing letfn does not compile Created: 30/Dec/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Gabe Johnson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: cljs, go-macro
Environment:

[org.clojure/clojure "1.8.0-RC3"]
[org.clojure/clojurescript "1.7.189"]
[org.clojure/core.async "0.2.374"]


Approval: Triaged

 Description   

The following compiles and runs correctly:

(ns cljs-letfn-go-bug.core
  (:require [clojure.core.async :refer [go]]))

(go
  (letfn [(foo [x] x)] (foo 1)))

However, this fails to compile with `clojure.lang.ExceptionInfo: bindings must be vector of even number of elements...`:

(ns cljs-letfn-go-bug.core
  (:require-macros [cljs.core.async.macros :refer [go]]))

(go
  (letfn [(foo [x] x)] (foo 1)))


 Comments   
Comment by Gabe Johnson [ 30/Dec/15 11:09 AM ]

Please forgive the markdown. I haven't used JIRA in quite some time and don't appear to have permissions to edit the description.





[ASYNC-90] Pub/sub leaks memory Created: 13/Sep/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ziyang Hu Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: pub
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]

java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) (reproducible on OpenJDK 7 as well)

OS X 10.9.4 (reproducible on Ubuntu Linux 14.04 as well)


Approval: Triaged

 Description   

The following code will cause OOME:

(require '[clojure.core.async :refer [chan close! <! pub sub unsub go timeout]])

(def p* (chan))

(def p (pub p* :topic))

(go
  (loop []
    (let [s (chan)
          t (rand-int Integer/MAX_VALUE)]
      (sub p t s)
      (<! (timeout 10))
      (unsub p t s)
      (close! s)
      (recur))))

(It grows slowly: to see the OOME in a reasonable amount of time, either give JVM very small
memory like 64m, or remove the timeout step.)

I tried to profile the code, and the reason seems to be that even though I
unsubed the channel from the port, something is still retained which causes
the heap to be used up.



 Comments   
Comment by Ziyang Hu [ 13/Sep/14 8:50 AM ]

Here is the problem:

https://github.com/clojure/core.async/blob/96de9a47ac511d9bb4309645a3bc594a2fc0c33a/src/main/clojure/clojure/core/async.clj#L826-L828

When unsub* is called, it just untaps the channel from the mult specified by the topic. The mult still remains in the atom called mults even if the mult has no taps left.

I can't think of a clean fix for this problem, since currently the channels which are tapping a mult aren't exposed, i.e., we currently have no way of knowing if a mult has any taps on it.

Comment by Ghadi Shayban [ 15/Oct/14 11:27 PM ]

I also cannot think of a clean fix, as mults do not expose their registrants. (The notion of "current" is a concurrent system is subtle)
Besides saying (and perhaps amending the docstring) that pubs are indeed resources, their footprint grows by the # of seen topics, and that resources should almost always be bounded.

Comment by Daniel Compton [ 11/Oct/15 5:02 PM ]

This bit us too in https://github.com/apa512/clj-rethinkdb/issues/97. My suggestion for a docstring addition is:

pubs are resources, and their footprint grows by the number of seen topics. unpub does not reclaim this resource. You should not use pub to subscribe to an unbounded number of topics.

Are there any changes/suggestions? Would you like me to create a patch for this?

Comment by Daniel Compton [ 11/Oct/15 5:04 PM ]

Also, is a pub able to be garbage collected when it's no longer used? When I was doing some limited testing, it didn't seem like it was, but I may be wrong.

Comment by Daniel Compton [ 15/Oct/15 4:45 PM ]

A partial workaround (I think?) is to use

(unsub-all pub topic)
if you know you don't need a topic anymore.





[ASYNC-92] go macro removes binding forms that are intialized with logical false value Created: 03/Oct/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Oleh Palianytsia Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: go
Environment:

org.clojure/core.async "0.1.346.0-17112a-alpha"


Attachments: File fix-async-92.diff    
Approval: Triaged

 Description   
(require '[clojure.core.async :as a])

(a/go (let [a nil] (a/alts! (if a <whatever> <whatever>)))) // Unable to resolve a
(a/go (let [a nil] (a/<! (if a <whatever> <whatever>))) // Unable to resolve a

Seems that 'go' macro removes falsely initialized symbols that are used as channels, because
in both cases there's exception, that says " Unable to resolve symbol: a in this context".



 Comments   
Comment by Willy Blandin [ 17/Oct/14 12:19 PM ]

Confirmed.
Bug was introduced between 0.1.278.0-76b25b-alpha and 0.1.295.0-9ea6ef-alpha.

Comment by Willy Blandin [ 17/Oct/14 12:27 PM ]

Worked around with:

(defmacro workaround-async-92
  "Hack to workaround core.async bug
   cf. http://dev.clojure.org/jira/browse/ASYNC-92"
  []
  ;; has to be a list
  `(do nil))

(let [a (workaround-async-92)]
  ...)
Comment by Leon Grapenthin [ 23/Oct/14 11:55 AM ]

modifies two methods of the RawCode inst so that they check:collected-locals in locals via contains? before ignoring them

Comment by Ghadi Shayban [ 23/Oct/14 5:19 PM ]

Hi Leon, thanks for the patch. Can you fill out a Contributor Agreement? http://clojure.org/contributing

Comment by Leon Grapenthin [ 24/Oct/14 7:17 AM ]

I did, yesterday. Got an automatic confirmation email saying Rich Hickey signed it. Anything else I should do with it?





[ASYNC-125] Closing a tap with a pending item blocks the mult input channel Created: 07/Jun/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Klaus Wuestefeld Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: mult
Environment:

org.clojure:clojure:1.7.0-alpha5
org.clojure:core.async:0.1.346.0-17112a-alpha


Approval: Triaged

 Description   

Closing a tap without a pending item is OK but closing a tap with a pending item blocks the mult input channel:

(require '[clojure.core.async :refer :all])
(def c (chan))
(def m (mult c))
(def t (chan))
(tap m t)
(>!! c :a)
(close! t)
(>!! c :b)  ; BLOCKS


 Comments   
Comment by Klaus Wuestefeld [ 07/Jun/15 6:22 PM ]

A more general case:

Doing this:

(go (println (>! c 42)))

and then closing c will cause the >! to block, instead of returning false.

If c is closed before that, >! will return false.

Is this race condition the intended behavior?





[ASYNC-42] (CLJS) (try (catch :default)) inside a cljs go block raises an exception Created: 07/Dec/13  Updated: 13/Feb/17

Status: Reopened
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Travis Vachon Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: try-catch

Attachments: Text File async_42.patch     Text File async_42.v2.patch     Text File try-catch-default.patch    

 Description   

test demonstrating issue and patch fixing it here: https://github.com/clojure/core.async/pull/39



 Comments   
Comment by Travis Vachon [ 08/Dec/13 8:51 PM ]

code + test

Comment by Travis Vachon [ 10/Dec/13 9:53 AM ]

add {{git am}}able patch

Comment by Yehonathan Sharvit [ 24/Aug/14 12:56 AM ]

What is the status of this issue?
Why isn't it merged into core.async code?

Comment by David Nolen [ 14/Oct/14 6:13 AM ]

Can we get an updated patch? Thanks.

Comment by Travis Vachon [ 14/Oct/14 2:41 PM ]

updated patch

Comment by David Nolen [ 14/Oct/14 7:33 PM ]

fixed https://github.com/clojure/core.async/commit/bd71ba73b43ea09e242d37a54eaaa0fed761504c

Comment by Kevin Downey [ 16/May/16 4:16 PM ]

the fix for this is not complete, the catch of js/Object in the catch in the io macro output needs to be a catch of :default

https://groups.google.com/d/msg/clojure/lGbbbCiLgcg/N2Ap9ddrBAAJ





[ASYNC-49] (CLJS) cljs IOC macro issue with double-dot form Created: 12/Jan/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Paul Butcher Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

ClojureScript 0.0-2138
core.async 0.1.267.0-0d7780-alpha



 Description   

This ClojureScript compiles and runs as expected:

(let [circle (dom/getElement "circle")]
  (go-loop [x 20]
    (<! (timeout 10))
    (set! (.-value (.-baseVal (.-cx circle))) x)
    (recur (inc x))))

But if I change the set! line to:

(set! (.. circle -cx -baseVal -value) x)

I get the following error:

Wrong number of args (3) passed to: core$-DOT

For further discussion see https://groups.google.com/d/topic/clojurescript/ONMaEho4K0c/discussion



 Comments   
Comment by Andrew S [ 05/Jun/15 8:49 AM ]

I too noticed this, but only when using the .. operator in a go loop. The linked thread further suggests this is a core.async problem with this operator.





[ASYNC-51] (CLJS) Core.async memory leak in Safari and Mobile Safari Created: 21/Jan/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Bruce Hauman Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mobile Safari on iOS 7.0; Safari 6.1 on OSX Lion



 Description   

Chaining together channels apparently leaks memory in Safari.

A statement such as

(def test [input-chan]
  (map< identity (map< identity (map< identity input-chan))))

will leak memory when the channels are used. The longer the chain the more memory.

I have created an example repository and an example page.

The repo and the example code is here:
https://github.com/bhauman/checkmemleak/blob/master/src/checking_safari_leak/core.cljs

The demonstration page is here:
This link will leak
http://rigsomelight.com/checkmemleak/?leak=true
This link will not
http://rigsomelight.com/checkmemleak

The leak is pretty darn severe.



 Comments   
Comment by Bruce Hauman [ 21/Jan/14 4:55 PM ]

I have just confirmed this is also a problem in Safari Version 7.0.1 (9537.73.11) on Mavericks.

Comment by Bruce Hauman [ 23/Jan/14 7:13 PM ]

I updated the example page.

http://rigsomelight.com/checkmemleak/index.html

This link wont leak:
http://rigsomelight.com/checkmemleak/index.html#comp-partial

This link will leak:
http://rigsomelight.com/checkmemleak/index.html#map<

There is navigation to try the different implementations and optimization modes.

To be clear map<-chain, custom-map, map<, and map> all exhibit the leak in :none and simple optimization modes.

Comment by David Nolen [ 14/Oct/14 6:00 AM ]

To make sure this ticket has all the necessary information - if I remember correctly even advanced optimization is affected? Am I correct Bruce?

Comment by Bruce Hauman [ 15/Oct/14 12:53 PM ]

Yes, it affected advanced optimization. I haven't checked if this is still an issue.





[ASYNC-54] MAX-QUEUE-SIZE has a wrong type-hint Created: 11/Feb/14  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File 0001-Fix-MAX-QUEUE-SIZE-type-hint-def-evaluates-the-metad.patch    

 Description   

Since `def` evaluated the metadata on the symbol by design, primitive type hints needs to be quoted to avoid resolving to the homonimous clojure.core function



 Comments   
Comment by Alex Miller [ 23/Jun/14 3:22 PM ]

Actually, I think no hint is needed here at all and it should just be removed. Clojure will just inline a primitive long for the MAX-QUEUE-SIZE where it is used.

Comment by Alex Miller [ 13/Feb/17 4:28 PM ]

Applied patch





[ASYNC-63] (CLJS) Variable called 'new' in vector in go block in CLJS causes "Object has no method 'call'" error Created: 07/Apr/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: James Henderson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: cljs
Environment:

ClojureScript 0.0-2202, core.async 0.1.267.0-0d7780-alpha, openjdk 7u51


Attachments: File compiled-cljs.js     File test_cases.clj    

 Description   

I seem to be having trouble with the combination of:

  • a variable named 'new'
  • wrapped in a vector
  • after a <! call in a go block.

Removing any of these conditions seems to work fine.

I've attached a few minimal examples in test_cases.clj and the JavaScript that the first error compiles down to in compiled-cljs.js - the errors occur on line 4 (I think inst_39771 is 21 in this case)

An obvious workaround is not to name the variable 'new'

Please let me know if you need anything more from me, and please go easy on me if this is in the wrong place/badly labelled etc - it's my first Clojure JIRA bug report

Thanks,

James



 Comments   
Comment by James Henderson [ 07/Apr/14 4:07 PM ]

Sorry, just spotted there's a newer release of core.async - I've reproduced this with 0.1.278.0-76b25b-alpha as well.

James





[ASYNC-73] (CLJS) try-catch-finally broken inside go-blocks Created: 21/Jun/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Moritz Ulrich Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: try-catch
Environment:

clojure 1.6.0, clojurescript 0.0-2234, core.async 0.1.303.0-886421-alpha1



 Description   

The following will cause the failed assertion (ioc_helpers.cljs:155)
when evaluated as a whole, and will correctly catch the Error when
just the `try' gets evaluated. The `finally' block runs only if the
inner block is evaluated:

(go
  (try
    (throw (js/Error. "asdf"))
    (catch ExceptionInfo e
      (println "ExceptionInfo"))
    (catch js/Error e
      (println "js/Error"))
    (finally
      (println "finally"))))

Another notable observation is that changing the order of the `catch'
blocks will change the behavior: If the (catch js/Error ...) is the
first catch block, it will work just as expected.



 Comments   
Comment by Kevin Downey [ 17/May/16 7:48 PM ]

this appears to be very similar to http://dev.clojure.org/jira/browse/ASYNC-169, async-169 is on the clojure side of the library, but it looks like both sides have similar problematic exception handling logic





[ASYNC-77] (CLJS) StackOverflowError in go macro with cemerick.cljs.test Created: 04/Jul/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Daniel Skarda Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]
[org.clojure/clojurescript "0.0-2261"]
[com.cemerick/clojurescript.test "0.3.1"]



 Description   

I got an StackOverflowError during compilation of CLJS tests using clojurescript.test from Chas Emerick.

I narrowed the problem to following code:

(ns async.overflow.cljs
  (:require-macros [cljs.core.async.macros :refer [go]]
                   [cemerick.cljs.test :refer [deftest is]])
  (:require [cemerick.cljs.test]))

(deftest foobar
  (go
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))))

If you do not get the exception, try to increase number of repetitions of 'is' call.

I was not able to replicate the issue in CLJ. cemerick.cljs.test/is macro is probably far more complex than original CLJ version in clojure.test/is.






[ASYNC-79] (CLJS) go macro not correctly transforming (case) within a macro Created: 23/Jul/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tom Locke Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: cljs, macro
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/clojurescript "0.0-2234"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]


Attachments: File core_async_bug.tgz    

 Description   

(case-let) is a macro to handle messages of the form [:message-tag, arg1, arg2, ...] with the arguments bound to local variables. It fails to work correctly when used within a go block. Note that a simple macro with a case, e.g. (defmacro my-case [expr & cases] `(case ~expr ~@cases)) does work.

(Sample project attached)

(case-let) definition:

(ns core-async-bug.macros)

(defmacro case-let [expr & cases]
  (let [msg (gensym)]
    `(let [~msg ~expr]
       (case (first ~msg)
         ~@(apply concat (for [[key [args & body]] (partition 2 cases)]
                    [key `(let [~args (rest ~msg)] ~@body)]))))))

ClojureScript test code:

(ns core-async-bug.core
  (:require-macros [cljs.core.async.macros :refer [go]]
                   [core-async-bug.macros :refer [case-let]])
  (:require [cljs.core.async :refer[<! put! chan]]))

(enable-console-print!)

; go block with manual case + lets - works
(let [c (chan)]
  (go
    (let [msg (<! c)]
      (case (first msg)
        :a (let [[x] (rest msg)] (println "First :a" x))
        :b (let [[y] (rest msg)] (println "First :b" y)))))
  (put! c [:b 123]))

; case-let outside of go - works
(case-let [:b 123]
  :a ([x] (println "Second :a" x))
  :b ([y] (println "Second :b" y)))

; case-let inside go - broken
(let [c (chan)]
  (go
    (case-let (<! c)
      :a ([x] (println "Third :a" x))
      :b ([y] (println "Third :b" y))))
  (put! c [:b 123]))

Browser console output:

Second :b 123
First :b 123
Third :a 123          <-- Should not be there!
Third :b 123


 Comments   
Comment by Tom Locke [ 23/Jul/14 3:46 AM ]

More discussion here: https://groups.google.com/forum/#!topic/clojurescript/w21nNWkKR-c

Comment by Tom Locke [ 23/Jul/14 1:58 PM ]

I've discovered an easy workaround for this problem. During macro-expansion core names like case become fully qualified, i.e. cljs.core/case, and it seems that the go macro then fails to recognise the case as such. Replacing case with ~'case in the definition of let-case fixes the problem.

I would hope this leads to an easy fix for someone who knows the core.async codebase.

The working macro is:

(defmacro case-let [expr & cases]
  (let [msg (gensym)]
    `(let [~msg ~expr]
       (~'case (first ~msg)
         ~@(apply concat (for [[key [args & body]] (partition 2 cases)]
                    [key `(let [~args (rest ~msg)] ~@body)]))))))




[ASYNC-91] (CLJS) 'and' does not short circuit within go block in clojurescript Created: 14/Sep/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jan Krueger Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Environment:

core.async 0.1.338.0-5c5012-alpha



 Description   

I have the following piece of ClojureScript code within a go block:

(cond
(and (vector? x) (= (first x) :some-key)) ...)

This generates the following piece of JavaScript code for the cond case:

if (40 === f) { return d = b[9], f = cljs.core.vector_QMARK_.call(null, d), d = cljs.core.first.call(null, d), d = cljs.core._EQ_.call(null, d, new cljs.core.Keyword(null, "some-key", "some-key", -978969032)), cljs.core.truth_(f && d) ? b[1] = 42 : b[1] = 43, new cljs.core.Keyword(null, "recur", "recur", -437573268); }

This looks to me like both and arguments would actually get evaluated. As a result my code crashes whenever it hits this cond case and 'x' is not seqable.



 Comments   
Comment by Francis Avila [ 25/Sep/14 2:52 AM ]

CLJS-864 is likely the same issue. (It also has a small test project to reproduce.)

Comment by Rangel Spasov [ 07/Nov/16 4:29 AM ]

This happened to me as well with the latest version of core.async on ClojureScript 0.2.395.

I would bump this to not minor since it severely breaks the promise that (and ...) makes.

(def cards :loading)
(go (and (vector? cards) (< 0 (count cards))))

This throws an exception like this:

No protocol method ICounted.-count defined for type cljs.core/Keyword: :loading

I was able to go around this by nesting the checks like this:
(go (if (vector? cards)
(if (< 0 (count cards))
(println "counting cards..."))))





[ASYNC-96] (CLJS) FixedBuffer's full checking in cljs Created: 13/Oct/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nguyễn Tuấn Anh Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File cljs-fixed-buffer-overflow.patch    
Patch: Code and Test

 Description   

In the cljs implementation of fixed buffer, "full?" uses "=" instead of ">=". This makes it incorrect after an overflow.






[ASYNC-97] (CLJS) alts! sporadically failing to unblock despite channel activity (Safari 7) Created: 15/Oct/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Oliver Charles Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File gistfile1.txt     File main.cljs     File socket.cljs    

 Description   

Hi all, bit of a tricky bug to report here... We're seeing some problems with using core.async in ClojureScript on Safari 7. Our application is built around a large event loop that blocks on a message from one of many channels that correspond to user activity or API calls. The problem seems to lie within this event loop - we are using alts! to pull a message out of any available channel, but sometimes logging shows that we reach alts! and never unblock. However, with a little more logging, I can see that there are subsequent writes to one of the channels in the list of channels passed to alts!, so I'm not really sure what's going on.

That's the high level overview, now on to some code.

Our main event loop is as follows:

(log "Entering main event loop.")
  (go
    (while true
      (log "alts! channel hashes: " (map hash (:channels @app)))
      (let [[message channel] (alts! (seq (:channels @app)))]
        (log "alts! unblocked, calling our process-message")
        (swap! app process-message message channel)
        (log "process-message completed, looping"))))

process-message here is our a function internal to our application, but I don't think it's details are necessarily important. In the scenario where Safari gets stuck, the log looks like:

[Log] process-message completed, looping (main.js, line 62)
[Log] alts! channel hashes:  (16 12 19 33) (main.js, line 82)
[Log] Socket connected. (socket.js, line 309)
[Log] put! to channel with hash  19 (socket.js, line 86)
[Log] The message is [:metronome [:staff [{:description nil, :deletable true, :email nil, :isAdmin true, :isTrainer false, :telephone nil, :name "Fynder Admin", :picture nil, :userId 1} {:description nil, :deletable fa... (socket.js, line 87)
[Log] put! callback gave us true (socket.js, line 89)
[Debug] Metronome: staff data decoded. put! complete.: 12.282ms (socket.js, line 93)
[Log] put! to channel with hash  19 (socket.js, line 86)
[Log] The message is [:metronome [:class-types [{:deletable false, :picture nil, :name "CycleCore", :id 2, :description "CycleCore is a 55-minute dual workout concept that combines 30 minutes of intense cardiovascular ... (socket.js, line 87)
[Log] put! callback gave us true (socket.js, line 89)
[Debug] Metronome: class-types data decoded. put! complete.: 1.288ms (socket.js, line 93)
[Log] put! to channel with hash  19 (socket.js, line 86)
[Log] The message is [:metronome [:locations [{:studios [{:deletable false, :name "Kensington", :id 1, :locationId 1, :description "Studio (11a) sits just off Stratford Road in Stratford Studios. To find us, just pass ... (socket.js, line 87)
[Debug] Metronome: locations data decoded. put! complete.: 0.884ms (socket.js, line 93)

Note that we see a log entry for "alts! channel hashes", but we never seen "alts! unblocked". However, note the list of hashes passed to alts!. Channel 19 is mentioned, but subsequently we put! to channel 19... yet we still don't get unblocked. Something that also strikes me as suspicious, is that while we're blocked at alts!, two calls to put! have succeeded immediately, for a channel that is bounded to contain only one element at a time. Maybe I'm misunderstanding something, but I wouldn't expect the immediate-put callback to be invoked more than once. Interestingly that last put! doesn't invoke the callback.

Unfortunately, reproduction of this bug is reasonably difficult. I can somewhat reliably reproduce it by quitting Safari, re-opening it, and navigating to the dev server. About 1 in 15 attempts get stuck in this fashion. I wondered if it was something to do with Safari's MessageChannel implementation - you can see in the log entries where nexttick.js calls its callback, which seems to be how dispatch is working in my browser.

I'd be very happy to help provide any more information that's useful, but this problem is now outside my ability to debug it. While the code is proprietary, I'd be happy to temporarily add people to the Github project in order to try and get this fixed. We have development APIs servers that you can point at, so it should be just a case of running lein cljs.

I've attached our code for our Socket.io wrapper and our main event loop. Sadly I do not yet have a minimal test-case - I wouldn't really know where to begin.



 Comments   
Comment by Oliver Charles [ 15/Oct/14 7:37 AM ]

I went deep into the guts of the Google Closure library and changed getSetImmediateEmulator_ to:

goog.async.nextTick.getSetImmediateEmulator_ = function() {
  // Fall back to setTimeout with 0. In browsers this creates a delay of 5ms
  // or more.
  return function(cb) {
    goog.global.setTimeout(cb, 0);
  };
};

and I haven't been able to get it stuck. So maybe MessageChannel has problems in Safari...

Comment by Ghadi Shayban [ 15/Oct/14 10:40 PM ]

Hi Oliver, seems like a race, and we'll figure this out.

Would you mind compare running upon 0.1.319.0-6b1aca-alpha vs 0.1.346.0-17112a-alpha ?

alts! should be passed an indexed collection/vector btw. Passing a seq wouldn't cause this bug, just something to note.

Comment by Oliver Charles [ 16/Oct/14 6:53 AM ]

Hi Ghadi,

0.1.319.0-6b1aca-alpha is what the initial report was against - I should have mentioned that. So 0.1.319.0-6b1aca-alpha does get stuck.

0.1.346.0-17112a-alpha however does not get stuck, which is odd - as I'm sure I tried upgrading to this! I've tried on two Macs that are normally problematic, and they didn't get stuck once. I'm pushing this out to more of our testers and will see what happens.

Comment by Oliver Charles [ 16/Oct/14 7:16 AM ]

Aha, I knew it wouldn't be that easy! Upon releasing this to production, it immediately froze again. The dev server runs with very different optimisations though, so I'm going to build a production release and serve that locally - will see what happens there.

Comment by Oliver Charles [ 16/Oct/14 7:28 AM ]

Yep, definitely a problem with optimisations. Here is my Shadow Build configuration

(ns fynder.shadowbuild
  (:require [clojure.java.io :as io]
            [shadow.cljs.build :as cljs]))

(defn define-modules [state]
  (-> state
      (cljs/step-configure-module :cljs '[cljs.core clojure.walk clojure.string cljs.reader cljs.core.async] #{})
      (cljs/step-configure-module :test-support '[inflections.core no.en.core enfocus.bind fynder.winchan] #{:cljs})
      (cljs/step-configure-module :devel '[fynder.devel] #{:cljs})
      (cljs/step-configure-module :admin '[fynder-admin.main] #{:cljs})
      (cljs/step-configure-module :trainer '[fynder-trainer.main] #{:cljs})
      (cljs/step-configure-module :mobile '[fynder-mobile.main] #{:cljs})
      (cljs/step-configure-module :sweatybetty '[fynder-sweatybetty.main] #{:cljs})
      (cljs/step-configure-module :loader '[fynder-loader.loader] #{:cljs})))

(defn dev
  "build the project, wait for file changes, repeat"
  [& args]
  (let [state (-> (cljs/init-state)
                  (cljs/enable-source-maps)
                  (assoc :optimizations :advanced
                         :pretty-print false
                         :work-dir (io/file "target/cljs-work")
                         :public-dir (io/file "resources/dev")
                         :public-path "")
                  (cljs/step-find-resources-in-jars)
                  (cljs/step-find-resources "src/cljs/")
                  (cljs/step-finalize-config)
                  (cljs/step-compile-core)
                  (define-modules))]
    ;; compile, flush, reload, repeat
    (loop [state state]
      (let [new-state (try
                        (-> state
                            (cljs/step-compile-modules)
                            (cljs/flush-unoptimized)
                            (cljs/wait-and-reload!))
                        (catch Throwable t
                          (prn [:failed-to-compile t])
                          (.printStackTrace t)
                          (cljs/wait-and-reload! state)))]
        (recur new-state)))))

(defn prod
  "build the project, wait for file changes, repeat"
  [& args]
  (-> (cljs/init-state)
      (cljs/enable-emit-constants)
      (cljs/enable-source-maps)
      (assoc :optimizations :advanced
             :pretty-print false
             :work-dir (io/file "target/cljs-work")
             :public-dir (io/file "resources/prod")
             :externs ["react/externs/react.js"
                       "externs/base32.js"
                       "externs/jquery.js"
                       "externs/react_addons.js"
                       "externs/fastclick.js"
                       "externs/socket.io.js"
                       "externs/moment.js"
                       "externs/papa.js"
                       "externs/markdown.js"
                       "externs/xss.js"
                       "externs/facebook.js"
                       "externs/checkout.js"])
      (cljs/step-find-resources-in-jars)
      (cljs/step-find-resources "src/cljs/")
      (cljs/step-finalize-config)
      (cljs/step-compile-core)
      (define-modules)
      (cljs/step-compile-modules)
      (cljs/closure-optimize)
      (cljs/flush-to-disk)
      (cljs/flush-modules-to-disk)))

If I run in the dev profile, then I can't get it stuck. If I switch over to the production profile (and serve the result of lein publish with python's SimpleHTTPServer), then Safari does get stuck.





[ASYNC-108] (CLJS) cljs to-chan hangs on infinite lazy sequences Created: 20/Nov/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Felix Andrews Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: cljs
Environment:

[org.clojure/core.async "0.1.346.0-17112a-alpha"]


Patch: Code

 Description   

e.g. this hangs the browser:

(go-loop [ch (to-chan (range))]
(println (<! ch))
(<! (timeout 1000))
(recur ch))

The problem is in
https://github.com/clojure/core.async/blob/53bf7866f195e6ba247ff7122b99784e66e9f1bb/src/main/clojure/cljs/core/async.cljs#L362
where the order of arguments to bounded-count is wrong.






[ASYNC-109] Clarify timeout doc to mention that close! should not be called on a timeout channel Created: 11/Dec/14  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ryan Neufeld Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: docstring, documentation

Attachments: Text File clarify-timeout-doc.patch    
Patch: Code

 Description   

After running into a race-condition involving closed timeout channels, it seems like it would be appropriate to mention that `close!` should never be called on a timeout channel in its docstring. The attached patch tweaks the doc string to that effect. Please advise if you'd like the wording changed a bit.



 Comments   
Comment by Howard Lewis Ship [ 12/Dec/14 11:03 AM ]

Alternately/additionally, it would be nice if close! on a timeout channel would throw an exception.

Comment by Erik Assum [ 30/Dec/14 8:25 AM ]

or alternately, make it a no-op?





[ASYNC-116] (CLJS) Convert core.async ClojureScript tests to cljs.test Created: 12/Feb/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: David Nolen Assignee: David Nolen
Resolution: Unresolved Votes: 1
Labels: None


 Description   

This is a good dog-fooding opportunity for cljs.test's async testing functionality.






[ASYNC-117] (CLJS) let-bindings unavailable in #js literals in go-blocks Created: 13/Feb/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Rasmus Erik Voel Jensen Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Environment:

[org.clojure/clojure "1.7.0-alpha5"]
[org.clojure/clojurescript "0.0-2760"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]



 Description   

As far as I can see, let bindings are unavailable in #js-literals within go-blocks, ie:

(go (let [a 1] (js/console.log #js[a])))

prints `[ undefined ]` instead of `[1]`

It happens both with `#js[..]` and `#js{..}`

To make it easier to reproduce the bug, I've created a minimal repository github.com/rasmuserik/cljs-bug with project.clj etc.






[ASYNC-118] (CLJS) A let-binding named 'arguments' not bound in go block on nodejs Created: 14/Mar/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Sean Doig Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: binding, cljs, nodejs
Environment:

OS X 10.10, node 0.10.35, Clojure 1.6.0, ClojureScript 0.0-3030, core.async 0.1.346.0-17112a-alpha



 Description   

Discovered this while using tools.cli 0.3.1 where it's handy to destructure one of their returned maps which has a key :arguments, but you can reproduce it like so:

(let [arguments [:a :b]]
    (println "Args Sync:" arguments)
    (go (println "Args Async:" arguments)))

which prints

Args Sync: [:a :b]
Args Async: #js {:0 #js [#<function (state_12638){
switch(arguments.length){
case 0:
return dls$rip$core$go_BANG__$_state_machine__6252__auto____0.call(this);
case 1:
return dls$rip$core$go_BANG__$_state_machine__6252__auto____1.call(this,state_12638);
}
throw(new Error('Invalid arity: ' + arguments.length));
}> 1 nil nil nil nil #<[object Object]>]}

I haven't tested this in the browser yet, so it might not be limited to node.



 Comments   
Comment by Sean Doig [ 14/Mar/15 12:34 PM ]

Just tried, can't reproduce in Chrome.





[ASYNC-119] (CLJS) ClojureScript: combine cljs.core.async.macros and cljs.core.async Created: 15/Mar/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: J. S. Choi Assignee: Unassigned
Resolution: Unresolved Votes: 7
Labels: None

Attachments: Text File ASYNC-119.patch    

 Description   

The macros in cljs.core.async.macros can now be moved into the cljs.core.async namespace, now that combined .cljs/.clj namespaces are possible in ClojureScript. This, along with ClojureScript 0.0-2755's recent improvements, would enable users to use `(require [cljs.core.async :refer [go go-loop ...])` without having to distinguish between functions and macros, and without having to use `require-macros` at all.



 Comments   
Comment by J. S. Choi [ 14/Oct/15 9:19 PM ]

ASYNC-142 (http://dev.clojure.org/jira/browse/ASYNC-142) proposes renaming the ClojureScript namespace from cljs.core.async to clojure.core.async, which would supersede this improvement.

Personally, I would love for ASYNC-142 to happen, though if it ends up being held back for a major change across all clojure.core projects, as Alex Miller suggested on its ticket, then this improvement may still be worth doing in the meantime.

Comment by Tim Gilbert [ 19/Aug/16 1:35 PM ]

There have been a few recent developments to ClojureScript that might make this a bit more feasible now.

Notably, the 1.9.216 ClojureScript will now import cljs.* namespaces which are referenced as clojure.* via CLJS-1692, and it will infer which referenced things from a namespace are macros versus functions via CLJS-1507.

One can currently import the functions in cljs.core.async using the same syntax as Clojure:

(ns my.app 
  (:require [clojure.core.async :refer [<! >!]]))

To get the macros, however, a separate (:require-macros) call is needed:

(ns my.app 
  (:require-macros [cljs.core.async.macros :refer [go go-loop]]))
  (:require [clojure.core.async :refer [<! >!]]))

If the macros were moved from cljs.core.async.macros into cljs.core.async, one could use the same syntax from either Clojure or ClojureScript:

(ns my.app 
  (:require [clojure.core.async :refer [<! >! go go-loop]]))

On the ClojureScript Slack channel David Nolen indicated that this would be a breaking change, and with questionable value relative to that. I think it might be worth talking about, though - in theory, at least, I could imagine an implementation that moves the macros into cljs.core.async and leaves aliases or shim versions around in the old cljs.core.async.macros namespace for backwards compatibility.

Comment by Juho Teperi [ 19/Aug/16 2:31 PM ]

Here is a patch which moves the macros but adds shim vars to the macros ns. The idea for shim vars has been take from Potemkin: https://github.com/ztellman/potemkin/blob/master/src/potemkin/namespaces.clj

There is one minor case where this will break user's code:

(:require-macros [cljs.core.async.macros :as a)
(:require [cljs.core.async :as a])

It has been possible to use same alias for macro and cljs namespace, but this won't work if macro namespace exists with the same name as cljs namespace. BUT I don't think this is a problem, because this has never been intended to be supported.

Most people probably use :require-macros with :refer, and this continues to work after this patch:

(:require-macros [cljs.core.async.macros :refer [go])
(:require [cljs.core.async :as a])

Tested with clojurescript 1.9.89 and 1.9.225.





[ASYNC-121] (CLJS) compilation warning when calling 'satisfies?' inside a go block Created: 21/Apr/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Yehonathan Sharvit Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: compiler
Environment:

[org.clojure/core.async "0.1.346.0-17112a-alpha"]
[org.clojure/clojurescript "0.0-3196"]



 Description   

This piece of code is causing the following compilation warning:

(defprotocol a)
(defrecord b [])
(go (satisfies? a (b.)))

WARNING: Use of undeclared Var cljs-explore.me/bit_4884auto_ at line 12 src/cljs_explore/me.cljs






[ASYNC-123] Channel operations fail in for comprehension. Created: 25/May/15  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Christian Weilbach Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]



 Description   
(<!! (go (for [a (<! (go [1 2 3]))]
           a)))
;=> (1 2 3)

works fine. But:

(go (for [a [1 2 3]
          :let [b (<! (go a))]]
      b))
java.lang.IllegalArgumentException
   No method in multimethod '-item-to-ssa' for dispatch value: :fn

                  MultiFn.java:  160  clojure.lang.MultiFn/getFn
                  MultiFn.java:  227  clojure.lang.MultiFn/invoke
                ioc_macros.clj:  492  clojure.core.async.impl.ioc-macros/item-to-ssa
                ioc_macros.clj:  547  clojure.core.async.impl.ioc-macros/let-binding-to-ssa/fn
                ioc_macros.clj:  122  clojure.core.async.impl.ioc-macros/all/fn/fn
               ArrayChunk.java:   58  clojure.lang.ArrayChunk/reduce
                 protocols.clj:   98  clojure.core.protocols/fn
                 protocols.clj:   19  clojure.core.protocols/fn/G
                 protocols.clj:   31  clojure.core.protocols/seq-reduce
                 protocols.clj:   54  clojure.core.protocols/fn
                 protocols.clj:   13  clojure.core.protocols/fn/G
                      core.clj: 6289  clojure.core/reduce
                ioc_macros.clj:  120  clojure.core.async.impl.ioc-macros/all/fn
                ioc_macros.clj:  554  clojure.core.async.impl.ioc-macros/eval22522/fn/fn
                ioc_macros.clj:  600  clojure.core.async.impl.ioc-macros/eval22585/fn/fn
                ioc_macros.clj:  806  clojure.core.async.impl.ioc-macros/parse-to-state-machine/fn
                ioc_macros.clj:   80  clojure.core.async.impl.ioc-macros/get-plan
                ioc_macros.clj:  802  clojure.core.async.impl.ioc-macros/parse-to-state-machine
                ioc_macros.clj: 1066  clojure.core.async.impl.ioc-macros/state-machine
                     async.clj:  384  clojure.core.async/go
                   RestFn.java:  442  clojure.lang.RestFn/invoke
                      Var.java:  388  clojure.lang.Var/invoke
                      AFn.java:  160  clojure.lang.AFn/applyToHelper
                      Var.java:  700  clojure.lang.Var/applyTo
                 Compiler.java: 6552  clojure.lang.Compiler/macroexpand1
                 Compiler.java: 6630  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6445  clojure.lang.Compiler/analyze
                 Compiler.java: 6406  clojure.lang.Compiler/analyze
                 Compiler.java: 3719  clojure.lang.Compiler$InvokeExpr/parse
                 Compiler.java: 6646  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6445  clojure.lang.Compiler/analyze
                 Compiler.java: 6406  clojure.lang.Compiler/analyze
                 Compiler.java: 5782  clojure.lang.Compiler$BodyExpr$Parser/parse
                 Compiler.java: 5217  clojure.lang.Compiler$FnMethod/parse
                 Compiler.java: 3846  clojure.lang.Compiler$FnExpr/parse
                 Compiler.java: 6642  clojure.lang.Compiler/analyzeSeq
                 Compiler.java: 6445  clojure.lang.Compiler/analyze
                 Compiler.java: 6700  clojure.lang.Compiler/eval
                 Compiler.java: 7130  clojure.lang.Compiler/load
                          REPL:    1  geschichte.p2p.hooks/eval32348
                 Compiler.java: 6703  clojure.lang.Compiler/eval
                 Compiler.java: 6666  clojure.lang.Compiler/eval
                      core.clj: 2927  clojure.core/eval
                      main.clj:  239  clojure.main/repl/read-eval-print/fn
                      main.clj:  239  clojure.main/repl/read-eval-print
                      main.clj:  257  clojure.main/repl/fn
                      main.clj:  257  clojure.main/repl
                   RestFn.java: 1523  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   67  clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn
                      AFn.java:  152  clojure.lang.AFn/applyToHelper
                      AFn.java:  144  clojure.lang.AFn/applyTo
                      core.clj:  624  clojure.core/apply
                      core.clj: 1862  clojure.core/with-bindings*
                   RestFn.java:  425  clojure.lang.RestFn/invoke
        interruptible_eval.clj:   51  clojure.tools.nrepl.middleware.interruptible-eval/evaluate
        interruptible_eval.clj:  183  clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
        interruptible_eval.clj:  152  clojure.tools.nrepl.middleware.interruptible-eval/run-next/fn
                      AFn.java:   22  clojure.lang.AFn/run
       ThreadPoolExecutor.java: 1145  java.util.concurrent.ThreadPoolExecutor/runWorker
       ThreadPoolExecutor.java:  615  java.util.concurrent.ThreadPoolExecutor$Worker/run
                   Thread.java:  745  java.lang.Thread/run

Possibly this is due to the function boundaries of go-blocks, but I think it would be nice if this would work similar to the binding for `a`. I have basically one big for-comprehension to map a highly nested seq and rewriting it will make the code much more verbose. Note that I know that I cannot execute channel ops inside the body of the for-comprehension due to the function boundaries in lazy seqs. This is no problem, but I cannot fix the for-comprehension bindings easily.
I played around with extending the -item-to-ssa multimethod, but some pointers of how to fix it properly would be nice. Then I could try to implement a patch.



 Comments   
Comment by Christian Weilbach [ 14/Jun/15 9:11 AM ]

This is actually a problem for many more cases, only a take operation in the first binding works, because the for-comprehension macro in core.clj wraps every following binding in a lazy-seq. The best fix to me seemed to be a custom version of the for-comprehension macro.

I have simplified the macro to remove lazyness and rewritten it with go-loops. This works fairly well, but it is eager and channel operations are sideeffects, so channels representing constants on which iteration happens repeatedly should be put in an immutable collection before (e.g. let binding around the comprehension).

https://github.com/ghubber/geschichte/blob/c57bacdcf8e3335d17dc370941384c559ae80373/src/cljx/geschichte/go_for.cljx#L25

Comment by Christian Weilbach [ 18/Oct/15 12:40 PM ]

Can be closed, is fixed with go-for for me. The irritation came from the first binding working as a channel op, while nested ones are wrapped in lazy-seq fns.

Comment by Alex Miller [ 13/Feb/17 4:17 PM ]

closed at op's request





[ASYNC-165] (CLJS) the binding macro/let binding is wrongly inlined Created: 06/Apr/16  Updated: 13/Feb/17

Status: In Progress
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Christian Weilbach Assignee: Kevin Downey
Resolution: Unresolved Votes: 0
Labels: compiler
Environment:

[org.clojure/clojure "1.7.0"]
[org.clojure/clojurescript "1.8.34"]
[org.clojure/core.async "0.2.374"]


Attachments: Text File 0001-ASYNC-165-if-a-local-aliases-a-global-actually-do-th.patch     Text File 0002-ASYNC-165-if-a-local-aliases-a-global-actually-do-th.patch     Text File 0003-ASYNC-165-if-a-local-aliases-a-global-actually-do-th.patch    
Patch: Code
Approval: Triaged

 Description   

(def ^:dynamic foo 42)

(go
(let [old foo]
(set! foo 45)
(println old foo)
(set! foo old)))

leaves the binding with the value 45 (same code as with binding+with-redefs). The problem is that the let binding is somehow inlining the reference to foo. For instance the println statement compiles to:

...
var inst_43089 = cljs.core.println.call(null,full.async.foo,full.async.foo);
...

I am currently having a look at ioc_macros.clj, but I couldn't find the problematic part yet. Any hints are helpful.



 Comments   
Comment by Kevin Downey [ 06/Apr/16 6:21 PM ]

for the curious the cleaned up macro expansion of the above (in clojurescript) is:

(let* [c__9201__auto__ (chan 1)]
      (run
        (fn []
          (let [f__9202__auto__ (let [switch__9186__auto__ (fn [state_9258]
                                                             (let [state_val_9259 (aget state_9258 1)]
                                                               (cond
                                                                (== state_val_9259 1) (let [inst_9254 (set! foo 45)
                                                                                            inst_9255 (println foo foo)
                                                                                            inst_9256 (set! foo foo)
                                                                                            state_9258 (aset-all! state_9258 7 inst_9255 8 inst_9254)]
                                                                                        (return-chan state_9258 inst_9256)))))]
                                  (fn state-machine__9187__auto__
                                    ([] (aset-all! (make-array 9) 0 state-machine__9187__auto__ 1 1))
                                    ([state_9258] (let [ret-value__9188__auto__ (try
                                                                                  (loop []
                                                                                    (let [result__9189__auto__ (switch__9186__auto__ state_9258)]
                                                                                      (if (keyword-identical? result__9189__auto__ :recur)
                                                                                        (recur)
                                                                                        result__9189__auto__)))
                                                                                  (catch js/Object ex__9190__auto__
                                                                                    (aset-all! state_9258 5 ex__9190__auto__)
                                                                                    (process-exception state_9258)
                                                                                    :recur))]
                                                    (if (keyword-identical? ret-value__9188__auto__ :recur)
                                                      (recur state_9258)
                                                      ret-value__9188__auto__)))))
                state__9203__auto__ (-> (f__9202__auto__)
                                        (aset-all! USER-START-IDX c__9201__auto__))]
            (run-state-machine-wrapped state__9203__auto__))))
      c__9201__auto__)

the issue is definitely present in the macro expansion

Comment by Kevin Downey [ 06/Apr/16 7:00 PM ]

it looks like the issue is, let binds essentially disappear at runtime
because the ioc macros bind every expression to a name, so let binds
are just mapped to those names at compile time. in that mapping global
names essentially map to themselves, so let bound names that get their
value from a global just disappear and become references to the
global.

If you look at the `:symbol` case for `-item-to-ssa` in
ioc_macros.clj, there is a commented out `(add-instruction (->Const
x))`, if you uncomment out that and comment out the fn above it, I
think what you get out has the behavior you are looking for, at the
cost of creating a local for every global read.

You could do some kind of picking of the behavior based on if the
globals are declared to be dynamic, but that would still leave issues
for with-redefs. In JVM clojure with-redefs works on global names
regardless of if they are dynamic or not. You could generate some kind
of dirty set if you see a set! of a global, and use that to toggle the
behavior, but I think (not sure) that could run in to issues because
the analysis is local in scope.

Comment by Kevin Downey [ 06/Apr/16 7:04 PM ]

if you make the change I mentioned in my comment above, the relevant section of the cleaned up macro expansion looks like:

(let [inst_9257 foo
      inst_9258 foo
      inst_9259 (set! inst_9258 45)
      inst_9260 println
      inst_9261 foo
      inst_9262 (inst_9260 inst_9257 inst_9261)
      inst_9263 foo
      inst_9264 (set! inst_9263 inst_9257)
      state_9266 (aset-all! state_9266 7 inst_9259 8 inst_9262)]
  (return-chan state_9266 inst_9264))

that actually has a bug too, it turns out, because the set! is changing the value of a local instead of the global `foo`

Comment by Kevin Downey [ 06/Apr/16 8:46 PM ]

someone should check to see if the clojure ioc macros do this same
thing, because you could have a similar issue, something like:

(def ^:dynamic foo 42)

(go (binding [foo 5] (let [x foo] (set! foo 20) (println x)))

I would expect that to print 5, if there is some kind of aliasing bug, it might print 20

Comment by Kevin Downey [ 06/Apr/16 8:50 PM ]

the 001 patch causes local bindings that are initialized from a global to actually create a local and initialize it from the global, instead of reading from the global.

Comment by Kevin Downey [ 06/Apr/16 10:16 PM ]

patch 0002 is patch 0001, but using the same code path in let binding inits and loops binding inits

Comment by Kevin Downey [ 07/Apr/16 12:49 PM ]

0003 adds a test for the expected local binding behavior when aliasing a global





[ASYNC-130] Improve put! docstring to mention no more than 1024 pending puts are allowed Created: 21/Jun/15  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: docstring


 Description   

The put! docstring says:

> "Asynchronously puts a val into port, calling fn0 (if supplied) when
complete. nil values are not allowed. Will throw if closed. If
on-caller? (default true) is true, and the put is immediately
accepted, will call fn0 on calling thread. Returns nil."

It would be good if it also mentioned that it can throw if more than 1024 pending puts are already on the channel. It could be something like:

> "Asynchronously puts a val into port, calling fn0 (if supplied) when
complete. nil values are not allowed. Will throw if closed or if there are more than 1024 pending puts. If
on-caller? (default true) is true, and the put is immediately
accepted, will call fn0 on calling thread. Returns nil."



 Comments   
Comment by Alex Miller [ 13/Feb/17 3:53 PM ]

Not going to document this for now as it may change in the future.





[ASYNC-131] (CLJS) go! or "go-now" for CLJS Created: 29/Jun/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Thomas Heller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I'd like to see a variant of the "go" macro that executes immediately until the first parkable operation instead of dispatching (effectively calling "setTimeout" twice). This is primarily for CLJS since CLJ can get away with blocking operations but it still might be useful there as well.

My Use-Case:

I currently use core.async to orchestrate CSS animations together and while it works perfectly fine with "go" it does require careful attention to detail on the order of operations since "go" will yield control back to the browser which may decide to reflow/repaint the screen although the animation wasn't properly started yet.

Example:

(defn show-toast [message]
  (let [toast (create-toast-dom message)]
    (dom/append js/document.body toast)
    ;; X - PROBLEM: REPAINT HAPPENS NOW
    (go (<! (anim/slide-in-from-bottom animation-time toast))
        (<! (timeout 3000))
        (<! (anim/fade-out animation-time toast))
        (dom/remove toast)
        )))

This example tries to create a "Toast" [1] DOM node which is animated in from the bottom and removed after 3 seconds. A real implementation would require handling clicks and so forth but the example is easy enough to describe the problem. The (anim/...) functions set the appropriate CSS and return a (async/timeout time) channel which lets us wait for the "completion".

The problem at X is that the go is dispatched and control is given back to the browser. Therefore the "setup" part of the animation is delayed and the toast dom node is rendered by the browser as is. Usually this means that the browser will do a repaint. This will have a "flicker" effect as the node appears and then disappears again and animates (depending on the default CSS).

The simple solution is to move the dom/append call into the go block, which is easy enough in this example but hard for more complex. Another solution is to start the animation outside the go:

(defn show-toast [message]
  (let [toast (create-toast-dom message)]
    (dom/append js/document.body toast)
    (let [slide-in (anim/slide-in-from-bottom animation-time toast)]
      ;; REPAINT, but slide-in animation already started so no "flicker"
      (go (<! slide-in)
          (<! (timeout 3000))
          (<! (anim/fade-out animation-time toast))
          (dom/remove toast)
          ))))

Again, simple enough here, hard for more complex things.

What I would prefer to see is a "go!" variant which instead of dispatching the go block just starts executing it.

(defn show-toast [message]
  (let [toast (create-toast-dom message)]
    (dom/append js/document.body toast)
    (go! (<! (anim/slide-in-from-bottom animation-time toast)) ;; REPAINT ON <!
         (<! (timeout 3000))
         (<! (anim/fade-out animation-time toast))
         (dom/remove toast)
         )))

The subtle difference is that the browser does not get a chance to do anything before we are ready and would wait anyway. After fine-tuning a few of those animation effects myself it always requires moving stuff out of or into go blocks as it stands now.

I'm aware that this is a rather specific use-case and core.async might not even be the best way to do this at all but so far I had great success with it and the code is very easy to reason about.

I have not looked too much into the internals of core.async but go! should be a pretty simple addition. If this change is accepted I would start looking into things and create a patch, just wanted to make sure there were no objections before doing so.

Hope my intent is clear. CSS animations can be a bit counter-intuitive which might not make it obvious where the problem actually lies.

[1] http://www.google.com/design/spec/components/snackbars-toasts.html#snackbars-toasts-specs



 Comments   
Comment by Thomas Heller [ 06/Jul/15 6:06 AM ]

I looked into the issue to find out what needed to be done on the core.async side to facilitate this change, turns out nothing.

(defmacro go!
  "just like go, just executes immediately until the first put/take"
  [& body]
  `(let [c# (cljs.core.async/chan 1)
         f# ~(ioc/state-machine body 1 &env ioc/async-custom-terminators)
         state# (-> (f#)
                    (ioc/aset-all! cljs.core.async.impl.ioc-helpers/USER-START-IDX c#))]
     (cljs.core.async.impl.ioc-helpers/run-state-machine state#)
     c#))

Works perfectly fine so far and can live perfectly well inside a library, I'm beginning to think that this might be a better "default" for CLJS since there are no "threads" that could start running the "go".

With the normal "go" the flow of execution is:

code before go
setTimeout/nextTick
code inside go until first take/put
setTimeout/nextTick

With go! it is:

code before go!
code inside go! until first take/put
setTimeout/nextTick

Given the non-blocking nature of Javascript the extra setTimeout does not hurt it doesn't provide much benefit either.

Since no changes to core.async are required I can simply keep this macro in my library. My use-case is rather specific so feel free to close the issue if there is no interest to make this "common".

Comment by Leon Grapenthin [ 06/Jul/15 5:46 PM ]

As a default this would violate the async execution guarantee of go which would be a breaking change. Some observations from my user perspective: For readability I'd prefer both examples you proposed for usage of go over go-now because go conveys async execution of its entire body whereas in go-now I'd have to scan the body for a parkable operation to determine the part which executes synchronously. I find it difficult to convince myself that there could really be problems so complex that go-now would be preferable over refactoring go. (I don't think you can safely rely on impl namespaces in library code )





[ASYNC-132] (CLJS) Can't close over locals in #js in go Created: 09/Jul/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

CLJS 3308



 Description   
(go
  (let [bar 42]
    #js{:foo bar}))

Use of undeclared Var my.core/bar






[ASYNC-137] (CLJS) Make (<! (timeout 0)) be closer to 0 Created: 03/Aug/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Thompson Assignee: Unassigned
Resolution: Unresolved Votes: 5
Labels: None


 Description   

Currently `(<! (timeout 0))` will take about 4ms on a modern browser because it ends up as `(js/setTimeout f 0)`. On a modern browser, setTimeout won't come back in under 4ms, even if given a 0. And 4ms is too long if you are just wanting to hand back control to the browser for the minimum about of time.

See:
https://github.com/clojure/core.async/blob/d073896192fa55fab992eb4c9ea57b86ec5cf076/src/main/clojure/cljs/core/async/impl/timers.cljs#L161-L164

followed by:
https://github.com/clojure/core.async/blob/d073896192fa55fab992eb4c9ea57b86ec5cf076/src/main/clojure/cljs/core/async/impl/dispatch.cljs#L35-L36

I was imagining this tiny rewrite:

(defn queue-delay [f delay]
(if (= 0 delay)
(goog.async.nextTick f) ;; <---- new path for 0
(js/setTimeout f delay)) ;; <---- existing path



 Comments   
Comment by Patrick O'Brien [ 03/Aug/15 10:31 PM ]

I think the use of goog.Timer.callOnce should work for any amount of time and would still solve the problem posed by the use of js/setTimeout.

Comment by Mike Thompson [ 04/Aug/15 2:38 AM ]

From what I can see, goog.Timer.callOnce use setTimeout. But that's specifically what we're trying to avoid in the case of a 0ms duration. So I don't see callOnce as a solution.

Comment by Patrick O'Brien [ 04/Aug/15 7:27 AM ]

I think you are right, Mike. Hard as it is to believe, since obviously Google knows about the problem. But now that I'm rereading the goog.Timer code I can't see that they've fixed it here. Looks like goog.async.nextTick is the only solution.

Comment by Patrick O'Brien [ 04/Aug/15 7:33 AM ]

I just wanted to mention that this issue is for the ClojureScript version of core.async, and that it appears to be fairly idiomatic to use (<! (timeout 0)) as a way to temporarily pass control back to the js event processing. So I think it is very important to make this efficient.

Comment by Ghadi Shayban [ 06/Aug/15 1:56 PM ]

In reading this "bug", it seems that the actual problem is having more varied goblock dispatch in CLJS. It could/should be improved, but making (<! (timeout 0)) – non-sensical code – behave differently is not addressing the problem in a meaningful way.

We should eliminate the need for this hack at the root, rather than make undefined behavior more specific. ASYNC-131 and ASYNC-43 are related and probably better starting points.

Comment by Mike Thompson [ 06/Aug/15 10:32 PM ]

@Ghadi

  1. I can't see how issue 131 is closely enough related to act as a "better starting point". It has a different thrust, represents different needs, and will likely have a different solution (as evidenced by the current suggestions).
  1. It almost looks to me as if Issue 43 could be closed with the switch to Google Closure dispatch noted by David Nolen. BTW, you'll note that my suggestion is to indeed use goog.async.nextTick (so I'm adhering to the use Google Closure approach).

Also, I would disagree with your assertion that (<! (timeout 0)) is "non-sensical code". In a single threaded environment, like the browser, there is sometimes a need to explicitly "hand back control" to the browser from within go loops (if that go loop is, for a time, hogging the CPU). At the moment (<! (timeout 0)) is the only way I know to do that. It could reworked to be (<! (yield-cpu)) in order to make the intent clearer, and that might be nice, but what I care about is having a way of yielding which doesn't always cost 4ms. I believe this need is genuine, and very "sensical"

Comment by Mike Thompson [ 06/Aug/15 11:06 PM ]

@Ghadi
Just to be clear ... you refer to this as a "bug" in double quotes. Almost as if it isn't a bug.

But for me this is a sufficiently real "bug" that I'll have to either work with a fork OR move away from using core.async entirely.

I need (<! (timeout 0)) to mean 0, or as close to it as possible. I need it to work as advertised. The current implementation does not deliver on 0 (or as close to it as possible), and instead delivers the same as (<! (timeout 4)) and 4ms is FOREVER compared to 0ms (or close to it). At 4ms each time around, a goloop can only iterate 250 times a second.

Anyone who has a goloop listening to the output of a bursty websocket will have this problem. They need to process like crazy (no 4ms delays) BUT also hand back control so the browser can do what it needs to, every now and again. All of this get exacerbated when a browser page loses "focus", goes into the background, and the js gets throttled, animation frames slow down, etc.

Comment by Patrick O'Brien [ 15/Aug/15 9:59 PM ]

In case it helps anyone who might be reading this, I have begun using the following as a temporary solution:

(defn yield []
  (let [ch (chan)]
    (goog.async.nextTick #(close! ch))
    ch))

(go
  ; Some code...
  (<! (yield))  ;; Instead of (<! (timeout 0))
  ...)
Comment by Mike Thompson [ 25/Aug/15 5:04 AM ]


That's nice! Certainly provides a placeholder solution.

Comment by Mike Thompson [ 03/Nov/16 12:13 AM ]

After further discussion with halgari, I believe this should be closed as "Won't Fix" (I can't figure out how to close).

https://www.reddit.com/r/Clojure/comments/5aprn3/in_coreasync_using_timeout_with_0_delay_causes_a/





[ASYNC-142] (CLJS) Rename ClojureScript namespace from cljs.core.async to clojure.core.async Created: 18/Aug/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: None


 Description   

I'm not sure what the original reasoning for using separate namespaces for the Clojure and ClojureScript versions of core.async were, but now with Reader Conditionals people are able to write cross platform code that targets both at the same time. For that reason, it would be nice to only need to require clojure.core.async so that the require didn't need to be split into two platform specific requires for the same library.

The biggest downside to this is that it would break backwards compatibility. However core.async is still in alpha, and updating code to reference the new namespace should be simple and able to be done quickly. It may be possible to leave a shim namespace in place for some time to allow people to migrate their code.

Thoughts?



 Comments   
Comment by Alex Miller [ 18/Aug/15 8:36 AM ]

I think we should do this, but it will likely be in tandem with a bigger set of changes (build, moving away from Clojure 1.6, etc).

Comment by Thomas Heller [ 03/Oct/15 5:48 AM ]

The biggest problem here are macros. The CLJS implementation (and cljs.core.async.macros) is very different from the CLJ one. Given that CLJS still requires :require-macros to use macros I doubt we can come up with a solution that does NOT require Reader Conditionals in their NS form.

CLJS can easily do away with macros requiring the extra :include-macros or :refer-macros (instead of just :refer) though. David Nolen rejected my initial implementation but was open to addressing the issue somewhere else [1]. I never went to pursue this via JIRA but it is probably something that should be solved prior to renaming things from cljs.core.async.

[1] http://dev.clojure.org/jira/browse/CLJS-948?focusedCommentId=37666&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-37666

Comment by Tim Gilbert [ 19/Aug/16 1:17 PM ]

cljs.* to clojure.* namespace aliasing was added in CLJS-1692 (ClojureScript 1.9.198), which fixes part of this issue; one can now use this form to import functions from core.async in ClojureScript:

(ns my.app 
  (:require [clojure.core.async :refer [<! >!]]))

Unfortunately, the core.async macros, notably (go), still live in a different namespace in the ClojureScript version of the library. ASYNC-119 proposes collapsing the two namespaces into one. I would think that this ticket itself could be closed now, however.

Comment by Alex Miller [ 13/Feb/17 3:51 PM ]

Is there some way this could be handled by the auto namespace aliasing of clojure->cljs now done in cljs?





[ASYNC-148] (CLJS) Shorten core.async go block variable names Created: 28/Sep/15  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: PNG File Screenshot of Google Chrome.png    

 Description   

When using core.async go blocks in ClojureScript, code is transformed into asynchronous functions. When an error occurs inside one of these go blocks, the name is printed as part of the stack trace. These names can get very long, for example one recent stack trace I saw was:

day8.mwi_enhancer.steps.load_mwi.handlers.load_previous_steps_state.cljs.core.async.impl.dispatch.run.call.day8$mwi_enhancer$steps$load_mwi$handlers$load_previous_steps_state_$_state_machine__24889__auto____1

This name is so long that it doesn't even fit on the display of a 13" Macbook in the Chrome console, and part of the name and location of the code is wrapped.

Can anything be done about making the name shorter? In this case there is a lot of duplication in the name, but there is probably a good reason why it was done this way.






[ASYNC-158] (CLJS) and with multiple <! does not short-circuit in go Created: 26/Jan/16  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Asher Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Environment:

org.clojure/clojurescript "1.7.48", org.clojure/clojure "1.7.0-RC1", org.clojure/core.async "0.2.374"



 Description   

I have an and condition inside a go block, where the two conditions take from channels.
When the first condition returns false, I would expect the second condition not to be checked at all. But, it is.

Here is the code:

(defn cc [x]
   (go 
     (println "cc: " x)
   x))

  (defn foo [x]
    (go (when (and (= :ok (<! (cc x)))
                   (= :ok (<! (cc 2))))
          (print "after and"))))

(foo 1)
(foo :ok)

When running (foo 1), I would expect only cc: 1 to be printed, but actually cc: 2 is printed as well.

Happens only in ClojureScript, not in Clojure core.async.



 Comments   
Comment by Asher [ 26/Jan/16 6:14 AM ]

Note: This only happens in ClojureScript. In Clojure it works as expected.

Also, sorry for the poor formatting.. this is my first Jira issue.

Comment by Wang Xing [ 14/Mar/16 9:39 PM ]

Seems and works abnormal in many cases, not just with <!.
I tested this in:
clojure-1.8.0, clojurescript-1.7.228, core.async-0.2.374.
this test need nodejs with the request module.

(def request (js/require "request"))

Now I try to send request to an unaccessible site: google.
Yes, I can't use google.

(def result-chan (chan))

(request
  "http://www.google.com"
  (fn [err res body]
    (go
      (if (and (not err) (= (.-statusCode res) 200))
        (>! result-chan body)
        (>! result-chan [])))))

the result should be "[]" (the request will timeout make "err" non-nil)

(go
  (println "result:" (<! result-chan)))

but actually I get:
TypeError: Cannot read property 'statusCode' of undefined ...
I have printed err and I am sure it's a JS object in this case, not nil or false. that's why res is undefined.
And it also means the short-circuit for and didn't work.
to make things right, use "if" instead of "and"

(def result-chan (chan))

(request
  "http://www.google.com"
  (fn [err res body]
    (go
      (if (not err)
        (if (= (.-statusCode res) 200)
          (>! result-chan body)
          (>! result-chan []))
        (>! result-chan [])))))

then test it again:

(go
  (println "result:" (<! result-chan)))

work as expected now:

result: []




[ASYNC-172] (CLJS) ClojureScript `catch :default` in `go` fails with non-`js/Error` classes Created: 27/Jul/16  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Alex Gunnarson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: try-catch


 Description   

In ClojureScript, `(catch :default e ...) in a `go` block fails with non-`js/Error` classes. That is, throwing e.g. a number, a String, etc. fails (not that this is good practice anyway, but still). I came upon this while porting Slingshot (scgilardi/slingshot) to CLJS.

Given the following tests:

```
(defn test0
"Prints 'Error: nil'."
[]
(go (println "Error:"
(<! (go (throw (js/Error.)))))))

(defn test1
"Prints 'Error: #object[Error Error]'."
[]
(go (println "Error:"
(<! (go (try (throw (js/Error.))
(catch :default e e)))))))

(defn test2
"Doesn't print anything in addition to the console error
msg. Chrome says 'Uncaught 123'."
[]
(go (println "Error:"
(<! (go (try (throw 123)
(catch :default e e)))))))
```

`test2` should print 'Error: 123', but doesn't.

(As a sidenote, shouldn't `test0` return the thrown js/Error instead of printing to the console that there was an uncaught error and returning nil?)



 Comments   
Comment by Alex Gunnarson [ 27/Jul/16 3:59 PM ]

Sorry, I should have specified that this is for ClojureScript version 1.9.93 and core.async version 0.2.385. Also forgive the lack of markup.





[ASYNC-175] (CLJS) excluded bounded-count in use in cljs.core.async Created: 06/Sep/16  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mike Fikes Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

With ASYNC-171, bounded-count was excluded. But, it is actually being used in cljs.core.async/to-chan here:

https://github.com/clojure/core.async/blob/43139e44b04dd09bb5faa555ea8ebd2ca09e25ce/src/main/clojure/cljs/core/async.cljs#L395

It has evidently been in use (inadvertently incorrectly) since 2013 (well prior to it having been added as a core function), with it picking up a previous private definition that had existed in cljs.core all along (but with the arguments reversed, evidently fortuitously without breaking things completely).

Here is a REPL transcript illustrating the regression that ASYNC-171 causes:

ClojureScript Node.js REPL server listening on 52771
To quit, type: :cljs/quit
cljs.user=> (require 'cljs.core.async)
WARNING: Use of undeclared Var cljs.core.async/bounded-count at line 395 file:/Users/mfikes/Projects/core.async/target/core.async-0.1.0-SNAPSHOT.jar!/cljs/core/async.cljs
WARNING: Use of undeclared Var cljs.core.async/bounded-count at line 395 .cljs_node_repl/cljs/core/async.cljs
nil
cljs.user=> (cljs.core.async/to-chan [])
TypeError: Cannot read property 'call' of undefined
    at cljs$core$async$to_chan (/Users/mfikes/Projects/clojurescript/.cljs_node_repl/cljs/core/async.js:3042:70)
    at repl:1:113
    at repl:9:3
    at repl:14:4
    at Object.exports.runInThisContext (vm.js:54:17)
    at Domain.<anonymous> ([stdin]:41:34)
    at Domain.run (domain.js:221:14)
    at Socket.<anonymous> ([stdin]:40:25)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)

You can also presumably see this if you run the ClojureScript core.async unit tests. In particular the cljs.core.async.pipeline-test namespace makes use of to-chan. (I say presumably—I haven't figured out how to run them in JVM ClojureScript, but ran the same in self-hosted ClojureScript and discovered this regression running these unit tests with the fork of core.async for self-hosted ClojureScript here: https://github.com/mfikes/andare






[ASYNC-176] (CLJS) A local named `new` breaks go macro in cljs (not in clj) Created: 12/Sep/16  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Christian Weilbach Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.8.0"]
[org.clojure/clojurescript "1.8.34"]
[org.clojure/core.async "0.2.374"]



 Description   
(<! (go
      (let [new 42]
        [new])))

yields: #object[TypeError TypeError: 42.call is not a function]

`new` is not properly handled.






[ASYNC-177] Typo in `full?` docstring of the Buffer protocol Created: 14/Sep/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Marcus Spiegel Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File Fix-typo-in-docstring-for-full.patch    
Patch: Code

 Description   

The boolean return value is mixed up in the `full?` function of the Buffer protocol (clj and cljs).



 Comments   
Comment by Alex Miller [ 13/Feb/17 3:39 PM ]

Applied for next release.





[ASYNC-182] (CLJS) Support usage from bootstrapped clojurescript Created: 14/Dec/16  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: James Laver Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: cljs


 Description   

Spinning up a JVM to build small clojurescript applications is painful on older and embedded computers. I understand there isn't official support for the bootstrapped clojure build, but it would be nice at least to be able to use core.async from cljs without requiring a jvm - it would improve our development story for less capable devices massively.

The first immediate problem is there is a (redundant?) import of `java.util.concurrent.locks.Lock` in `cljs.core.async.impl.ioc-macros` which doesn't appear to be referenced. The tests still pass if I remove it. Moving beyond there and attempting to cljc-ify the `.clj` files takes me well out of my ability to work on this, however.






[ASYNC-70] documentation of thread macro should include behavior of nil (closes the channel) Created: 15/May/14  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: reader
Environment:

0.1.278.0


Attachments: Text File async-70-patch.1.txt    
Patch: Code

 Description   

I needed the ability to invoke some code in a thread, and have the channel close if nil was returned.

Digging though the code, I discovered it already does this, but it is no documented in the docstring.

I'll supply a patch shortly.



 Comments   
Comment by Alex Miller [ 23/Jun/14 11:15 AM ]

The thread docstring says: "Returns a channel which will receive the result of the body when completed." The special case of a null return value is actually handled by ignoring it (because you are not allowed to explicitly put nils on a channel). The channel is closed on completion regardless. I'm not sure I get what needs to be added here and no patch, so closing. Reopen if there is a concrete suggestion to evaluate.

Comment by Howard Lewis Ship [ 14/Nov/14 10:41 AM ]

It's a tiny little patch. DRY is great for code, less so for docs!

Comment by Alex Miller [ 13/Feb/17 3:27 PM ]

Different smaller docstring change applied in 73afa79.





[ASYNC-155] Preserve loop binding metadata when inside a go block Created: 15/Dec/15  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 2
Labels: None

Attachments: Text File 0001-ASYNC-155-preserve-loop-binding-metadata.patch    
Patch: Code
Approval: Triaged

 Description   

Description:
Currently this causes a reflection warning because core.async loses the type hint on the binding:

user=> (set! *warn-on-reflection* true)
true
user=> (require '[clojure.core.async :refer [go chan <!]])
nil
user=> (go (loop* [^String foo "foo"](.substring foo 0) (<! (chan)) (recur nil)))
Reflection warning, /tmp/form-init4315251430191485724.clj:1:1 - call to method substring can't be resolved (target class is unknown).
#object[clojure.core.async.impl.channels.ManyToManyChannel 0xcb98635 "clojure.core.async.impl.channels.ManyToManyChannel@cb98635"]

This is a problem since macros like `doseq` expand to similar code and cause reflection warnings

The attached patch preserves that metadata and removes the reflection warning

Patch: 0001-ASYNC-155-preserve-loop-binding-metadata.patch



 Comments   
Comment by Marcus Crestani [ 04/Jan/16 3:17 AM ]

I can confirm that this patch also fixes ASYNC-157. Thanks!

Comment by Marcus Crestani [ 21/Oct/16 7:08 AM ]

Can this patch be included in a next release of core.async?

Comment by Alex Miller [ 13/Feb/17 3:08 PM ]

Can we get a better example in the ticket? This one seems broken.

Comment by Nicola Mometto [ 13/Feb/17 3:13 PM ]

Fixed the example, I omitted by accident a call to (chan) while copy pasting from the repl

Comment by Alex Miller [ 13/Feb/17 3:20 PM ]

Why are we using loop* in the example?

Comment by Nicola Mometto [ 13/Feb/17 3:23 PM ]

No particular reason, `loop` would manifest the same issue aswell. I don't remember the details but I think I got to this minimal repro case by macroexpanding a doseq form and reducing the code to this, hence why `loop*` instead of `loop`

Comment by Alex Miller [ 13/Feb/17 3:24 PM ]

gotcha, thx

Comment by Alex Miller [ 13/Feb/17 3:24 PM ]

Applied





[ASYNC-154] (require :reload-all) causes NullPointerException Created: 29/Nov/15  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Aaron Cummings Assignee: Unassigned
Resolution: Completed Votes: 1
Labels: None
Environment:

Both Oracle and IBM JDK (1.7) affected.


Approval: Triaged

 Description   

I'm seeing a problem with (require :reload-all ...) with clojure.core.async, where running with :reload-all causes a NullPointerException as shown below.

Curiously, this does not happen with 'lein repl'; it seems the :reload-all is being somehow inhibited there.

So, starting the REPL like this:

java -cp asm-all-4.2.jar:clojure-1.7.0.jar:core.async-0.2.374.jar:core.cache-0.6.4.jar:core.memoize-0.5.8.jar:data.priority-map-0.0.7.jar:tools.analyzer-0.6.7.jar:tools.analyzer.jvm-0.6.9.jar:tools.reader-1.0.0-alpha1.jar clojure.main

Results below:

Clojure 1.7.0
user=> (require ['clojure.core.async])
nil
user=> (require :reload-all ['clojure.core.async])
CompilerException java.lang.NullPointerException, compiling:(clojure/core/async.clj:1138:8) 
user=> *e
#error {
 :cause nil
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "java.lang.NullPointerException, compiling:(clojure/core/async.clj:1138:8)"
   :at [clojure.lang.Compiler analyzeSeq "Compiler.java" 6730]}
  {:type java.lang.NullPointerException
   :message nil
   :at [clojure.tools.analyzer.passes.jvm.warn_on_reflection$eval10493$fn__10494 invoke "warn_on_reflection.clj" 58]}]
 :trace
 [[clojure.tools.analyzer.passes.jvm.warn_on_reflection$eval10493$fn__10494 invoke "warn_on_reflection.clj" 58]
  [clojure.lang.MultiFn invoke "MultiFn.java" 229]
  [clojure.lang.Var invoke "Var.java" 379]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8436 invoke "passes.clj" 166]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.core$partial$fn__4529 invoke "core.clj" 2500]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk$walk__8336$walk__8337 invoke "ast.clj" 96]
  [clojure.tools.analyzer.ast$_update_children$fn__8327 invoke "ast.clj" 51]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 333]
  [clojure.core$reduce invoke "core.clj" 6518]
  [clojure.tools.analyzer.ast$_update_children invoke "ast.clj" 49]
  [clojure.tools.analyzer.ast$update_children_reduced invoke "ast.clj" 64]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk$walk__8336$walk__8337 invoke "ast.clj" 96]
  [clojure.tools.analyzer.utils$mapv_SINGLEQUOTE_ invoke "utils.clj" 208]
  [clojure.tools.analyzer.ast$_update_children$fn__8327 invoke "ast.clj" 51]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 333]
  [clojure.core$reduce invoke "core.clj" 6518]
  [clojure.tools.analyzer.ast$_update_children invoke "ast.clj" 49]
  [clojure.tools.analyzer.ast$update_children_reduced invoke "ast.clj" 64]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk$walk__8336$walk__8337 invoke "ast.clj" 96]
  [clojure.tools.analyzer.ast$_update_children$fn__8327 invoke "ast.clj" 51]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 333]
  [clojure.core$reduce invoke "core.clj" 6518]
  [clojure.tools.analyzer.ast$_update_children invoke "ast.clj" 49]
  [clojure.tools.analyzer.ast$update_children_reduced invoke "ast.clj" 64]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk invoke "ast.clj" 95]
  [clojure.tools.analyzer.ast$walk invoke "ast.clj" 92]
  [clojure.tools.analyzer.ast$prewalk invoke "ast.clj" 108]
  [clojure.tools.analyzer.passes$compile_passes$analyze__8443 invoke "passes.clj" 170]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.tools.analyzer.jvm$analyze$fn__11679 invoke "jvm.clj" 469]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invoke "core.clj" 630]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1868]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.tools.analyzer.jvm$analyze invoke "jvm.clj" 456]
  [clojure.core.async.impl.ioc_macros$state_machine invoke "ioc_macros.clj" 1109]
  [clojure.core.async$go doInvoke "async.clj" 413]
  [clojure.lang.RestFn invoke "RestFn.java" 442]
  [clojure.lang.Var invoke "Var.java" 388]
  [clojure.lang.AFn applyToHelper "AFn.java" 160]
  [clojure.lang.Var applyTo "Var.java" 700]
  [clojure.lang.Compiler macroexpand1 "Compiler.java" 6631]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6709]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyze "Compiler.java" 6485]
  [clojure.lang.Compiler$BodyExpr$Parser parse "Compiler.java" 5861]
  [clojure.lang.Compiler$LetExpr$Parser parse "Compiler.java" 6179]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6723]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6711]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyze "Compiler.java" 6485]
  [clojure.lang.Compiler$BodyExpr$Parser parse "Compiler.java" 5861]
  [clojure.lang.Compiler$FnMethod parse "Compiler.java" 5296]
  [clojure.lang.Compiler$FnExpr parse "Compiler.java" 3925]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6721]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6711]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler access$300 "Compiler.java" 38]
  [clojure.lang.Compiler$DefExpr$Parser parse "Compiler.java" 577]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6723]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyze "Compiler.java" 6485]
  [clojure.lang.Compiler eval "Compiler.java" 6786]
  [clojure.lang.Compiler load "Compiler.java" 7227]
  [clojure.lang.RT loadResourceScript "RT.java" 371]
  [clojure.lang.RT loadResourceScript "RT.java" 362]
  [clojure.lang.RT load "RT.java" 446]
  [clojure.lang.RT load "RT.java" 412]
  [clojure.core$load$fn__5448 invoke "core.clj" 5866]
  [clojure.core$load doInvoke "core.clj" 5865]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [clojure.core$load_one invoke "core.clj" 5671]
  [clojure.core$load_all$fn__5389$fn__5392 invoke "core.clj" 5688]
  [clojure.core$load_all$fn__5389 invoke "core.clj" 5687]
  [clojure.lang.AFn call "AFn.java" 18]
  [clojure.lang.LockingTransaction run "LockingTransaction.java" 273]
  [clojure.lang.LockingTransaction runInTransaction "LockingTransaction.java" 229]
  [clojure.core$load_all invoke "core.clj" 5685]
  [clojure.core$load_lib$fn__5397 invoke "core.clj" 5711]
  [clojure.core$load_lib doInvoke "core.clj" 5710]
  [clojure.lang.RestFn applyTo "RestFn.java" 142]
  [clojure.core$apply invoke "core.clj" 632]
  [clojure.core$load_libs doInvoke "core.clj" 5749]
  [clojure.lang.RestFn applyTo "RestFn.java" 137]
  [clojure.core$apply invoke "core.clj" 632]
  [clojure.core$require doInvoke "core.clj" 5832]
  [clojure.lang.RestFn invoke "RestFn.java" 421]
  [user$eval7383 invoke "NO_SOURCE_FILE" 2]
  [clojure.lang.Compiler eval "Compiler.java" 6782]
  [clojure.lang.Compiler eval "Compiler.java" 6745]
  [clojure.core$eval invoke "core.clj" 3081]
  [clojure.main$repl$read_eval_print__7099$fn__7102 invoke "main.clj" 240]
  [clojure.main$repl$read_eval_print__7099 invoke "main.clj" 240]
  [clojure.main$repl$fn__7108 invoke "main.clj" 258]
  [clojure.main$repl doInvoke "main.clj" 258]
  [clojure.lang.RestFn invoke "RestFn.java" 421]
  [clojure.main$repl_opt invoke "main.clj" 324]
  [clojure.main$main doInvoke "main.clj" 422]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [clojure.lang.Var invoke "Var.java" 375]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.Var applyTo "Var.java" 700]
  [clojure.main main "main.java" 37]]}
user=>


 Comments   
Comment by Alex Miller [ 22/Dec/15 4:09 PM ]

dupe of and fixed by ASYNC-152

Comment by Nicola Mometto [ 22/Dec/15 6:00 PM ]

Just for the record, this is neither a core.async nor a tools.analyzer bug.
It turns out that re-evaluating a defmulti expression completely nukes any metadata on that defmulti, breaking tools.analyzer's scheduler since it relies on that metadata to order the passes.

Comment by Nicola Mometto [ 22/Dec/15 6:08 PM ]

Alex Miller I'm reopening this, pending an assessment on CLJ-1870.
While it is true that the ASYNC-154 patch fixed this specific NPE, `(require :reload-all)` will still break things since as explained in the ticked description of CLJ-1870, t.a passess will be ordered randomly

Comment by Alex Miller [ 13/Feb/17 3:19 PM ]

Both ASYNC-152 and CLJ-1870 have now been applied and I don't think we're going to do anything additional with this, so closing.





[ASYNC-168] Bad interaction between go, try and fn body Created: 11/May/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Moritz Heidkamp Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: try-catch
Environment:

[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.374"]



 Description   

The following code incorrectly uses <! in a fn body within go. This should usually lead to a corresponding AssertionError but if there is a catch clause which catches Exception, this error is silently swallowed and the execution goes into an infinite loop.

(require '[clojure.core.async :as async])

(async/go
  (try ((fn []
          (println "Hello")
          (async/<! (async/timeout 1000))))
       (catch Exception e)))

Output:

Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
...

This only happens when there are (catch ...) clauses for any exception types which aren't in the inheritance hierarchy of AssertionError, e.g. it works fine with (catch Error ...) but also fails with (catch RuntimeException ...).



 Comments   
Comment by Kevin Downey [ 11/May/16 11:14 AM ]

It seems like using <! outside of a go block would be much better handled as a compilation error, instead of a runtime error. if the defns for the operators that are handled specially by go blocks were removed, then we would get compile time errors for incorrect usage. I am not sure what the motivation was for adding them, maybe for doc strings? Seems like a case were trying to make it easier to find docs made the library harder to use.

Comment by Moritz Heidkamp [ 11/May/16 4:23 PM ]

That would most likely fix this issue, indeed. The docstring situation is a bit unfortunate, though. This would also affect alts! and alt!, by the way.

However, it might be worth looking into some more beyond that: The buggy behavior isn't triggered when replacing (async/<! (async/timeout 1000)) with something like (assert false), for example, so something seems to be going wrong in the state machine transformation there when <! et. al. are present.

Comment by Kevin Downey [ 17/May/16 7:38 PM ]

actually, I am pretty sure this is due to general brokeness in the go macros exception handling, see http://dev.clojure.org/jira/browse/ASYNC-169 (there is a patch that could use testing if you are inclined)

Comment by Alex Miller [ 13/Feb/17 3:02 PM ]

With ASYNC-169 applied, I see:

Hello
Exception in thread "async-dispatch-1" java.lang.AssertionError: Assert failed: <! used not in (go ...) block

which is what I'd expect, so marking as dupe of ASYNC-169.





[ASYNC-174] Executor service cannot be shutdown Created: 13/Aug/16  Updated: 13/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stephen Brady Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

JVM Clojure



 Description   

Previous to this change (https://github.com/clojure/core.async/commit/a690c4f3b7bf9ae9e7bdc899c030955d5933042d), the executor service used by core async was accessible via a var. While the var was not part of the public api, it is now completely inaccessible as a resource that can be managed. Most importantly, it cannot be shut down without shutting down the JVM. This creates all kinds of operational difficulties and side effects for managing the JVM, especially with class loaders.

It would be helpful to at least lift out the executor back to a var - the way it was before. Ideally, core async would expose the executor as part of the API, where the ordinary ExecutorService interface would apply, or at least have a shutdown function.






[ASYNC-78] deadlock in multi catch Created: 05/Jul/14  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Lars Bohl Assignee: Unassigned
Resolution: Duplicate Votes: 1
Labels: try-catch
Environment:

ubuntu 14.04
Leiningen 2.4.2 on Java 1.7.0_60 Java HotSpot(TM) 64-Bit Server VM
[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]



 Description   

The following apparently never returns:

(defn thread-death []
  (<!! (go
        (<!
         (let [ch (chan 1)]
           (>! ch 0)
           (>! ch (try (<! ch)
                       (assert false)
                       (catch Exception _ 5)
                       (catch AssertionError _ 4)))
           ch)))))

After removing the (catch Exception _ 5) catch block it returns 4 as expected.



 Comments   
Comment by Kevin Downey [ 17/May/16 7:52 PM ]

this is the same faulty exception handling also reported in http://dev.clojure.org/jira/browse/ASYNC-169

Comment by Alex Miller [ 13/Feb/17 2:48 PM ]

Marking as dupe of ASYNC-169





[ASYNC-178] First pending put into a channel always succeeds even if the buffer is full Created: 16/Sep/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alexander Yakushev Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.9.0-alpha10"]
[org.clojure/core.async "0.2.385"]



 Description   

Say we have a channel with a fixed buffer and a generating transducer (e.g. cat), and a pending put is scheduled into that channel. Then, the take from the channel will invoke the first pending put, even if the buffer is still full after the take.

Example scenario:

(def c (a/chan 10 cat))

(go-loop [i 0]
  (let [batch (range i (+ i 3))]
    (when (>! c batch)
      (println "Put succeeded:" batch)
      (recur (+ i 3)))))

;; Put succeeded: (0 1 2)
;; Put succeeded: (3 4 5)
;; Put succeeded: (6 7 8)
;; Put succeeded: (9 10 11)

;; // Predictable so far. Buffer filled up to 12 items and the next put blocked.

(<!! c) ; => 0
;; Put succeeded: (12 13 14)

(<!! c) ; => 1
;; Put succeeded: (15 16 17)

(<!! c) ; => 2
;; Put succeeded: (18 19 20)

;; // What? The buffer keeps growing after each single take!

Because of this, I cannot use backpressure with generative transducers, since the size of the fixed buffer, in this case, may grow infinitely.

I believe, the issue is here: https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L182. If the buffer was checked for fullness before the first pending put is executed, the issue would be solved. If this is confirmed to be a bug (not the intended behavior) I will submit a patch.



 Comments   
Comment by Alex Miller [ 13/Feb/17 2:41 PM ]

This is the expected behavior. Expanding transducers must always be able to put and are allowed to do so even when the buffer is full. If this is an issue for back-pressure for you, you should pull the transducer out of the channel and do the expansion in a go process.





[ASYNC-122] Parking in finally block replaces result Created: 06/May/15  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Vesa Karvonen Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: try-catch
Environment:

[org.clojure/clojurescript "0.0-3211"]
[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]


Approval: Triaged

 Description   

Because

(try 1 (finally 2))

evaluates to 1, I would expect

(go (println (try 1 (finally (<! (go 2))))))

to print 1, but it prints 2 in both CLJ and CLJS. This can be worked around by replacing try-finally with try-catch.



 Comments   
Comment by Nicola Mometto [ 17/Dec/15 10:06 AM ]

This appears to be a bug caused by nesting `go` blocks

Comment by Kevin Downey [ 17/May/16 7:54 PM ]

the patch on http://dev.clojure.org/jira/browse/ASYNC-169 appears to fix this

Comment by Alex Miller [ 13/Feb/17 2:39 PM ]

Marking as duplicate of ASYNC-169.





[ASYNC-179] parking take in finally clause in go block causes try expression to yield the finally clause result Created: 20/Sep/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Valentin Waeselynck Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: try-catch
Environment:

core.async 0.2.391, Clojure 1.8.0,

java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)

OS X Yosemite 10.10.5, MacBook Air (11-inch, Early 2015)



 Description   

Inside a go block, in a try-catch-finally clause,
if the finally-clause consists of a parking take (<!),
the try-catch-finally will yield the result of the finally, instead of the try body.

e.g

(a/<!! (go
         (try :a
           (finally :b))))
=> :a ;; expected

;; whereas:
(a/<!! (go
         (try :a
           (finally (<! (go :b))))))
=> :b ;; wrong

Repro project on Github.



 Comments   
Comment by Valentin Waeselynck [ 20/Sep/16 10:48 AM ]

Duplicate of http://dev.clojure.org/jira/browse/ASYNC-122.

Comment by Alex Miller [ 13/Feb/17 2:38 PM ]

Marking as duplicate of ASYNC-169.





[ASYNC-180] (go (try ... (finally ...))) block with parking operations in finally body loses try body return value Created: 07/Oct/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nick Alexander Assignee: Unassigned
Resolution: Duplicate Votes: 1
Labels: None
Environment:

[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.385"]



 Description   

As it says. A parking operation in the finally body loses the try body return value. See the following test case:

(ns finally-testcase
  (:require
   [clojure.core.async :as a :refer :all ;[go go-loop <! >! go timeout]
    ]))

(def t1
  (go
    (try
      1
      (finally
        2))))

(def t2
  (go
    (try
      (<! (timeout 100))
      1
      (finally
        2))))

(def t3
  (go
    (try
      1
      (finally
        (<! (timeout 100))
        2))))

(println (<!! t1)) ;; => 1, as expected
(println (<!! t2)) ;; => 1, as expected
(println (<!! t3)) ;; => nil -- not expected!


 Comments   
Comment by Nick Alexander [ 07/Oct/16 3:39 PM ]
Unable to find source-code formatter for language: clojure. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
(ns finally-testcase
  (:require
   [clojure.core.async :as a :refer :all ;[go go-loop <! >! go timeout]
    ]))

(def t1
  (go
    (try
      1
      (finally
        2))))

(def t2
  (go
    (try
      (<! (timeout 100))
      1
      (finally
        2))))

(def t3
  (go
    (try
      1
      (finally
        (<! (timeout 100))
        2))))

(println (<!! t1)) ;; => 1, as expected
(println (<!! t2)) ;; => 1, as expected
(println (<!! t3)) ;; => nil -- not expected!
Comment by Nick Alexander [ 07/Oct/16 3:39 PM ]

Somebody with edit rights can format the original comment correctly.

Comment by Kevin Downey [ 10/Oct/16 7:27 PM ]

the patch on http://dev.clojure.org/jira/browse/ASYNC-169 fixes this

Comment by Alex Miller [ 13/Feb/17 2:37 PM ]

Marking as duplicate of ASYNC-169.





[ASYNC-173] Infinite loop in go block when non-Exception thrown Created: 08/Aug/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Matt Keller Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: try-catch
Environment:

core.async 0.2.385; clojure 1.9.0-alpha6



 Description   

A Throwable - other than Exception - thrown from a go block can cause an infinite loop.

(defn endless-fun [f]
  (let [in-ch (a/to-chan (range 1 10))]
    (->>
     (a/go
       (try
         (loop []
           (when-let [value (a/<! in-ch)]
             (f value)
             (recur)))
         (catch Exception e
           (.printStackTrace e)
           (throw e))))
     (a/<!!))))


(endless-fun (fn [i] (println i))) => 1, 2, ... 9

(endless-fun (fn [i] (println i) (throw (RuntimeException. (str i))))) => 1 

(endless-fun (fn [i] (println i) (throw (Throwable. (str i))))) => 1, 1, 1, ... infinite loop!

If the try/catch is absent, you don't get an infinite loop - it looks like the RuntimeException case above.

May also be related to http://dev.clojure.org/jira/browse/ASYNC-100 or http://dev.clojure.org/jira/browse/ASYNC-169.



 Comments   
Comment by Alex Miller [ 13/Feb/17 2:31 PM ]

Marking as a duplicate of ASYNC-169, which seems to fix this issue.





[ASYNC-100] core.async with multiple catch blocks causing weird loop behaviour Created: 27/Oct/14  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tom Coupland Assignee: Unassigned
Resolution: Duplicate Votes: 7
Labels: try-catch

Attachments: Text File ASYNC-100_2.patch    
Patch: Code and Test
Approval: Triaged

 Description   

I've been seeing this weird looping behavior with some go loops over the last few days. An exception is being thrown from a function within the loop and rather than logging and looping back around to a waiting take, the loop seems to just hop back to the line before the function call.

I've managed to boil it down to the following:

(def s (chan))
(def single
  (go-loop []
    (try
      (prn "Awaiting single")
      (<! s)
      (prn "Single")
      (throw (new Throwable))
      (catch Throwable t
        (prn t)))
    (recur)))

(def d (chan))
(def double
  (go-loop []
    (try
      (prn "Awaiting double")
      (<! d)
      (prn "Double")
      (throw (new Throwable))
      (catch Exception re
        (prn re))
      (catch Throwable t
        (prn t)))
    (recur)))

Now if you (>!! s :a), you'll see the throwable printed out and the loop go back to waiting on the s channel. However, (>!! d :a) and you'll get to enjoy an infinite stream of 'Double'. In actual fact you can remove the -loop from double and get the same result.

Not sure what's going on here at all. In the macro expanded version of double '(prn t)' doesn't appear at all (it does in single's expansion), so it looks like it's not surviving the move into the state machine and instead is routing back to (prn "Double") or the take isn't really completing somehow, leaving the :a on the chan.



 Comments   
Comment by Paavo Parkkinen [ 03/Dec/14 5:30 PM ]

From my experiments, it seems what's causing the issue isn't the two catch clauses, but the fact that you are catching an Exception first, and then the Throwable. When I switched the two catch clauses (Throwable first) the issue went away.

You don't even need the two catch clauses to trigger it, a single catch clause of an Exception will do.

;; Works
(require '[clojure.core.async :as async])
(def c (async/chan))
(def st
  (async/go-loop []
    (try
      (prn "Awaiting")
      (async/<! c)
      (prn "Received")
      (throw (new Throwable))
      (catch Throwable t
             (prn t)))
    (recur)))
(async/>!! c :a)
;; Doesn't work
(require '[clojure.core.async :as async])
(def c (async/chan))
(def se
  (async/go-loop []
    (try
      (prn "Awaiting")
      (async/<! c)
      (prn "Received")
      (throw (new Throwable))
      (catch Exception t
             (prn t)))
    (recur)))
(async/>!! c :a)
Comment by Paavo Parkkinen [ 04/Dec/14 12:18 AM ]

Attaching a patch that fixes the original issue, but not the one in my comment above with an uncaught exception. All test cases pass.

Instead of just picking the first catch block, and creating a state machine block and exception frame for that one, I create blocks and frames for all of them.

I tried creating a test case too, but was unable to create one that would reproduce the error. I'll spend some more time trying to write a test case, but I wanted to submit the code patch without the test coverage first.

Comment by Paavo Parkkinen [ 04/Dec/14 7:45 PM ]

Attached patch with test case included.

Comment by Stuart Sierra [ 18/Jun/15 2:52 PM ]

Possible workaround until this is fixed: Just have one catch clause, catching all Throwable. Then examine the type of the throwable object inside the catch block to decide what to do.

Comment by Ken Allen [ 29/Oct/15 1:34 PM ]

This is even more troublesome when the exception is thrown from code that's not part of what the user wrote in the go block. You don't even need a loop for it to loop forever:

(require '[clojure.core.async :as async])
(def c (async/chan))
(doseq [_ (range 1024)] (async/put! c :test))
(async/go (try (println "BEFORE") (async/>! c :test) (catch Exception ex (println "ERR"))))

This was confusing as hell when we ran into it but catching Throwable instead does indeed work around the problem so that's what we've been doing. Was hoping the latest release of core.async would fix this.

Comment by Thomas Getgood [ 26/Jan/16 10:46 AM ]

As it is, you need to catch Throwable or nothing. Unhandled Throwables will cause this as well.

(require '[clojure.core.async :refer [go <! put! chan]])
(def ch (chan))
(go (try (print (<! ch)) (assert false) (catch Exception e nil)))
(put! ch 1)

Will print an infinite stream of 1s.

It was a lot of fun figuring out why functions failing :pre conditions caused infinite loops.

Comment by Kevin Downey [ 17/May/16 8:08 PM ]

the patch here no longer applies. http://dev.clojure.org/jira/browse/ASYNC-169 is a dupe of this, but it has a patch that applies

Comment by Alex Miller [ 13/Feb/17 2:29 PM ]

Marking as duplicate of ASYNC-169 which seems to address this issue as well.





[ASYNC-71] exception behavior for thread macro is hard coded Created: 15/May/14  Updated: 05/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: try-catch
Environment:

0.1.278.0-76b25b-alpha



 Description   

Currently, the thread macro's behavior when an exception occurs is to print the exception (with println), and swallow it.

Although I make a habit of wrapping a try around code inside a thread form, I still find this a bit limited; it would be nice if there was a function that could be dynamically bound, that handled the case of an exception inside a thread.



 Comments   
Comment by Alex Miller [ 06/Aug/14 10:29 AM ]

Patches on ASYNC-76 have been applied and exceptions will now flow up to the top of the thread, where they can be caught by the standard Thread uncaught exception handler mechanism or ultimately by the default uncaught exception handler, which can be set for the application. There may still be further changes to support exception handling in thread/go.





[ASYNC-61] Exceptions thrown inside go/thread blocks propagate up and out of ThreadPoolExcecutor Created: 24/Mar/14  Updated: 05/Feb/17

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: try-catch
Environment:

core.async 0.1.267.0-0d7780-alpha on JDK 1.7



 Description   

If a go or thread blocks throws an exception, there is nothing in Clojure to catch and handle (or report) the exception. Instead, it propagates up to the ThreadExcecutor which invokes its NO-OP afterExecute() method and is re-thrown, ultimately being displayed on the System.err:

Exception in thread "async-dispatch-32" java.lang.IllegalStateException: Fall down, go boom!
at flashiz.resources.orders$index.invoke(orders.clj:26)
at clojure.lang.Var.invoke(Var.java:411)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.core$apply.invoke(core.clj:617)
at io.aviso.rook$rook_dispatcher.invoke(rook.clj:225)
at flashiz.async$wrap_sync_handler$fn__9005.invoke(async.clj:34)
at flashiz.resources$authorize_async_rook_middleware$fn_9356$fn9402$state_machine3245auto__9403$fn_9405.invoke(resources.clj:21)
at flashiz.resources$authorize_async_rook_middleware$fn_9356$fn9402$state_machine3245auto___9403.invoke(resources.clj:21)
at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:945)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:949)
at clojure.core.async.impl.ioc_macros$take_BANG_$fn__3261.invoke(ioc_macros.clj:958)
at clojure.core.async.impl.channels.ManyToManyChannel$fn__2256.invoke(channels.clj:80)
at clojure.lang.AFn.run(AFn.java:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

It would be nice if execution of the go/thread block was wrapped in an exception handler that delegated to a default function to report the exception. My goal is to be able to alter that function to report it more nicely and/or write it to a persistent log file.



 Comments   
Comment by Alex Miller [ 06/Aug/14 10:30 AM ]

Patches on ASYNC-76 have been applied and exceptions will now flow up to the top of the thread, where they can be caught by the standard Thread uncaught exception handler mechanism or ultimately by the default uncaught exception handler, which can be set for the application. There may still be further changes to support exception handling in thread/go.

Comment by Christian Weilbach [ 29/Dec/16 8:57 AM ]

I have hit this issue repeatedly as well, so I have implemented a comprehensive error-handling strategy as a library for `core.async`: https://github.com/replikativ/superv.async

It has certain limitations, e.g. there is no dynamic binding across async boundaries in cljs so the supervisor needs to be carried lexically, and I would prefer a default error-handling concept along the Erlang philosophy to have composable libraries built on `core.async`. Having said that, it works quite well and should provide you with the necessary means to install a logging supervisor.





[ASYNC-126] Add public channel "closed?" predicate Created: 08/Jun/15  Updated: 26/Jan/17

Status: In Progress
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Unresolved Votes: 5
Labels: None

Attachments: Text File 0001-ASYNC-126-Add-public-channel-closed-predicate.patch     Text File 0002-ASYNC-126-Add-public-channel-closed-predicate.patch    
Patch: Code
Approval: Triaged

 Description   

Current patch file: 0002-ASYNC-126-Add-public-channel-closed-predicate.patch

Add a public function closed? in clojure.core.async to determine if a channel is closed without putting or taking any values.

This is a trivial wrapper around clojure.core.async.impl.protocols/closed?

There is still a race between closed? and close!. Correct code should never rely on closed? to check that a channel is open. The closed? predicate should only be used to avoid extra work producing a value for a closed channel.

Example use case

A producer process can check if a channel is closed? before doing the work to produce the next value.

(thread
  (loop []
    (when-not (closed? ch)
      (>!! ch (do-expensive-work))
      (recur))))


 Comments   
Comment by Alejandro [ 18/Jul/15 11:41 AM ]

The patch only adds 'closed?' to the ClojureScript version of core.async, is there a reason for not adding it to the Clojure version?

Comment by Stuart Sierra [ 19/Jul/15 12:07 PM ]

Previous patch was missing the Clojure(JVM) implementation. Fixed in new patch 0002-ASYNC-126-Add-public-channel-closed-predicate.patch.

Comment by Brandon Bloom [ 12/Feb/16 1:30 PM ]

Please do not add a closed? predicate to the public API. It's a race condition in 99.9% of cases. Go does not provide such a predicate and after growing accustom to CSP via both core.async and Go, I have never felt the need for it. Having the predicate would invite its use. I enjoy clojure.core's preference for omission of questionable functions and expect the same from core.async.

If somebody desperately feels the need for this predicate, ask in IRC or Slack, I'm sure somebody can help you fix the structuring of your channels/processes/etc.

Comment by Ghadi Shayban [ 12/Feb/16 2:16 PM ]

Echoing Brandon, I'm also unconvinced. This is a bug waiting to happen.





[ASYNC-46] Add optional default channel to 'pub' Created: 09/Dec/13  Updated: 07/Dec/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Steffen Dienst Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None

Attachments: Text File 0001-Add-default-channel-for-pub.patch    
Patch: Code and Test

 Description   

If there is no subscriber for a topic the message gets silently dropped. I suggest to add an optional default channel that receives all those messages. The default channel can then be used in scenarios, where the complete set of topics might be unknown beforehand.
This functionality resembles the 'dead letter' pattern in messaging systems.



 Comments   
Comment by Terje Norderhaug [ 07/Dec/16 6:28 PM ]

I second supporting a default channel for pub. Use cases include:

1. Report missing/unsubscribed topics, aiding in development and debugging.
2. Catch messages entering before topics have been subscribed, providing a means to expose messages otherwise dropped due to late/delayed subscriptions.
3. Chain publications with different topic-fns.





[ASYNC-181] Expose unblocking-buffer? on channels Created: 11/Oct/16  Updated: 15/Oct/16  Resolved: 13/Oct/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Nick Alexander Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

For a generic listener implementation, where consumers are not controlled, I want to ensure that put! doesn't block. `unblocking-buffer?` does exactly this... but it's not exposed on channels. The work-around is to access the underlying buffer, which is an implementation detail. Can we expose this (unblocking?) on channels directly?



 Comments   
Comment by Richard Newman [ 11/Oct/16 3:23 PM ]

Specifically:

(defn unblocking-channel? [chan]
  (a/unblocking-buffer? (#?(:cljs .-buf :clj .buf) chan))
Comment by Alex Miller [ 13/Oct/16 9:34 AM ]

You can use offer! for this...

Comment by Nick Alexander [ 15/Oct/16 3:28 PM ]

I have a mult for which I want to ensure the taps do not block the distribution. I do not see how to use offer! to ensure that taps that can accept, do accept; and that slow taps do not block the mult. Can you elaborate?





[ASYNC-110] 0.1.346, using >! crashes with no method 'setTimeout', when targeting node Created: 23/Dec/14  Updated: 21/Sep/16  Resolved: 12/Feb/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: William Assignee: Unassigned
Resolution: Completed Votes: 3
Labels: bug, nodejs
Environment:

[[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]
[org.clojure/clojurescript "0.0-2511"]]


Attachments: Text File async-110-exportSymbol.patch     Text File br.log    

 Description   

When targeting node, running a simple >! crashes immediately:

(let [a (chan)]
  (go (>! a :hi))
  (go (println (<! a))))

Crashes with:

goog.global.setTimeout(cb, 0);
   ^
TypeError: Object #<Object> has no method 'setTimeout'

This works fine with 0.1.338.0-5c5012-alpha.



 Comments   
Comment by Joel Wilsson [ 01/Feb/15 3:34 PM ]

The problem is really in the ClojureScript compiler. It does not export those NodeJS global functions. I have a fix that works for me and seems sound, so I submitted a pull request: https://github.com/clojure/clojurescript/pull/45

Comment by David Nolen [ 01/Feb/15 3:47 PM ]

I'm not convinced yet that there are any issues with the ClojureScript compiler nor that your PR represents the solution we should pursue.

Before we even begin assessing any solution first someone needs to identify precisely why the goog.async.nextTick detection doesn't pick up the Node.js globals. The first thing that Google Closure library does is map the global environment to goog.global. I'm curious as to why this isn't working in this case.

Comment by Joel Wilsson [ 01/Feb/15 3:47 PM ]

Patch from the pull request attached.

Comment by Joel Wilsson [ 03/Feb/15 3:49 PM ]

Yes, the first thing the Google Closure library is set up a reference to the global environment with goog.global = this;

However, this is not the global environment in Node.js. From http://nodejs.org/api/vm.html#vm_sandboxes:

the this expression within the global scope of the context evaluates to the empty object ({}) instead of to your sandbox.

If you start a Node.js REPL and enter console.log(this) you will see that this is in fact the global environment. However, if you put that statement in a file and run it with nodejs <filename>, the output will be "{}".

Some possible solutions:

  1. Use goog.exportSymbol to get these functions into goog.global, as in the patch I attached. A drawback of this approach is that it is using goog to fix goog, so it must be done after goog has been loaded.
  2. Explicitly set properties on this before doing anything else. This is the simplest solution. Preamble:
    this.setTimeout = setTimeout;
    this.clearTimeout = clearTimeout;
    this.setInterval = setInterval;
    this.clearInterval = clearInterval;
  3. Use vm.Script.runInThisContext, as bootstrap_node.js does to load base.js with this referencing the global environment found in the variable global.
    A bit tricky in this case, since the file we're executing needs to read itself, but it can be done with this preamble:
    if (!this.__nodeFilename) {
        var vm = require('vm');
        var fs = require('fs');
        global.__nodeFilename = __filename;
        vm.runInThisContext.call(global, fs.readFileSync(__nodeFilename), __nodeFilename);
        process.exit(0);
    }

    For this to work the hashbang must be removed, since #!/usr/bin/env node is not valid JavaScript. Since node is a symlink to ax25-node on Ubuntu, and the Node.js binary is called "nodejs", this might be a good idea either way.
    While this solution is the closest to bootstrap_node.js, it is complex and may be a bit too clever.

  4. Convince the Closure library maintainers to check if base.js is being loaded in Node.js and do goog.global = global; if that is the case.

With 3 and 4, goog.global will contain a lot of Node.js stuff. That may or may not be a good thing. Considering the effort in bootstrap node to hide module and exports, I'm guessing it's a bad thing.

From what I can tell, programs using the Closure library, and which have been compiled by the Closure compiler into a single file, have never worked properly if they have been using anything that needs setTimeout and friends.

Comment by David Nolen [ 03/Feb/15 3:53 PM ]

Joel, many thanks for all the extra information. Will sort through this and think about it.

Comment by David Nolen [ 10/Feb/15 12:00 PM ]

Issue is confirmed on this end. Will look into.

Comment by David Nolen [ 10/Feb/15 12:41 PM ]

So I looked into this a bit, this does not actually appear to be a real issue, at least not yet. The Node.js environment must be bootstrapped for ClojureScript to work. However this process could be simpler and we will address the usability bits in ClojureScript. When it's resolved there and we can confirm via automated testing in core.async we will close this ticket.

Comment by Stuart Mitchell [ 11/Feb/15 8:16 PM ]

Not quite sure what you mean by 'not a real problem' as I have a clojurescript project that targets atom-shell that worked with 0.1.338.0-5c5012-alpha but now doesn't.

Do you mean that its actually a clojurescript issue, if so is there a bug report/ work around?

Comment by Joel Wilsson [ 12/Feb/15 7:58 AM ]

Stuart, could you please try the four lines from option 2 in my previous comment as a preamble and let me know if that solves the problem for you? Ie. put them in a file preamble.js on your project's classpath (I use resources/) and add :preamble "preamble.js" to your :compiler settings. Assuming that you are using lein-cljsbuild, that is.

This is a real issue for me as well, because it stops me from using core.async on AWS Lambda.

Comment by David Nolen [ 12/Feb/15 8:10 AM ]

Sorry Stuart I'm not saying it's not real a issue but rather that I'm trying to determine the scope and precise nature of the issue. I noticed that the problem does not manifest when I use :optimizations :none. Are you and Joel using :simple, :advanced? Thanks.

Comment by Joel Wilsson [ 12/Feb/15 8:23 AM ]

I use :simple to get a single JavaScript file, which I can then use with AWS Lambda with this in main.js:

var r = require("./clojurescript-output.js");
exports.handler = r.hello_lambda.core.handler;

where clojurescript-output.js is generated from a file like

(ns hello_lambda.core)
(enable-console-print!)
(defn ^:export handler [event context]
  (println "Hello AWS Lambda")
  (.done context))

:advanced doesn't work for me: it just creates a JavaScript file with a single (function () {...})(), which makes it useless for creating a module (the symbols are not exported properly), but that's another issue.

Comment by David Nolen [ 12/Feb/15 9:11 AM ]

this was in fact an upstream ClojureScript issue, fixed as of this commit https://github.com/clojure/clojurescript/commit/e8ae2c8a9c3f2429555e136cc17c1669be5e993a

Comment by Dusan Maliarik [ 12/Jul/16 3:41 PM ]

Not entirely fixed. If you use cljs.core.async/pub in the namespace,

eg. http://stackoverflow.com/questions/24698536/how-to-memoize-a-function-that-uses-core-async-and-non-blocking-channel-read/38069830#38069830
(then using it in some namespace)

(ns foo
  (:require ....))

(def foo
  (memoize-async
    (fn [...] ...)))

then goog.global.setTimeout is called before the cljs.nodejscli is evaled. Therefore causing

/home/skrat/Workspace/xxx/yyy.drie/bin/convert:37684
    goog.global.setTimeout(a, 0);
                ^

TypeError: goog.global.setTimeout is not a function
    at Function.setImmediate_ (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:37684:17)
    at Object.goog.async.nextTick (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:37630:270)
    at Object.cljs.core.async.impl.dispatch.queue_dispatcher (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:37718:21)
    at Object.cljs.core.async.impl.dispatch.run (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:37722:40)
    at Function.cljs.core.async.pub.cljs$core$IFn$_invoke$arity$3 (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:40775:33)
    at Function.cljs.core.async.pub.cljs$core$IFn$_invoke$arity$2 (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:40703:30)
    at Object.yyy.drie.io.async.memoize_async (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:42231:89)
    at Object.<anonymous> (/home/skrat/Workspace/xxx/yyy.drie/bin/convert:59163:41)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)

I tried to require cljs.nodejscli in the ns that contains my main fn, but it results in

SEVERE: cljs.nodejscli:1: ERROR - namespace "cljs.nodejscli" cannot be provided twice
goog.provide('cljs.nodejscli');
^

Jul 12, 2016 2:34:34 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_DUPLICATE_NAMESPACE_ERROR. namespace "cljs.nodejscli" cannot be provided twice at cljs.nodejscli line 1 : 0
Comment by Dusan Maliarik [ 12/Jul/16 3:44 PM ]

I believe the solution is to include cljs.nodejscli as soon as possible, instead of appending it at the end. This also means it has to be split into two pieces, one dealing with goog.global (included asap), and another one dealing with main-cli-fn (included at the end as is the case now).

Comment by Nick Alexander [ 21/Sep/16 5:34 PM ]

I am seeing this, with `optimizations :advanced` and `:simple`, with the following project https://github.com/ncalexan/datomish-user-agent-service/commit/af99ad3a16eece881a48fedd0b0f62517bb5cb25. It's not going to be 100% easy to duplicate, since an underlying library isn't published on Maven Central yet, but I can reproduce this consistently.

Any help?

Comment by Nick Alexander [ 21/Sep/16 6:10 PM ]

For the next poor sod who runs into this, it is triggered by having a `go` at top-level.

Hat tip to user "Roc King" in the post http://clojure.2344290.n4.nabble.com/Will-advanced-optimizations-be-officially-supported-on-nodejs-tp7762p7855.html, who writes [sic]:

After got more familiar with clojurescript, I realized that ASYNC-110 fully described(and gave several working solutions!)[3] this issue.
But if "goog.global" was used before "goog.global = global"(the solution used in clojurescript[4]), the same problem will occur.
This is the exactly situation I have encountered several days ago: I wrote '(go (<! (timeout 1212)) ...)' in top level.

This was not clear to me until I read Roc's note. After I remove my top-level `go`, everything is happy.





[ASYNC-170] binding in go block causes thread binding error: "Pop without matching push" Created: 26/May/16  Updated: 20/Sep/16

Status: In Progress
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Christian Weilbach Assignee: Kevin Downey
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Clojure 1.8.0, core.async 0.2.374


Attachments: Text File 0001-ASYNC-170-update-binding-frame-on-loop-exit.patch    
Patch: Code and Test
Approval: Triaged

 Description   

Reproduced in https://github.com/whilo/async-binding.

(def ^:dynamic *foo* nil)

(defn -main [& args]
  (go
    (binding [*foo* nil]
      (<! (go 42))
      (println "done.")))

  (Thread/sleep 30000))

$ lein uberjar
$ java -jar ...

done.
Exception in thread "async-dispatch-3" java.lang.IllegalStateException: Pop without matching push
	at clojure.lang.Var.popThreadBindings(Var.java:331)
	at clojure.core$pop_thread_bindings.invokeStatic(core.clj:1839)
	at clojure.core$pop_thread_bindings.invoke(core.clj:1839)
	at async_binding.core$_main$fn__6354$state_machine__4495__auto____6355$fn__6357.invoke(core.clj:8)
	at async_binding.core$_main$fn__6354$state_machine__4495__auto____6355.invoke(core.clj:8)
	at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:1011)
	at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1010)
	at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:1015)
	at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1013)
	at clojure.core.async.impl.ioc_macros$take_BANG_$fn__4511.invoke(ioc_macros.clj:1024)
	at clojure.core.async.impl.channels.ManyToManyChannel$fn__313$fn__314.invoke(channels.clj:95)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)


 Comments   
Comment by Kevin Downey [ 27/May/16 1:46 AM ]

this issue only reproduces when the code is aot compiled

Comment by Christian Weilbach [ 27/May/16 6:55 AM ]

Right, I missed to add the AOT condition here, only in the repository. I cannot update the issues myself, right? I tried also to close another issue I opened, but couldn't find a way to close it.





[ASYNC-171] bounded-count appears in both clojure.core and clojure.core.async Created: 05/Jul/16  Updated: 06/Sep/16  Resolved: 05/Jul/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Chelimsky Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None
Environment:

clojure-1.9.0-alpha8 and core.async-0.2.385



 Description   

WARNING: bounded-count already refers to: #'clojure.core/bounded-count in namespace: clojure.core.async, being replaced by: #'clojure.core.async/bounded-count



 Comments   
Comment by Alex Miller [ 05/Jul/16 8:33 AM ]

Fixed in master for next release

Comment by Mike Fikes [ 06/Sep/16 10:18 PM ]

See ASYNC-175 for an associated regression.





[ASYNC-157] Reflection warning with doseq Created: 04/Jan/16  Updated: 17/Jun/16  Resolved: 17/Jun/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Marcus Crestani Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Environment:

OS X 10.11.2
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
[org.clojure/clojure "1.7.0"]
[org.clojure/core.async "0.2.374"] and also core.async GIT HEAD, latest commit 6e8b1aa



 Description   

A doseq within a go block that contains some async calls causes a reflection warning.
Here is a simple example:

(ns dummy
  (:require [clojure.core.async :as async]))

(defn dummy
  [ch cs]
  (async/go
    (doseq [c cs]
      (async/>! ch c))))
Reflection warning, /Users/crestani/activegroup/factorylink/dummy.clj:6:3 - call to method nth can't be resolved (target class is unknown).

How can we avoid these warnings?



 Comments   
Comment by Nicola Mometto [ 04/Jan/16 2:53 AM ]

Dupe of ASYNC-155

Comment by Alex Miller [ 17/Jun/16 11:04 AM ]

closing as dupe





[ASYNC-161] `<!` inside nested `let` inside `go` block erroneously yields "<! not used inside go block" error Created: 21/Feb/16  Updated: 17/Jun/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jonathan Leonard Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mac OS X.
Node 4.1.1.
Clojure 1.7.0
ClojureScript 1.7.170



 Description   

Title says it all.



 Comments   
Comment by Jonathan Leonard [ 21/Feb/16 3:08 PM ]

Actually, my code is a bit more complicated:

(defn func []
   (go
     (let [...]
       (for [[a b c d] e]
         (let [...]
            (when pred
               (let [val (<! ...)]  ...)))))))
Comment by Nicola Mometto [ 21/Feb/16 3:20 PM ]

You cannot user `for` inside a go block

Comment by Jonathan Leonard [ 21/Feb/16 3:39 PM ]

The `for` cannot appear anywhere inside a `go`? Or the `for` cannot contain inside it any async calls; i.e., is it ok for `for` to appear somewhere in the let binding as a sibling to a call to <!?

Comment by Nicola Mometto [ 21/Feb/16 3:48 PM ]

You can't use `<!` inside a `for` inside a `go` block, code like this OTOH should be valud:

(go 
  (let [.. (for [..] ..)] 
    (<! ..)))
Comment by Jonathan Leonard [ 21/Feb/16 4:45 PM ]

Ahh, perfect! Thx!





[ASYNC-163] Off-by-two in pipeline-async parallelism Created: 17/Mar/16  Updated: 17/Jun/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Moritz Heidkamp Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.374"]



 Description   

The actual parallelism of pipeline-async is greater by 2 than what was passed. Here's the simplest reproduction case I could come up with:

(let [message-count 20
      parallelism 5
      parallel-count (atom 0)
      input (async/to-chan (range message-count))
      output (async/chan message-count)]
  (async/pipeline-async parallelism
                        output
                        (fn [_ result]
                          (async/go
                            (swap! parallel-count inc)
                            (async/<! (async/timeout 1000))
                            (async/>! result @parallel-count)
                            (swap! parallel-count dec)
                            (async/close! result)))
                        input)
  (async/<!! (async/into [] output)))

Results in something like this:

[7 4 7 5 3 7 2 7 7 7 5 3 1 3 6 4 3 1 2 6]


 Comments   
Comment by Kevin Downey [ 17/Mar/16 2:07 PM ]

I suspect this is just a lack of clarity around what parallelism means
in the context of pipeline-async.

A parallelism of 'n' for pipeline-async means there are 'n' logical
threads spinning off async tasks, and because those tasks are async,
those logical threads spin off the task, and proceed to the next. The
other thing that 'n' effects is the size of the buffer for the
`results` channel internally in the pipeline. because the tasks are
asynchronous, the only throttle is effectively the size of the buffer
on the output channel.

If the only throttle is the buffer, which is of size 'n', why do we
seem to have n+2 logical threads running at once, well because there
are a few extra channels involved which combined with the logic for
copying from channel to channel, effectively adds another buffer of
size 2.

This behavior does seem arcane. But I would argue that the
behavior of pipeline-async is so different from pipeline and
pipeline-blocking that it really should be something distinct from
them. The other pipelines take transducers, and using the parallelism
'n' to spin up a number of logical threads for the other pipelines
actually does throttle the number of tasks run in parallel.

If you have not spent time evaluating which `pipeline` variant you
needed to use, and just grabbed `pipeline-async` because that is the
one that seemed intuitively the right choice (because it has `-async`
and this is `core.async`) you almost certainly should be using the
vanilla `pipeline` variant, its behavior is much more intuitive it is
much more likely to be what you want.

Comment by Moritz Heidkamp [ 21/Mar/16 5:30 PM ]

Hi Kevin, thanks for your reply! After reading it, I think your point
regarding clarity around what parallelism means in the context of
pipeline-async is spot on It seems to be quite a different
beast from the other pipelines, indeed.

We actually did evaluate the various options and from the
documentation it seemed to be the right tool in our case: We want to
launch a maximum number of parallel asynchronous HTTP requests. We
immediatley ruled out pipeline-blocking since we are not doing
blocking requests after all and don't want to hog up n threads
just for waiting for the requests to finish. We also figured that this
isn't really "computational parallelism" (as the documentation of
pipeline puts it), so pipeline-async seemed like the best
fit. Its API also looked well suited for this case: We just needed to
launch the HTTP request in af and then fill and close the
conveniently supplied result channel in the the request's completion
callback. This seemed to work fine – until we actually hit the limit
of the connection pool, that is.

I'm not sure how we would use pipeline to the same effect. And
moreover, it's now rather unclear to me what pipeline-async's
intended use case is. Can you perhaps enlighten me some more? That
would be much appreciated

Comment by Kevin Downey [ 21/Mar/16 7:05 PM ]

I dunno.

That does sort of sound like a thing for which you would use pipeline-async, but my experience with nio predates core.async, so I am not sure what the best way to stick the two together is. I am sort of skeptical about combining even "async" io libraries like http clients with core.async, since the async of the io system is different from the async of core.async, and when combining the two it is easy to screw up and use some blocking convenience function because async is hard.

depending on the nature of the http request and how you consume the results it would, of course, be way simpler to use a synchronous http library and pipeline-blocking. if you are bounding the number of http requests at a time to a certain number, I suspect you will run out of io capacity before the jvm has trouble with that number of threads.

if pipeline-async is working for you otherwise, besides the +2 thing, then, uh, maybe just subtract 2





[ASYNC-167] Fix docstring for put! and friends Created: 27/Apr/16  Updated: 28/Apr/16  Resolved: 28/Apr/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alan Thompson Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: docstring, documentation


 Description   

The docstring for `put!` and friends is misleading and incorrect. They repeatedly mention "port" where the user knows only the term "channel".

Also, description of `fn0` is missing & incorrect. The phrase "Will throw if closed" is incorrect. fn0 is called with a single arg, which is `true` upon success and `false` if the channel is closed.



 Comments   
Comment by Alex Miller [ 28/Apr/16 5:23 PM ]

docstring was already updated





[ASYNC-166] Clarify clojure.core.async/chan docstring Created: 20/Apr/16  Updated: 20/Apr/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Karlis Lauva Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring

Attachments: Text File chan-docstring.patch    
Patch: Code

 Description   

Update the `chan` docstring, so that it specifies that buffer size must be positive when transducers are supplied.






[ASYNC-162] Published API docs seem way out of date Created: 27/Feb/16  Updated: 31/Mar/16  Resolved: 26/Mar/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Avi Flax Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: documentation


 Description   

The home page says that the current release is 0.2.374. When I follow the link API Docs that page says:

API for clojure.core.async - Facilities for async programming and communication 0.1.0 (in development)

Which doesn't give me much confidence that the docs are accurate for the current version of the library.



 Comments   
Comment by Alex Miller [ 24/Mar/16 10:47 AM ]

I think that 0.1.0 must be hard-coded into the doc generation somewhere - I will track that down. The docs are current though for released version.

Comment by Alex Miller [ 26/Mar/16 8:12 AM ]

Updated docs to report the major/minor version based on the code.

The docs themselves were and are up to date with the latest release.

Comment by Avi Flax [ 31/Mar/16 10:08 AM ]

That's excellent! Thanks!





[ASYNC-160] `(vector :blah)` and `[:blah]` crashing with `inst_XXXX.call not a function` within core.async go routine Created: 20/Feb/16  Updated: 24/Mar/16  Resolved: 24/Mar/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jonathan Leonard Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Environment:

Node.js v4.1.1
Mac OS X 10.11.2
JVM 1.8.0_45



 Description   

Summary says it all.

[Worked around it by using a list rather than a vector (which compiled fine)].

I don't have time to create a minimal repro at the moment but if no one can repro this and it's necessary, I can devote time to that later.



 Comments   
Comment by Nicola Mometto [ 21/Feb/16 3:22 PM ]

This ticket doesn't have enough info to be useful:

user=> (go (vector :foo))
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x3e59dd7e "clojure.core.async.impl.channels.ManyToManyChannel@3e59dd7e"]
user=> (go [:foo])
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x70b36700 "clojure.core.async.impl.channels.ManyToManyChannel@70b36700"]
Comment by Jonathan Leonard [ 21/Feb/16 3:33 PM ]

The actual code was more like:

(defn func []
  (go
    (let [...]
      (swap! an-atom update-in [:foo] ...))))

I would imagine that the let bindings themselves aren't the culprit but if you need yet more detail I will try to obfuscate them enough to provide them here.

Comment by Jonathan Leonard [ 21/Feb/16 3:35 PM ]

Also, this was from ClojureScript; not Clojure.

Comment by Nicola Mometto [ 21/Feb/16 3:50 PM ]

(swap! x [:foo] ..)
will invoke
(apply [:foo] @x ..)
which is probably not what you want and the cause of the exception you're seeing. Not a core.async bug

Comment by Jonathan Leonard [ 21/Feb/16 4:46 PM ]

Sorry, it was:

(swap! an-atom update-in [:foo] ...
Comment by Jonathan Leonard [ 21/Feb/16 4:48 PM ]

Also, note that changing [:foo] to (list :foo) or '(:foo) side stepped the erroneous compiler output and allowed to code to succeed without any additional changes anywhere to the code.

Comment by Nicola Mometto [ 21/Feb/16 5:20 PM ]

I'm sorry but this is still not enough information.
You're going to have to post an actual failing case if you want somebody to be able to help you

Comment by Jonathan Leonard [ 21/Feb/16 5:24 PM ]

This isn't a matter of helping me: I already have a workaround (namely use `(list ...)` instead of `(vector ...)`).

I reported the bug for everyone's benefit.

Comment by Nicola Mometto [ 21/Feb/16 5:51 PM ]

Let me rephrase

Opening a ticket with only assumptions about the cause of an issue you're seeing and no details about the actual code that's causing it, is not helpful to anybody, all we can do is play guesswork.

If you want a ticket to be helpful (either to you, to the users of the library or to its maintainers in fixing the reported issue), you're at least going to have to post enough information for maintainers/contributors to reproduce the supposed bug.

In this particular case, even without a reproducible case I can speculate that it's much more likely that the problem lies in your own code than in core.async: nothing in the `go` macro would make a call to `list` compile any different than a call to `vector`

Comment by Jonathan Leonard [ 21/Feb/16 9:13 PM ]

It sounds like you are making at least as many and as great in magnitude of assumptions as me (in fact more since I know the code in question and its resultant compiler output).

I will distill the code and post it so that all doubts may be removed.

Comment by Jonathan Leonard [ 21/Feb/16 10:38 PM ]

Here it is in all its glory:

(def a (atom {:a {:b '() :c '()}}))
(defn- repro-async-160 []
  (go
    (let [_ (<! (to-chan [1 2 3 4]))
          new (for [order (range 100)] nil)]
      (swap! a update-in [:a] assoc :b '() :c '()))))

Note that changing `new` to `z` avoids the issue. I suspect that identifier shadowing in let binding is not playing nicely with all the other macros in play (i.e., for & go).

Is `new` some kind of [undocumented] reserved word?

Comment by Jonathan Leonard [ 21/Feb/16 10:42 PM ]

Actually it may be colliding with the JavaScript notion of 'new' (in which case this is a ClojureScript bug and not a core.async-- but they sent me here first actually).

Comment by Nicola Mometto [ 22/Feb/16 3:24 AM ]

This sounds like a dupe of ASYNC-63

Comment by Alex Miller [ 24/Mar/16 10:58 AM ]

Dupe of ASYNC-63

Comment by Jonathan Leonard [ 24/Mar/16 12:10 PM ]

I'm not convinced that this is a duplicate. The repro steps and behavior are slightly different. It may certainly be merely a different manifestation of the same underlying or related issue and any potential fix may resolve both issues but I don't think (as an outside observer) that that is necessarily so.





[ASYNC-164] Limited partition-all in pipeline Created: 19/Mar/16  Updated: 24/Mar/16  Resolved: 24/Mar/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Corin Lawson Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

a6dc01d2ebb692a39e923988b0853ff2bd1a431b



 Description   

Using partition-all as the transducer in a pipeline has unexpected results.

(let [to (chan)]
  (pipeline 1 to (partition-all 2) (to-chan (range)))
  (repeatedly 4 #(<!! to)))

Expected Output: ([0 1] [2 3] [4 5] [6 7])
Actual Output: ([0] [1] [2] [3])

It seems to me that the res channel should be created inside the dotimes form and closed when there are no more jobs. (This breaks the contract of the async function af). I have created a simplified version of pipeline (below) which exhibits the desired behaviour. (Note this is a modified version of the function that I submitted to ASYNC-150, i.e. it breaks the ordering of the output.) I attempted to modify the current definition of pipeline* but the p channels are confusing me (I don't understand why they are necessary) and failed in my effort to write a working patch.

(defn pipeline
  ([n to xf from] (pipeline n to xf from true))
  ([n to xf from close?] (pipeline n to xf from close? nil))
  ([n to xf from close? ex-handler]
   (assert (pos? n))
   (let [ex-handler (or ex-handler (fn [ex]
                                     (-> (Thread/currentThread)
                                         .getUncaughtExceptionHandler
                                         (.uncaughtException (Thread/currentThread) ex))
                                     nil))]
     (go
       (<! (async/merge
             (repeatedly n
                         #(let [res (chan 1 xf ex-handler)]
                            (go-loop []
                                     (if-let [v (<! from)]
                                       (do
                                         (>!! res v)
                                         (recur))
                                       (close! res)))
                            (go-loop []
                                     (let [v (<! res)]
                                       (when (and (not (nil? v)) (>! to v))
                                         (recur))))))))
       (when close? (close! to))))))


 Comments   
Comment by Kevin Downey [ 20/Mar/16 1:06 PM ]

this is not possible.

pipeline effectively runs a transducer in parallel, partition-all is based on sequential ordering. your rewritten pipeline will also fail to result in the partition you want.

if you look at the docstrings of transducer returning functions in core, some of them say "returns a transducer" some say "returns a stateful transducer", most stateful transducers will behave differently when run in parallel, because each concurrent instance will have its own state. partition-all is a stateful transducer.

Comment by Corin Lawson [ 20/Mar/16 4:02 PM ]

It is quite foolhardy to say my rewritten pipeline will fail to result in the partition I want, did you try it? I am very much aware that partition-all is a stateful tranducer; this fact is the crux of the problem, and the reason I wish to moved the res channel to the dotimes form, so that each of the n channels will maintain its own state. As I previously mentioned, order is unimportant to me but state is, the test that I actually need to pass is the following, in which my function succeeds and it also meets my other requirements. I share it here with you to show you the particular aspect of the behaviour that I expected from core.async/pipeline, nothing more.

(let [to (chan)]
  (pipeline 3 to (partition-all 2) (to-chan (range)))
  (repeatedly 4 #(-> to <!! count (= 2))))

Expected Output: (true true true true)

Comment by Kevin Downey [ 20/Mar/16 10:58 PM ]

perhaps I misunderstood, but in your example you said your expected output is

([0 1] [2 3] [4 5] [6 7])

which I read as an ordered processing of the input

Comment by Corin Lawson [ 20/Mar/16 11:02 PM ]

Perhaps, but the point I'm trying to make is about the size of the vectors in the output.

Comment by Kevin Downey [ 20/Mar/16 11:14 PM ]

the p channels are there to preserve the order of the output.

they are created and paired with jobs in the same order as jobs come in from the from channel, and put into the results channel in the same order, and then the final go loop copies from each channel in turn, preserving the order of results regardless of what order they generated in by the workers.

using the same channel(and transducer state) for the life time of the worker makes that ordering impossible. pipeline walks a line allowing a certain of random ordering of processing internally, but the order of the output is the order of the input.

there may be interest in something like this (I know I often don't really care about order) but it would be a large departure from the existing behavior of pipeline.

Comment by Corin Lawson [ 20/Mar/16 11:27 PM ]

Okay, that makes sense to me now, thank you. I agree, unordered output is a departure from the existing behaviour. I also accept that it is unreasonable to enforce an order over the output, any such ordering would be at the loss of parallelism.

(Please close, am I able to close this ticket myself?)

Comment by Kevin Downey [ 21/Mar/16 12:08 AM ]

I honestly don't know. I am a random guy on the internet, so you may want to leave it open and see if someone more closely tied to the project has thoughts

Comment by Corin Lawson [ 21/Mar/16 12:13 AM ]

Ha! Thank you fellow random guy on the internet!

Comment by Alex Miller [ 24/Mar/16 10:55 AM ]

Note that the pipeline docstring says "Because it is parallel, the transducer will be applied independently to each element, not across elements, and may produce zero or more outputs per input." which explains the results.

I would not expect a transducer with a stateful reducing function to work with the parallelism in partition-all (other than with the caveat specified above).





[ASYNC-133] Expose 'closed?' predicate Created: 18/Jul/15  Updated: 12/Feb/16  Resolved: 18/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alejandro Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Description   

It can be useful to, given a channel, use a predicate to know wheter is closed or not. The 'closed?' predicate is currently not public and remains an implementation detail.



 Comments   
Comment by Alex Miller [ 18/Jul/15 11:36 AM ]

Dupe of CLJ-126

Comment by Brandon Bloom [ 12/Feb/16 1:27 PM ]

I think you mean dupe of http://dev.clojure.org/jira/browse/ASYNC-126





[ASYNC-152] Reflection warning when binding received values to names. Created: 20/Nov/15  Updated: 22/Dec/15  Resolved: 22/Dec/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Marcus Crestani Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None
Environment:

OS X 10.11.1
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
[org.clojure/clojure "1.7.0"]
[org.clojure/core.async "0.2.374"]


Attachments: Text File 0001-ASYNC-152-disable-t.a.jvm-s-warn-on-reflection-pass.patch    
Approval: Vetted

 Description   

After upgrading core.async in our project to 0.2.374 (we came
from 0.1.346.0-17112a-alpha), we now see many reflection warnings
that we did not see with the old version.

Here is a simple example code to reproduce the typical reason for
the warning:

(ns dummy
  (:require [clojure.core.async :as async]))

(defn dummy-1
  [ch]
  (async/go-loop []
    (let [msg (async/<! ch)]
      (if (= :close msg)
        (async/close! ch)
        (recur)))))

We see:

Reflection warning: dummy.clj:8:11 - call to static method equiv on clojure.lang.Util cannot be resolved

The warning is about (= :close msg).

Note that there is no reflection warning when we are not binding
the received value to a name, i.e. (= :close (async/<! ch))
works.

Why do we see these reflection warnings after upgrading?
How can we fix these warning?

Patch:
0001-ASYNC-152-disable-t.a.jvm-s-warn-on-reflection-pass.patch



 Comments   
Comment by Alex Miller [ 20/Nov/15 8:24 AM ]

I have seen some of these as well and I suspect it's most likely due to the more recent tools.analyzer versions. I think there are some useful notes here too, thanks.

I do not currently have any suggestion on how to stop these from happening.

Comment by Nicola Mometto [ 15/Dec/15 5:55 PM ]

Minimal case:

(fn [ch] (go (let* [msg (<! ch)] (= :foo msg)))
Comment by Nicola Mometto [ 15/Dec/15 6:02 PM ]

tools.analyzer.jvm's method matcher seems to be bugged, the reflection warning comes from t.a rather than from clojure, clojure doesn't actually require reflection for this.

This patch simply disables tools.analyzer's warn-on-reflection pass, since the clojure compiler will warn itself if something will reflect

Comment by Alex Miller [ 22/Dec/15 3:43 PM ]

applied





[ASYNC-151] Accept optional transducer in `to-chan` Created: 18/Nov/15  Updated: 18/Nov/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Karlis Lauva Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File to-chan-xducer.patch    
Patch: Code and Test

 Description   

Allow `clojure.core.async/to-chan` accept an optional transducer that gets passed to the newly created channel.
Why? With transducer passed directly in `to-chan` there is no need to use `pipe` or `onto-chan` with an explicitly created a channel that has transducer attached.

This is my first patch for a Clojure project. Please let me know if something was wrong / missing when creating the ticket.






[ASYNC-99] go block with <! inside protocol method invocation fails to compile (dispatch :protocol-invoke on '-item-to-ssa') Created: 23/Oct/14  Updated: 11/Nov/15  Resolved: 11/Nov/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Valentin Waeselynck Assignee: Unassigned
Resolution: Not Reproducible Votes: 4
Labels: go-macro, protocols
Environment:

[org.clojure/core.async "0.1.346.0-17112a-alpha"]
happened on both [org.clojure/clojure "1.7.0-alpha1"] and [org.clojure/clojure "1.6.0"]

java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)



 Description   

I was programming a function involving a core.async go block, when I stumbled on a strange compilation error :

CompilerException java.lang.IllegalArgumentException: No method in multimethod '-item-to-ssa' for dispatch value: :protocol-invoke, compiling:(NO_SOURCE_PATH:2:3)

I experimented a little to try and strip down the problem, and found it was very generic. Say I have any protocol MyProtocol :

(defprotocol MyProtocol
  (do-something [this param] "some method"))

The following code will not compile, failing with the exception I showed you above :

(defn uncompilable! [me ch] 
  (go 
    (do-something me (<! ch)) ;; apparently, it hurts to use <! in a protocol method invocation 
    ))

However, the following 2 will compile without any problem :

(defn compilable! [me ch] 
  (let [call-it #(do-something me %)] ; wrapping the protocol call in a function
    (go 
     (call-it (<! ch))
     )))

(defn compilable-2! [me ch] 
  (go 
    (let [my-value (<! ch)] ; taking out the <! call
      (do-something me my-value))
    ))

It seems to me the '<! inside protocol method invocation form' is a situation which the go macro fails to handle.



 Comments   
Comment by Alex Miller [ 11/Nov/15 5:13 PM ]

NR in latest core.async 0.2.374.





[ASYNC-149] Cannot compile a recur in a case statement when in a go-loop with a <! Created: 29/Oct/15  Updated: 11/Nov/15  Resolved: 11/Nov/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Josh Comer Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

When compiling the following code causes a Caused by: java.lang.UnsupportedOperationException: Can only recur from tail position exception to be thrown (from the 1 line of the case):

(defn foo
  [x]
  (let [c (a/chan)]
    (a/go-loop [o 1]
      (when-let [r (a/<! c)]
        (case o
          1 (recur 2)
          2 (recur 3)
          3 false)))))

Interestingly this code does not create the exception:

(defn foo-2
  [x]
  (let [c (a/chan)]
    (a/go-loop [o 1]
      (when-let [r true]
        (case o
          1 (recur 2)
          2 (recur 3)
          3 false)))))

The error causing code compiles fine using 0.1.346.0-17112a-alpha. My first guess would be this has to do with the updating of tools.analyzer.jvm



 Comments   
Comment by Alex Miller [ 11/Nov/15 4:19 PM ]

Fixed by upgrade to latest tools.analyzer.jvm in https://github.com/clojure/core.async/commit/f01c280b68f0aa013e6aa08e08f25354f78ae343





[ASYNC-140] transducer not applied with unbuffered channels Created: 17/Aug/15  Updated: 04/Nov/15  Resolved: 18/Aug/15

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Declined Votes: 0
Labels: None

Approval: Triaged

 Description   

Found this while testing some stuff on ASYNC-124.

(require '[clojure.core.async :as a :refer (>!! <!! chan onto-chan)])
(def c (chan 1 (mapcat #(range %))))
(onto-chan c [1 2 3])
(<!! (a/reduce conj [] c))
;; [0 0 1 0 1 2]

(def c (chan 0 (mapcat #(range %))))
(onto-chan c [1 2 3])
(<!! (a/reduce conj [] c))
;; [1 2 3]

Cause: For an unbuffered channel, a take will try to pair up with a put, in which case the two are satisfied directly without ever involving the buffer (since there isn't one). Because of this, add! is never called and the xf will not be applied. The example above uses an expanding transducer, but problem exists for any transducer and could perhaps happen even on buffered channels?

Approach: one approach would be to not satisfy the put unless there were enough pending takes to satisfy all the transduced values, which seems consistent with an unbuffered channel.



 Comments   
Comment by Alex Miller [ 17/Aug/15 5:13 PM ]

I suspect the problem is here:
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L115-L133

where a take can't get a value from the buffer (there is none) and instead satisfies itself by matching up directly with a put. In this case, add! is never called so the transducer is not applied.

In the case of an expanding transducer, a buffer is needed at this point where none exists.

Comment by Alex Miller [ 18/Aug/15 8:17 AM ]

I'm a dope and unbuffered channels (or 0-sized fixed buffer) channels do not support transducers per the chan docstring.

Comment by Alex Nixon [ 04/Nov/15 5:10 PM ]

I vote to reopen this.

As a user this feels like a very subtle bug - the docstring says "if a transducer is supplied, a buffer must be specified". In your example you have specified a buffer – one of size zero – and so it's very surprising when you find out (at some later stage in your program) that your transducer was silently ignored. If a fixed buffer of size zero is not a valid thing to construct then I would expect (buffer 0) to throw an IllegalArgumentException or similar.

And philosophically I don't understand why the combination of an unbuffered channel and a transducer should be forbidden. What's the argument here?

Comment by Alex Miller [ 04/Nov/15 9:17 PM ]

After talking to Rich, it is considered an error to created a fixed buffer of size 0 and that should throw. The ticket ASYNC-143 covers that check.

On the last question, a transducer needs to be (potentially) be able to produce multiple values with something like a mapcat transducer. Without a buffer, there is no place for those buffers to go, so an unbuffered channel is insufficient.





[ASYNC-150] Unordered pipeline Created: 01/Nov/15  Updated: 01/Nov/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Corin Lawson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I have a use case for clojure.core.async/pipeline whereby the order of the outputs is unimportant and it is desirable to have the nth output present on the to channel without having to wait for the first output to be ready.

I don't have a great deal of experience with clojure; I'm wondering if this is something that belongs in the core library or whether there's some idiomatic way of achieving an unordered pipeline without rewriting the pipeline function. In any case, I have made an attempt at rewriting pipeline, and learnt a thing or two in the process

Obviously, this code (below) cannot be merged as it completely breaks the existing behaviour (not to mention I've barely tested it), but I would be appreciative if you could provide some feedback and/or consider adding this functionality. Perhaps a new function, unordered-pipeline?

(defn- pipeline*
  ([n to xf from close? ex-handler type]
     (assert (pos? n))
     (let [ex-handler (or ex-handler (fn [ex]
                                       (-> (Thread/currentThread)
                                           .getUncaughtExceptionHandler
                                           (.uncaughtException (Thread/currentThread) ex))
                                       nil))
           jobs (chan n)
           results (chan n)
           process (fn [[v p :as job]]
                     (if (nil? job)
                       (comment closing results here would be too early)
                       (let [res (chan 1 xf ex-handler)]
                         (>!! res v)
                         (close! res)
                         (put! p res)
                         true)))
           async (fn [[v p :as job]]
                   (if (nil? job)
                     (comment closing results here would be too early)
                     (let [res (chan 1)]
                       (xf v res)
                       (put! p res)
                       true)))]
       (go-loop []
                (let [v (<! from)]
                  (if (nil? v)
                    (close! jobs)
                    (do
                      (comment removed the indirection and pass results as part of the job)
                      (>! jobs [v results])
                      (recur)))))
       (go-loop []
                (let [res (<! results)]
                  (if (nil? res)
                    (when close? (close! to))
                    (do (loop []
                          (let [v (<! res)]
                            (when (and (not (nil? v)) (>! to v))
                              (recur))))
                        (recur)))))
       (go
         (comment ensure results is closed after all jobs have been processed)
         (<! (async/merge
               (map #(case %
                       :blocking (thread
                                   (let [job (<!! jobs)]
                                     (when (process job)
                                       (recur))))
                       :compute (go-loop []
                                         (let [job (<! jobs)]
                                           (when (process job)
                                             (recur))))
                       :async (go-loop []
                                       (let [job (<! jobs)]
                                         (when (async job)
                                           (recur)))))
                    (repeat n type))))
         (close! results)))))


 Comments   
Comment by Corin Lawson [ 01/Nov/15 1:51 AM ]

I've only tried the :compute scenario...

(snooze [] (Thread/sleep (+ 5 (rand-int 10))))
(defn slowly [f] (fn
  ([] (snooze) (let [out (f)] (snooze) out)) 
  ([r] (snooze) (let [out (f r)] (snooze) out))
  ([r i] (snooze) (let [out (f r i)] (snooze) out))))
(def cout (chan))
(pipeline 3 cout (comp slowly (map inc)) (async/to-chan (range 18)))
(<!! (async/into [] cout))
-> [1 3 2 4 5 6 8 9 7 10 12 11 13 14 15 16 17 18]

Each group of 3 (n) is outputted in a random order





[ASYNC-124] put! into channel that has mapcat transducer fails to dispatch more than one waiting takes Created: 30/May/15  Updated: 28/Oct/15  Resolved: 28/Oct/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Jozef Wagner Assignee: Alex Miller
Resolution: Completed Votes: 1
Labels: None
Environment:

Clojure 1.7.0-RC1, latest core.async head (May 2015)


Attachments: Text File async-124-2.patch     Text File async-124-3.patch     Text File async-124-4.patch     Text File async-124-5.patch     Text File async-124-6.patch     Text File async-124-7.patch     Text File async-124-8.patch     Text File async-124.patch    
Patch: Code and Test
Approval: Ok

 Description   

This issue is similar to one that has caused the revert of ASYNC-103. When channel has multiple pending takes, one put will dispatch only one take, even if transducer attached to the channel produces multiple values.

(require '[clojure.core.async :refer (chan put! take!)])
(let [xf (mapcat #(range % (+ % 5)))
        c (chan 10 xf)]
    (dotimes [x 5]
      (take! c #(println (str x %))))
    (put! c 10)
    (put! c 15))

The output of the above code is

0 10
1 11

but it should be

0 10
1 11
2 12
3 13
4 14

Note that if we change the order and execute puts before takes, the result will be as expected.

Cause: When executing a write on a channel, no more than one pending take will be satisfied. Other pending takes will wait until the next put. When a put could release at most one take, this was sufficient but with expanding transducers, a put can potentially satisfy many pending takers.

Approach: Once a put succeeds into the buffer (through the transducer), instead of finding a single taker, do the prior work in a loop, pairing up callbacks with values removed from the buffer. Once the loop is complete, release the mutex and dispatch all paired takes.

Patch: async-124-8.patch

Screened by:



 Comments   
Comment by Jozef Wagner [ 30/May/15 10:30 AM ]

patch with test

Comment by Jozef Wagner [ 30/May/15 10:32 AM ]

Note that with attached patch the actual dispatch/run on takes happens before (.unlock mutex) and before (when done? (abort this)) are called.

Also note that added test only works with 1.7.0 version of clojure

Comment by Jozef Wagner [ 30/May/15 12:46 PM ]

Added new patch async-124-2.patch that dispatches after unlocking and aborting. Test was also changed as it had race condition.

Comment by Alex Miller [ 06/Jul/15 9:23 AM ]

We can't add a test that is 1.7-only. See https://github.com/clojure/core.async/blob/master/src/test/clojure/clojure/core/pipeline_test.clj#L7 for an example of how to patch in a transducer impl that will work in pre-1.7.

Comment by Jozef Wagner [ 06/Jul/15 10:41 AM ]

Added async-124-3.patch that works with Clojure <1.7

Comment by Alex Miller [ 10/Aug/15 3:53 PM ]

Jozef, I don't see any reason why the independent callbacks in your test will necessarily conj in order as they contend for the atom. When I run the test I usually get a failure. If you agree, could wrap expected and actual in a set. Or if that seems wrong, then let's decide why.

Comment by Jozef Wagner [ 11/Aug/15 10:49 AM ]

Attached async-124-4.patch that uses unordered comparison in test

Comment by Alex Miller [ 14/Aug/15 1:01 PM ]

Added async-124-5.patch which should be semantically identical as -4 but retains the structure of a single dispatch point.

Comment by Alex Miller [ 14/Aug/15 3:03 PM ]

New -6 patch tries to retain more of the original algorithm and leverages the original loop instead of adding a new one.

Comment by Alex Miller [ 18/Aug/15 8:25 AM ]

New -7 patch addresses some bugs in the prior version and cleans up the loop code, also adds better tests.

Comment by Alex Miller [ 28/Aug/15 11:53 AM ]

test tweaks added in -8, per Stuart Sierra

Comment by Stuart Sierra [ 28/Aug/15 12:01 PM ]

Screened. Clojure version good. Still needs corresponding fix to ClojureScript version.





[ASYNC-103] promise-chan Created: 05/Nov/14  Updated: 28/Oct/15  Resolved: 28/Oct/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Alex Miller
Resolution: Completed Votes: 3
Labels: None

Attachments: Text File async-103-2.patch     Text File async-103-3.patch     Text File async-103-4.patch     Text File async-103-5.patch     Text File async-103-6.patch     Text File async-103-cljs.patch     Text File async-103.patch    
Patch: Code and Test
Approval: Ok
Waiting On: Alex Miller

 Description   

A promise-chan is a new kind of core.async channel that gives promise semantics to a channel.

  • Takes block prior to put or close!
  • After a put, all pending takes are notified, and any subsequent takes return the put value immediately
  • After a close (and no put), all pending takes are notified with nil, and any subsequent takes return nil immediately

In practice, a promise-chan is a normal channel that uses a promise-buf:

  • buf can only take on one value (or nil if closed) ever
  • buf is never emptied: N consumers can all get the same, one and only value by reading from the channel

In order to make close! sensible with this new kind of buf

  • all buffers need to be closeable
  • closing a channel closes its buffer
  • all basic buffer types do nothing when closed
  • a promise-buf make a one-time transition to having no value when closed (if not previously delivered)
  • closing a buffer is implementation detail not exposed to end users

Approach:

  • Buffer protocol now has a close-buf! function (close! would have overlapped the Channel function close!). close-buf! is invoked on a buffer when the channel is closed, allowing it to update itself if necessary.
  • Existing buffers implement close-buf! and do nothing (buffer still available for taking)
  • New promise-buffer implementation. Makes a one-time transition when value is supplied or buffer is closed. value is held in an unsynchronized-mutable deftype field - updates via add!* or close-buf! always happen under the containing channel mutex.
  • New api functions: promise-chan creates a channel with a promise-buffer and promise-buffer.

Patch: async-103-6.patch, async-103-cljs.patch (catches up cljs to diffs in async-103-6)



 Comments   
Comment by Ghadi Shayban [ 07/Nov/14 4:06 PM ]

My initial gut reaction was that this is more related to semantics of the channel, not the buffer, and I'm wary of significant changes to esp. the impl protocols. But after seeing an impl, it looks straightforward and the changes aren't too significant. (Another possibility is to make another simpler implementation of a channel, with just slots for the value, lock and pending takers before the value is delivered. No slots for xfn or putters or buffer handling would be needed.)

Note an atom is not needed in the PromiseBuffer, just a set! on a mutable field to be inline with the other buffer flavors. If the patch continues using an atom, maybe guard val to not swap unnecessarily.

Comment by Alex Miller [ 07/Nov/14 11:09 PM ]

Good call on the atom - backed off to just using existing clojure.lang.Box. If that's too weird, will just deftype it.

Comment by Alex Miller [ 07/Nov/14 11:18 PM ]

Dur, just need the val field itself.

Comment by Fogus [ 09/Jan/15 4:27 PM ]

I'd really love to see the reason for the current impl over a more pointed promise channel (perhaps as described by Ghadi). This is a clear implementation, but with the addition of close-buf! some complexity is added for implementations in the future. Specifically, I'd like to at least see a dcostring in the protocol describing the meaning and desired semantics around close-buf! and what it should return and when. Like I said, this is a quality patch, but I'm just concerned that there are unstated assumptions around close-buf! that escape me.

Comment by Alex Miller [ 28/Jan/15 12:43 PM ]

Fogus, I looked into implementing the promise-chan a bit today but it requires replicating a significant portion of the existing chan implementation. I believe that's the primary reason to do it this way, just leveraging the majority chan code.

New -5 patch has two changes - I commented the Buffer protocol fns, and I removed promise-buffer from the api, as I think users should only use promise-chan.

Comment by Fogus [ 30/Jan/15 10:06 AM ]

The patch looks good. My only (minor) reservation is that the Buffer docstrings are written under the assumption that people will use instances only in the context of a guarded channel. I understand why of course, so I think I might be too pedantic here. Because of this I see no reason not to +1 this patch.

Comment by Alex Miller [ 23/Feb/15 8:32 AM ]

Applied patch.

Comment by Alex Miller [ 03/Apr/15 3:57 PM ]

Reopening (and likely reverting commit).

This example demonstrates that only one of the waiting blocks will be notified, instead of all, as we'd like:

(let [c (promise-chan)] 
  ;; takers
  (dotimes [_ 3] 
    (go (println (<! c)))) 

  ;; put, should get to all takers
  (go (>! c 1)))
Comment by Jozef Wagner [ 30/May/15 10:35 AM ]

ASYNC-124 fixes the issue mentioned by Alex

Comment by Leon Grapenthin [ 11/Jun/15 11:44 AM ]

Quote: "buf is never emptied: N consumers can all get the same, one and only value by reading from the channel"

This is not true with the new closing semantics introduced here, correct? If I deref a Clojure promise, I get the same value everytime. But what is the purpose in closing promise-chans after the promise has been delivered?

When is a good time to close a promise-chan? If I close after the put, which is what many core.async processes do, takers must have taken before the close which means it will often be a race condition unless explicit care is taken.

Is there a benefit or am I simply missing something here?

Comment by Herwig Hochleitner [ 08/Jul/15 2:23 PM ]

Here is my cljs implementation of a promise channel: https://github.com/webnf/webnf/blob/master/cljs/src/webnf/promise.cljs
I'm the sole author and you are free to use any of it.

Comment by Alex Miller [ 12/Aug/15 3:00 PM ]

I believe that this patch can be re-evaluated after ASYNC-124 is applied as that patch addresses the reason this was retracted.

Comment by Leon Grapenthin [ 23/Aug/15 6:46 AM ]

It seems that there are two different approaches to how a promise-chan is supposed to or can be used.

1. All consumers are supposed to be takers before the promise is delivered. Taking after delivery is not intended, hence producers should close the channel after delivery.
2. " can take at any time and either have to wait for the promise (take blocks) or they get its value right away.

(2) models the behavior of real promises. (1) rather models that of a mutable reference.

It seems that the current patch explicitly makes room for (1) by giving producers the possibility to close.

Now if you want to use a promise-chan like (1) as a producer you can't create and return a promise-chan. Instead you need to be passed a promise-chan which has (a) taker(s) already. On the consumer side, it seems like an annoying quirk to enqueue takes before being able to pass the promise-chan to the producer (since he could close the promise immediately). I can't think of a scenario where one would want to do such thing. So my new question is:

Given the lengths taken here to enable (1), are there practical usecases for that? Why is closing a promise-chan not just going to be a no op?

Comment by Alex Miller [ 24/Aug/15 12:56 PM ]

Taking after delivery is intended. It is hard sometimes to know whether a take will happen before or after delivery in asynchronous systems so both must work. close! will deliver nil to all pending or future takers - in this case (as with all channels) nil indicates a drained channel, meaning that no delivery will ever happen.

However, one good question here is - what happens when you close a promise-chan that has already been delivered?

Currently:

  • (def pc (promise-chan))
  • take from promise-chan (blocks)
  • (>!! pc 5) - delivers 5 to blocked taker(s)
  • take from promise-chan (immediately returns 5) ;; for any number of takes
  • (close! pc 5)
  • take from promise-chan (immediately returns nil) ;; ???

I could easily be persuaded that the last answer is wrong and that it should always deliver 5 regardless of whether the pc gets closed (that is closing has no effect on delivered promise-chans).

Comment by Alex Miller [ 24/Aug/15 1:03 PM ]

I just confirmed with Rich that once delivered, a promise-chan should always deliver that value to takers, regardless of whether it is subsequently closed.

Comment by Stuart Sierra [ 28/Aug/15 4:02 PM ]

Screened. Clojure version good, assuming ASYNC-124 is applied first. ClojureScript version may or may not need additional work.

I agree that the addition of close-buf! and the no-op implementations feel like working around some missing abstraction. (What does it even mean to "close" a buffer?) But the Buffer protocol is not part of the public API, it's buried down in clojure.core.async.impl.protocols. The Channel and Buffer implementations are already tightly coupled, so adding another point of collaboration does not seem likely to cause problems. Adding close-buf! is certainly preferable to duplicating the implementation of ManyToManyChannel.





[ASYNC-6] alts! assumes nth works on ports argument Created: 15/Jul/13  Updated: 14/Oct/15  Resolved: 27/Sep/13

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Brandon Bloom Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None


 Description   

The docstring for alts! reads:

ports is a set of channel endpoints, which
can be either a channel to take from or a vector of
[channel-to-put-to val-to-put], in any combination.

However, trying to use an actual set yields the following exception:

java.lang.UnsupportedOperationException: nth not supported on this type: PersistentHashSet



 Comments   
Comment by Brandon Bloom [ 07/Aug/13 2:32 PM ]

Discussed at length in IRC on Aug 7th: http://www.raynes.me/logs/irc.freenode.net/clojure/2013-08-07.txt

Comment by Rich Hickey [ 07/Aug/13 9:28 PM ]

The documentation should be changed. alts requires an efficiently nth-able collection (e.g. a vector), else it will be slow.

https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L183

If CLJS supports nth on sets it is broken.

Comment by Brandon Bloom [ 07/Aug/13 9:52 PM ]

> alts requires an efficiently nth-able collection

Is that the case? Or does alts actually only require efficiently sampling a single element from a collection?

Efficient nth is a reasonable proxy, since the only choice we have for randomness is rand-int. However, one could imagine a CollSample protocol that would allow efficient random choice leveraging data-structure-specific knowledge, much as CollReduce works. Given the world as it exists now, I agree that this is a doc bug.

> If CLJS supports nth on sets it is broken.

That's the conclusion we came to in IRC. David quickly fixed this:
https://github.com/clojure/clojurescript/commit/a113b08a8c2811b0590cc6a36b2e9e5adc1c4c1e

Comment by Timothy Baldridge [ 27/Sep/13 8:47 AM ]

fixed in master

Comment by Daniel Compton [ 14/Oct/15 5:30 PM ]

For those reading this in the future, this was fixed by changing the docstring to use vector instead of set. https://github.com/clojure/core.async/commit/ca633479f51481f4aee40b4d60120484839d20c6





[ASYNC-147] subscriber blocks other subscriber of a topic Created: 24/Sep/15  Updated: 28/Sep/15  Resolved: 28/Sep/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Sonny To Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

Here is code to reproduce the problem:

code
(require-macros '[cljs.core.async.macros :refer [go go-loop]])
(require '[cljs.core.async :refer [put! <! >! chan pub sub unsub]])

(defonce message-bus (chan))
(defonce message-publication (pub message-bus (fn [msg]
(if (vector? msg)
(first msg)
:no-topic))))

(def s1 (chan))
(def s2 (chan))

(sub message-publication :foo s1)
(go (while true (println "s1 " (<! s1))))
(put! message-bus [:foo "bar"]) ;works as expected

(sub message-publication :foo s2) ;blocks s1 from recieving messages
(put! message-bus [:foo "bar"]) ;s1 cannot recieve this message
(go (while true (println "s2 " (<! s2)))) ; unblocks s1; both s1 and s2 can recieve messages

code

This problem happens on both clojure and clojurescript



 Comments   
Comment by Alex Miller [ 28/Sep/15 12:30 PM ]

This is the expected behavior. Docstring for pub states:

"Each item is distributed to all subs in parallel and synchronously, i.e. each sub must accept before the next item is distributed. Use buffering/windowing to prevent slow subs from holding up the pub."

In this code, you are using an unbuffered channel as a sub and (according to the above), it will block delivery.





[ASYNC-146] poll! still not being supported Created: 18/Sep/15  Updated: 18/Sep/15  Resolved: 18/Sep/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Miro Kubicek Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: maven, poll!, release


 Description   

In the last releases (0.1.346.0-17112a-alpha and older) poll! is still not supported even though it has been implemented for awhile (I can see it in the trunk and it seemed to be implemented as a part of ASYNC-104 in Feb).
If I try to use poll! with the latest version from maven repo I get "java.lang.IllegalAccessError: poll! does not exist".
Further checking maven repo history it even seems that the latest release is from 22-Sep-2014 - i.e. a year ago? Or is it a glitch? I understand this is an alfa, but especially for that reason it would be great if there were newer "nightly builds" / versions available in maven/leiningen.
Would getting the trunk (master) code directly from git be a suggested approach? (as opposed to using maven/leiningen)



 Comments   
Comment by Alex Miller [ 18/Sep/15 10:32 AM ]

I'm closing this as it's really a question not a ticket.

We are working towards a release in the near future that will include this functionality. Unfortunately, the existing build setup for this particular library does not make it easy to release interim snapshots - you are welcome to build your own from master if you like.





[ASYNC-141] Convert project to use Reader Conditionals Created: 18/Aug/15  Updated: 18/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

There is a fair amount of duplicated code between the Clojure and ClojureScript versions of core.async. Converting the library to use Reader Conditionals would get rid of a lot of code volume and reduce the chances of changes in one library not being correctly implemented in the other. It may also make maintenance and source reading easier by making the platform specific code stick out more, where the 'essence' of core.async stays in pure Clojure. This could also be an opportunity to bring cljs.core.async.macros into the main async namespace using reader conditionals.

Despite these benefits there are a few downsides to this:

  • Currently the two ports are able to exist fairly separate from each other and the domain experts in each can work on their own part without affecting the other. If this was a concern, more tests may be able to be added to ensure there aren't breakages
  • Combining the two ports may make it harder to read either version in the same source file.

As this is a fairly large change, I wanted to get feedback on this proposal before going any further.



 Comments   
Comment by Alex Miller [ 18/Aug/15 8:34 AM ]

For the time being, we are still supporting Clojure 1.6 with core.async so this is not an option. At some point, that will change, not sure when.

Another aspect here is that reader conditionals are designed for code that is very similar across platforms but the code for clj and cljs is substantially different in many cases, so it is not an obvious candidate. Rather, I would expect to see alternate versions of the same namespace in many cases.

Changing namespaces would also of course break existing users, so we would need to plan ahead for this change.

While I think there is an endpoint with more consolidated code base (and a better build system, which I have been working on), I do not think it's wise for anyone outside the core team to work on it.





[ASYNC-139] inconsistent thread pool in async/pipeline-blocking function Created: 15/Aug/15  Updated: 16/Aug/15  Resolved: 15/Aug/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Miro Kubicek Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

platform non-specific


Attachments: PNG File pipeline diff screenshot .png    
Patch: Code

 Description   

Hi there,
in pipeline* function for the :blocking processing there is no loop within each thread - while for the async processing there is always a go-loop [] which can be re-written as (go (loop []..., for sync processing there is only "thread" macro with no loop in it and with a recur call at the end. I would assume recur without a loop jumps back to a given defn. Why is there no loop? If a thread is instantiated then it should continue doing its work.

Apologies if I am missing something and the recur really jumps back to the thread macro.

To illustrate this in the code - basically I would expect that the following:

pipeline*
:blocking (thread
                           (let [job (<!! jobs)]
                             (when (process job)
                              (recur))))

Should look more like this:

pipeline*
:blocking (thread (loop[]
                           (let [job (<!! jobs)]
                             (when (process job)
                              (recur)))))

Even if the recur without loop jumps back to the thread macro I would wonder whether it might lead to some unexpected behavior (each thread being recreated after every iteration) and if it jumps elsewhere it might lead to the thread halting after only one iteration is performed.

My (closed) pull request for this fix:
https://github.com/clojure/core.async/pull/47/commits

Cheers
Miro



 Comments   
Comment by Alex Miller [ 15/Aug/15 4:09 PM ]

thread is a macro that nests the body inside an anonymous fn, which serves as a recur target. Thus the code is working as expected.

Comment by Miro Kubicek [ 16/Aug/15 1:25 AM ]

Thanks for the explanation Alex!





[ASYNC-102] implement IDeref, IBlockingDeref for channels Created: 05/Nov/14  Updated: 12/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File async-102-2.patch     Text File async-102.patch    
Patch: Code and Test
Approval: Vetted
Waiting On: Alex Miller

 Description   
  • each deref will take a value from the channel
  • deref from a closed channel returns nil
  • deref with timeout must either timeout or take value from the channel (not both!). Might just want to call some variant of alt!! through the var
  • use internals efficiently
  • do NOT implement IPending/realized, neither interpretation is that great

Patch: async-102-2.patch

The implementation of deref and timed deref is conceptually straightforward, but is difficult to implement due to circular namespace dependencies. Suggestions welcome.



 Comments   
Comment by Stuart Halloway [ 09/Nov/14 8:24 AM ]

The async-102.patch does not match the description "a channel becomes realized when it is closed" – instead, a channel becomes realized when a value is currently available. Both of these interpretations seem problematic to me, and I think the definition of realized? needs clarification to support this new case. The other uses of realized? in Clojure all guarantee that subsequent derefs will be filled immediately, but that will not be true for channels if another consumer takes the value.

Once that is solved, I need better doc strings for the core.async implementation protocols before I can screen this. What guarantees are made about use of threads? Where does the work enqueue, and what happens if the queue is full?

Comment by Alex Miller [ 09/Nov/14 10:32 PM ]

My implementation of realized? is based on Rich's response to your question (from internal chat): "open+empty = false, else true" and I believe that is what's implemented and reflected in the tests. This also makes more sense to me than only being realized on close.

Your comment vs other cases of realized? seems accurate, so I agree that's a question.

Can you be more specific on which implementation protocol? I'm guessing you specifically mean Channel and Buffer. Any thread could be calling into the Channel. The M2MC protects its internal state with a mutex and will forward calls down to the Buffer. All calls into the buffer are protected by the channel mutex. M2MC enqueues pending puts and takes in the puts and takes lists. Those are bounded by the fixed limit clojure.core.async.impl.protocols/MAX-QUEUE-SIZE (1024), at which point an exception is thrown.

Comment by Alex Miller [ 05/Jan/15 12:33 PM ]

The conceptual implementations of IDeref and IBlockingDeref on ManyToManyChannel using existing async constructs is relatively straightforward:

(deftype ManyToManyChannel
  #_existing_code...
  IDeref
  (deref [this] (<!! this))

  IBlockingDeref
  (deref [this ms timeoutValue]
    (alt!!
      this ([val _] val)
      (timeout ms) timeoutValue)))

However, M2MC is defined in clojure.core.async.impl.channels. <!!, alt!!, and timeout are all defined in clojure.core.async, which depends on clojure.core.async.impl.channels, so there is a cyclic dependency problem here. The <!! and timeouts are pretty easy to deal with looking up the var behind delay like this:

(def ^:private <!!' (delay (find-var 'clojure.core.async/<!!)))

;; then:
  IDeref
  (deref [this] (@<!!' this))

However, I'm a little stumped on how to do the equivalent with alt!!, which is a macro around do-alt and alts!!.

Comment by Ghadi Shayban [ 05/Jan/15 1:14 PM ]

Don't rely on resolving the circular dependency. Instead make a shared alt-flag and enqueue the two handlers [1]. fret will be delivering a promise, and then deref the promise.

[1] https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L230-L231

Comment by Alex Miller [ 06/Jan/15 11:42 AM ]

New patch goes back to prior implementation of blocking deref (basically guts of <!!) and adds new alts-based version of timed deref.

Comment by Fogus [ 09/Jan/15 1:11 PM ]

> However, I'm a little stumped on how to do the equivalent with
> alt!Unable to render embedded object: File (, which is a macro around do-alt and alts) not found.!.

I wonder if the following will solve the alts!! problem:

(def ^:private ^:macro alts!!' (deref (find-var 'clojure.core.async/alts!!)))

And then using it in the deref impl behind IBlockingDeref. As it stands it's difficult to grok the code as written given that it's using some core.async implementation details (indeed, the same code). Whereas your implementation of IBlockingDeref.deref is crystal clear, the implementation prompted by the cyclic dependency is... less so. Not to belabor the point (which I seem to be), but alts!! is a clear implementation for this, but using the guts muddies the water. If I had my druthers, I'd prefer clarity.

That said, being a human macro-expander I was able to reason through the implementation eventually.

Comment by Alex Miller [ 28/Jan/15 2:23 PM ]

That doesn't seem to work to me. If you have some formulation that would, that would be great though!





[ASYNC-43] queue-dispatcher should fall back to process.nextTick if setImmediate is not available Created: 07/Dec/13  Updated: 07/Aug/15  Resolved: 07/Aug/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Travis Vachon Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: cljs

Attachments: Text File async_43.patch     Text File nextTick.patch    
Approval: Triaged

 Description   

discussion here:

https://groups.google.com/forum/#!searchin/clojurescript/nextTick/clojurescript/RW1FMv0UoPE/hsMHI4SLKXYJ

discussion of the differences between setImmediate and nextTick here:

http://stackoverflow.com/questions/15349733/setimmediate-vs-nexttick

it sounds to me like nextTick should be ok, but I'm not familiar with the design decisions in the current implementation

I'm happy to create a patch - will do that shortly.



 Comments   
Comment by Travis Vachon [ 07/Dec/13 7:28 PM ]

patch here: https://github.com/clojure/core.async/pull/40

Comment by Travis Vachon [ 08/Dec/13 8:52 PM ]

just the code, not entirely sure how to test this, but have tested manually in the parse.com cloud code environment

Comment by Travis Vachon [ 10/Dec/13 9:53 AM ]

add git am able patch

Comment by Travis Vachon [ 05/Mar/14 5:21 PM ]

just want to ping this issue - anything else it needs to be mergable? I'm using a core.async fork at the moment and would love to get back on the main line!

Comment by David Nolen [ 05/Nov/14 6:24 AM ]

The patch probably needs updating - we've since switched to Google Closure for choosing the underlying dispatch functionality for the JavaScript environment.

Comment by David Nolen [ 07/Aug/15 2:10 PM ]

we use Google Closure to make this decision.





[ASYNC-136] Add webinar to presentations section in README.md Created: 30/Jul/15  Updated: 30/Jul/15  Resolved: 30/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Michiel Borkent Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: None


 Description   

Add a link to the core.async webinar presented by David Nolen to the presentations section in README.md.
The link is: http://go.cognitect.com/core_async_webinar_recording



 Comments   
Comment by Alex Miller [ 30/Jul/15 8:21 AM ]

Added





[ASYNC-135] alt! with a closed channel will still return results from the closed channel Created: 27/Jul/15  Updated: 30/Jul/15  Resolved: 30/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

I'm not sure if this is a documentation bug, or an implementation bug, but alt! will always return nil from chan1 in the following code.

(let [chan1 (async/chan 10)
      chan2 (async/chan 10)]
  (async/close! chan1)
  (async/>!! chan2 "run")
  (async/go (async/alt! :priority true

               chan1 ([val] (println "chan1" val))

               chan2 ([val] (println "chan2" val)))))
;; => chan1 nil

The intent of my code was to alt! over the two channels, preferring chan1, but running chan2 if chan1 has nothing to take. However changing this would interfere with using timeout channels in an alt!, so it seems like documenting this might be the best fix?



 Comments   
Comment by Leon Grapenthin [ 30/Jul/15 3:25 AM ]

This is expected. Takes from closed channels return nil. The behavior is documented in the docstring of close!





[ASYNC-94] Allow user supplied executor/thread-pool (and potentially other options) to be used in go, thread macros and chan Created: 09/Oct/14  Updated: 17/Jul/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Max Penet