<< Back to previous view

[ASYNC-167] Fix docstring for put! and friends Created: 27/Apr/16  Updated: 28/Apr/16  Resolved: 28/Apr/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alan Thompson Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: docstring, documentation


 Description   

The docstring for `put!` and friends is misleading and incorrect. They repeatedly mention "port" where the user knows only the term "channel".

Also, description of `fn0` is missing & incorrect. The phrase "Will throw if closed" is incorrect. fn0 is called with a single arg, which is `true` upon success and `false` if the channel is closed.



 Comments   
Comment by Alex Miller [ 28/Apr/16 5:23 PM ]

docstring was already updated





[ASYNC-166] Clarify clojure.core.async/chan docstring Created: 20/Apr/16  Updated: 20/Apr/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Karlis Lauva Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring

Attachments: Text File chan-docstring.patch    
Patch: Code

 Description   

Update the `chan` docstring, so that it specifies that buffer size must be positive when transducers are supplied.






[ASYNC-159] promise-chan in ClojureScript is broken Created: 13/Feb/16  Updated: 18/Apr/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Rangel Spasov Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Google Chrome 48.0.2564.109 (64-bit) Mac OS X El Capitan 10.11



 Description   

promise-chan in ClojureScript does not seem to work. In Clojure on the JVM, the code below produces the expected result.

Example:

(ns hush-vendor-cljs.example
(:require-macros [cljs.core.async.macros :refer [go]])
(:require [cljs.core.async :refer [promise-chan <! >!]]))

(def p (promise-chan))

(def go-1 (go (let [r (<! p)] (println "got value on promise-chan::" r))))

(def go-2 (go (let [r (<! p)] (println "got value on promise-chan::" r))))

(go (>! p 1))
;This prints only once in ClojureScript, should be twice. It works on JVM Clojure (prints twice).
;=> got value on promise-chan:: 1



 Comments   
Comment by Alex Miller [ 24/Mar/16 11:02 AM ]

Can you update with what version of core.async you're using?

Comment by Johannes Gustafsson [ 24/Mar/16 12:51 PM ]

I can confirm the bug using:

Mac OS X El Capitan
Chrome 49.0.2623.87 (64-bit)
core.async 0.2.374

I created an empty figwheel project using lein new figwheel.

In the REPL, I evaluate each expression one by one. When I evaluate the last expression, only one row is printed. If I evaluate the last expressione one more time, then the second print shows up.

Comment by Johannes Gustafsson [ 24/Mar/16 12:54 PM ]

I get the same result in Firefox

Comment by Rangel Spasov [ 24/Mar/16 1:54 PM ]

Alex - This is under 0.2.374. Sorry for missing the version, this is my first bug report .

Comment by Marcin Kulik [ 18/Apr/16 6:40 AM ]

I am experiencing the same issue. This prints only once:

(let [p (promise-chan)]
  (go
    (<! (timeout 500))
    (println (<! p)))
  (go
    (<! (timeout 500))
    (println (<! p)))
  (go
    (<! (timeout 1000))
    (>! p :yup)))

While this prints twice as expected:

(let [p (promise-chan)]
  (go
    (<! (timeout 1500))
    (println (<! p)))
  (go
    (<! (timeout 1500))
    (println (<! p)))
  (go
    (<! (timeout 1000))
    (>! p :yup)))

I am under 0.2.374.





[ASYNC-165] the binding macro/let binding is wrongly inlined Created: 06/Apr/16  Updated: 07/Apr/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Christian Weilbach Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: compiler
Environment:

[org.clojure/clojure "1.7.0"]
[org.clojure/clojurescript "1.8.34"]
[org.clojure/core.async "0.2.374"]


Attachments: Text File 0001-ASYNC-165-if-a-local-aliases-a-global-actually-do-th.patch     Text File 0002-ASYNC-165-if-a-local-aliases-a-global-actually-do-th.patch     Text File 0003-ASYNC-165-if-a-local-aliases-a-global-actually-do-th.patch    

 Description   

(def ^:dynamic foo 42)

(go
(let [old foo]
(set! foo 45)
(println old foo)
(set! foo old)))

leaves the binding with the value 45 (same code as with binding+with-redefs). The problem is that the let binding is somehow inlining the reference to foo. For instance the println statement compiles to:

...
var inst_43089 = cljs.core.println.call(null,full.async.foo,full.async.foo);
...

I am currently having a look at ioc_macros.clj, but I couldn't find the problematic part yet. Any hints are helpful.



 Comments   
Comment by Kevin Downey [ 06/Apr/16 6:21 PM ]

for the curious the cleaned up macro expansion of the above (in clojurescript) is:

(let* [c__9201__auto__ (chan 1)]
      (run
        (fn []
          (let [f__9202__auto__ (let [switch__9186__auto__ (fn [state_9258]
                                                             (let [state_val_9259 (aget state_9258 1)]
                                                               (cond
                                                                (== state_val_9259 1) (let [inst_9254 (set! foo 45)
                                                                                            inst_9255 (println foo foo)
                                                                                            inst_9256 (set! foo foo)
                                                                                            state_9258 (aset-all! state_9258 7 inst_9255 8 inst_9254)]
                                                                                        (return-chan state_9258 inst_9256)))))]
                                  (fn state-machine__9187__auto__
                                    ([] (aset-all! (make-array 9) 0 state-machine__9187__auto__ 1 1))
                                    ([state_9258] (let [ret-value__9188__auto__ (try
                                                                                  (loop []
                                                                                    (let [result__9189__auto__ (switch__9186__auto__ state_9258)]
                                                                                      (if (keyword-identical? result__9189__auto__ :recur)
                                                                                        (recur)
                                                                                        result__9189__auto__)))
                                                                                  (catch js/Object ex__9190__auto__
                                                                                    (aset-all! state_9258 5 ex__9190__auto__)
                                                                                    (process-exception state_9258)
                                                                                    :recur))]
                                                    (if (keyword-identical? ret-value__9188__auto__ :recur)
                                                      (recur state_9258)
                                                      ret-value__9188__auto__)))))
                state__9203__auto__ (-> (f__9202__auto__)
                                        (aset-all! USER-START-IDX c__9201__auto__))]
            (run-state-machine-wrapped state__9203__auto__))))
      c__9201__auto__)

the issue is definitely present in the macro expansion

Comment by Kevin Downey [ 06/Apr/16 7:00 PM ]

it looks like the issue is, let binds essentially disappear at runtime
because the ioc macros bind every expression to a name, so let binds
are just mapped to those names at compile time. in that mapping global
names essentially map to themselves, so let bound names that get their
value from a global just disappear and become references to the
global.

If you look at the `:symbol` case for `-item-to-ssa` in
ioc_macros.clj, there is a commented out `(add-instruction (->Const
x))`, if you uncomment out that and comment out the fn above it, I
think what you get out has the behavior you are looking for, at the
cost of creating a local for every global read.

You could do some kind of picking of the behavior based on if the
globals are declared to be dynamic, but that would still leave issues
for with-redefs. In JVM clojure with-redefs works on global names
regardless of if they are dynamic or not. You could generate some kind
of dirty set if you see a set! of a global, and use that to toggle the
behavior, but I think (not sure) that could run in to issues because
the analysis is local in scope.

Comment by Kevin Downey [ 06/Apr/16 7:04 PM ]

if you make the change I mentioned in my comment above, the relevant section of the cleaned up macro expansion looks like:

(let [inst_9257 foo
      inst_9258 foo
      inst_9259 (set! inst_9258 45)
      inst_9260 println
      inst_9261 foo
      inst_9262 (inst_9260 inst_9257 inst_9261)
      inst_9263 foo
      inst_9264 (set! inst_9263 inst_9257)
      state_9266 (aset-all! state_9266 7 inst_9259 8 inst_9262)]
  (return-chan state_9266 inst_9264))

that actually has a bug too, it turns out, because the set! is changing the value of a local instead of the global `foo`

Comment by Kevin Downey [ 06/Apr/16 8:46 PM ]

someone should check to see if the clojure ioc macros do this same
thing, because you could have a similar issue, something like:

(def ^:dynamic foo 42)

(go (binding [foo 5] (let [x foo] (set! foo 20) (println x)))

I would expect that to print 5, if there is some kind of aliasing bug, it might print 20

Comment by Kevin Downey [ 06/Apr/16 8:50 PM ]

the 001 patch causes local bindings that are initialized from a global to actually create a local and initialize it from the global, instead of reading from the global.

Comment by Kevin Downey [ 06/Apr/16 10:16 PM ]

patch 0002 is patch 0001, but using the same code path in let binding inits and loops binding inits

Comment by Kevin Downey [ 07/Apr/16 12:49 PM ]

0003 adds a test for the expected local binding behavior when aliasing a global





[ASYNC-162] Published API docs seem way out of date Created: 27/Feb/16  Updated: 31/Mar/16  Resolved: 26/Mar/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Avi Flax Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: documentation


 Description   

The home page says that the current release is 0.2.374. When I follow the link API Docs that page says:

API for clojure.core.async - Facilities for async programming and communication 0.1.0 (in development)

Which doesn't give me much confidence that the docs are accurate for the current version of the library.



 Comments   
Comment by Alex Miller [ 24/Mar/16 10:47 AM ]

I think that 0.1.0 must be hard-coded into the doc generation somewhere - I will track that down. The docs are current though for released version.

Comment by Alex Miller [ 26/Mar/16 8:12 AM ]

Updated docs to report the major/minor version based on the code.

The docs themselves were and are up to date with the latest release.

Comment by Avi Flax [ 31/Mar/16 10:08 AM ]

That's excellent! Thanks!





[ASYNC-160] `(vector :blah)` and `[:blah]` crashing with `inst_XXXX.call not a function` within core.async go routine Created: 20/Feb/16  Updated: 24/Mar/16  Resolved: 24/Mar/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jonathan Leonard Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Environment:

Node.js v4.1.1
Mac OS X 10.11.2
JVM 1.8.0_45



 Description   

Summary says it all.

[Worked around it by using a list rather than a vector (which compiled fine)].

I don't have time to create a minimal repro at the moment but if no one can repro this and it's necessary, I can devote time to that later.



 Comments   
Comment by Nicola Mometto [ 21/Feb/16 3:22 PM ]

This ticket doesn't have enough info to be useful:

user=> (go (vector :foo))
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x3e59dd7e "clojure.core.async.impl.channels.ManyToManyChannel@3e59dd7e"]
user=> (go [:foo])
#object[clojure.core.async.impl.channels.ManyToManyChannel 0x70b36700 "clojure.core.async.impl.channels.ManyToManyChannel@70b36700"]
Comment by Jonathan Leonard [ 21/Feb/16 3:33 PM ]

The actual code was more like:

(defn func []
  (go
    (let [...]
      (swap! an-atom update-in [:foo] ...))))

I would imagine that the let bindings themselves aren't the culprit but if you need yet more detail I will try to obfuscate them enough to provide them here.

Comment by Jonathan Leonard [ 21/Feb/16 3:35 PM ]

Also, this was from ClojureScript; not Clojure.

Comment by Nicola Mometto [ 21/Feb/16 3:50 PM ]

(swap! x [:foo] ..)
will invoke
(apply [:foo] @x ..)
which is probably not what you want and the cause of the exception you're seeing. Not a core.async bug

Comment by Jonathan Leonard [ 21/Feb/16 4:46 PM ]

Sorry, it was:

(swap! an-atom update-in [:foo] ...
Comment by Jonathan Leonard [ 21/Feb/16 4:48 PM ]

Also, note that changing [:foo] to (list :foo) or '(:foo) side stepped the erroneous compiler output and allowed to code to succeed without any additional changes anywhere to the code.

Comment by Nicola Mometto [ 21/Feb/16 5:20 PM ]

I'm sorry but this is still not enough information.
You're going to have to post an actual failing case if you want somebody to be able to help you

Comment by Jonathan Leonard [ 21/Feb/16 5:24 PM ]

This isn't a matter of helping me: I already have a workaround (namely use `(list ...)` instead of `(vector ...)`).

I reported the bug for everyone's benefit.

Comment by Nicola Mometto [ 21/Feb/16 5:51 PM ]

Let me rephrase

Opening a ticket with only assumptions about the cause of an issue you're seeing and no details about the actual code that's causing it, is not helpful to anybody, all we can do is play guesswork.

If you want a ticket to be helpful (either to you, to the users of the library or to its maintainers in fixing the reported issue), you're at least going to have to post enough information for maintainers/contributors to reproduce the supposed bug.

In this particular case, even without a reproducible case I can speculate that it's much more likely that the problem lies in your own code than in core.async: nothing in the `go` macro would make a call to `list` compile any different than a call to `vector`

Comment by Jonathan Leonard [ 21/Feb/16 9:13 PM ]

It sounds like you are making at least as many and as great in magnitude of assumptions as me (in fact more since I know the code in question and its resultant compiler output).

I will distill the code and post it so that all doubts may be removed.

Comment by Jonathan Leonard [ 21/Feb/16 10:38 PM ]

Here it is in all its glory:

(def a (atom {:a {:b '() :c '()}}))
(defn- repro-async-160 []
  (go
    (let [_ (<! (to-chan [1 2 3 4]))
          new (for [order (range 100)] nil)]
      (swap! a update-in [:a] assoc :b '() :c '()))))

Note that changing `new` to `z` avoids the issue. I suspect that identifier shadowing in let binding is not playing nicely with all the other macros in play (i.e., for & go).

Is `new` some kind of [undocumented] reserved word?

Comment by Jonathan Leonard [ 21/Feb/16 10:42 PM ]

Actually it may be colliding with the JavaScript notion of 'new' (in which case this is a ClojureScript bug and not a core.async-- but they sent me here first actually).

Comment by Nicola Mometto [ 22/Feb/16 3:24 AM ]

This sounds like a dupe of ASYNC-63

Comment by Alex Miller [ 24/Mar/16 10:58 AM ]

Dupe of ASYNC-63

Comment by Jonathan Leonard [ 24/Mar/16 12:10 PM ]

I'm not convinced that this is a duplicate. The repro steps and behavior are slightly different. It may certainly be merely a different manifestation of the same underlying or related issue and any potential fix may resolve both issues but I don't think (as an outside observer) that that is necessarily so.





[ASYNC-164] Limited partition-all in pipeline Created: 19/Mar/16  Updated: 24/Mar/16  Resolved: 24/Mar/16

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Corin Lawson Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

a6dc01d2ebb692a39e923988b0853ff2bd1a431b



 Description   

Using partition-all as the transducer in a pipeline has unexpected results.

(let [to (chan)]
  (pipeline 1 to (partition-all 2) (to-chan (range)))
  (repeatedly 4 #(<!! to)))

Expected Output: ([0 1] [2 3] [4 5] [6 7])
Actual Output: ([0] [1] [2] [3])

It seems to me that the res channel should be created inside the dotimes form and closed when there are no more jobs. (This breaks the contract of the async function af). I have created a simplified version of pipeline (below) which exhibits the desired behaviour. (Note this is a modified version of the function that I submitted to ASYNC-150, i.e. it breaks the ordering of the output.) I attempted to modify the current definition of pipeline* but the p channels are confusing me (I don't understand why they are necessary) and failed in my effort to write a working patch.

(defn pipeline
  ([n to xf from] (pipeline n to xf from true))
  ([n to xf from close?] (pipeline n to xf from close? nil))
  ([n to xf from close? ex-handler]
   (assert (pos? n))
   (let [ex-handler (or ex-handler (fn [ex]
                                     (-> (Thread/currentThread)
                                         .getUncaughtExceptionHandler
                                         (.uncaughtException (Thread/currentThread) ex))
                                     nil))]
     (go
       (<! (async/merge
             (repeatedly n
                         #(let [res (chan 1 xf ex-handler)]
                            (go-loop []
                                     (if-let [v (<! from)]
                                       (do
                                         (>!! res v)
                                         (recur))
                                       (close! res)))
                            (go-loop []
                                     (let [v (<! res)]
                                       (when (and (not (nil? v)) (>! to v))
                                         (recur))))))))
       (when close? (close! to))))))


 Comments   
Comment by Kevin Downey [ 20/Mar/16 1:06 PM ]

this is not possible.

pipeline effectively runs a transducer in parallel, partition-all is based on sequential ordering. your rewritten pipeline will also fail to result in the partition you want.

if you look at the docstrings of transducer returning functions in core, some of them say "returns a transducer" some say "returns a stateful transducer", most stateful transducers will behave differently when run in parallel, because each concurrent instance will have its own state. partition-all is a stateful transducer.

Comment by Corin Lawson [ 20/Mar/16 4:02 PM ]

It is quite foolhardy to say my rewritten pipeline will fail to result in the partition I want, did you try it? I am very much aware that partition-all is a stateful tranducer; this fact is the crux of the problem, and the reason I wish to moved the res channel to the dotimes form, so that each of the n channels will maintain its own state. As I previously mentioned, order is unimportant to me but state is, the test that I actually need to pass is the following, in which my function succeeds and it also meets my other requirements. I share it here with you to show you the particular aspect of the behaviour that I expected from core.async/pipeline, nothing more.

(let [to (chan)]
  (pipeline 3 to (partition-all 2) (to-chan (range)))
  (repeatedly 4 #(-> to <!! count (= 2))))

Expected Output: (true true true true)

Comment by Kevin Downey [ 20/Mar/16 10:58 PM ]

perhaps I misunderstood, but in your example you said your expected output is

([0 1] [2 3] [4 5] [6 7])

which I read as an ordered processing of the input

Comment by Corin Lawson [ 20/Mar/16 11:02 PM ]

Perhaps, but the point I'm trying to make is about the size of the vectors in the output.

Comment by Kevin Downey [ 20/Mar/16 11:14 PM ]

the p channels are there to preserve the order of the output.

they are created and paired with jobs in the same order as jobs come in from the from channel, and put into the results channel in the same order, and then the final go loop copies from each channel in turn, preserving the order of results regardless of what order they generated in by the workers.

using the same channel(and transducer state) for the life time of the worker makes that ordering impossible. pipeline walks a line allowing a certain of random ordering of processing internally, but the order of the output is the order of the input.

there may be interest in something like this (I know I often don't really care about order) but it would be a large departure from the existing behavior of pipeline.

Comment by Corin Lawson [ 20/Mar/16 11:27 PM ]

Okay, that makes sense to me now, thank you. I agree, unordered output is a departure from the existing behaviour. I also accept that it is unreasonable to enforce an order over the output, any such ordering would be at the loss of parallelism.

(Please close, am I able to close this ticket myself?)

Comment by Kevin Downey [ 21/Mar/16 12:08 AM ]

I honestly don't know. I am a random guy on the internet, so you may want to leave it open and see if someone more closely tied to the project has thoughts

Comment by Corin Lawson [ 21/Mar/16 12:13 AM ]

Ha! Thank you fellow random guy on the internet!

Comment by Alex Miller [ 24/Mar/16 10:55 AM ]

Note that the pipeline docstring says "Because it is parallel, the transducer will be applied independently to each element, not across elements, and may produce zero or more outputs per input." which explains the results.

I would not expect a transducer with a stateful reducing function to work with the parallelism in partition-all (other than with the caveat specified above).





[ASYNC-138] Go blocks leak memory Created: 10/Aug/15  Updated: 24/Mar/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Brian Lubeski Assignee: Unassigned
Resolution: Unresolved Votes: 11
Labels: memory
Environment:

clojure 1.7.0
core.async 0.1.346.0-17112a-alpha
Java HotSpot(TM) Client VM 1.8.0_31-b13


Attachments: Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2-nested.patch     Text File 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2.patch    
Patch: Code
Approval: Triaged

 Description   

The following example, after running for a few minutes, generates an OutOfMemoryError.

(let [c (chan)]
  (go (while (<! c)))
  (let [vs (range)]
    (go
      (doseq [v vs]
        (>! c v)))))

By contrast, the following example will run indefinitely without causing an OutOfMemoryError.

(let [c (chan)]
  (go (while (<! c)))
  (go
    (let [vs (range)] 
      (doseq [v vs]
        (>! c v)))))

The only significant difference I see between the two examples is that the (range) is created outside the go block in the first example but is created inside the go block in the second example. It appears that the go block in the first example is referencing vs in such a way as to prevent if from being garbage-collected.

This behavior might also be the cause of ASYNC-32.

Patch: 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2.patch



 Comments   
Comment by Nicola Mometto [ 15/Dec/15 9:34 AM ]

attached is a WIP patch to fix this issue, would appreciate some testing on this

Comment by Nicola Mometto [ 15/Dec/15 5:52 PM ]

CLJ-1744 contributes to this bug

Comment by Nicola Mometto [ 15/Dec/15 6:23 PM ]

Updated patch to preserve type info

Comment by Nicola Mometto [ 17/Dec/15 10:16 AM ]

Note that the current patch breaks nested `go` blocks at compile time since the nested `go` block will be macroexpanded by tools.analyzer.jvm rather than Compiler.java and this &env will be different.

Is this something we want to support? There seem to be a lot of other cases that break when nesting `go` blocks.

If this is something that we do want to support 0001-ASYNC-138-allow-for-clearing-of-closed-over-locals-v2-nested.patch fixes this

Comment by Jan Rychter [ 21/Dec/15 6:18 AM ]

This just bit me badly when doing data processing work — I never expected that (async/to-chan (line-seq rdr)) can be the culprit of my heap getting exhausted in 30 seconds. Took quite some time to narrow it down. Thanks to Nicola for letting me know about this bug on Slack.





[ASYNC-163] Off-by-two in pipeline-async parallelism Created: 17/Mar/16  Updated: 21/Mar/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Moritz Heidkamp Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.374"]



 Description   

The actual parallelism of pipeline-async is greater by 2 than what was passed. Here's the simplest reproduction case I could come up with:

(let [message-count 20
      parallelism 5
      parallel-count (atom 0)
      input (async/to-chan (range message-count))
      output (async/chan message-count)]
  (async/pipeline-async parallelism
                        output
                        (fn [_ result]
                          (async/go
                            (swap! parallel-count inc)
                            (async/<! (async/timeout 1000))
                            (async/>! result @parallel-count)
                            (swap! parallel-count dec)
                            (async/close! result)))
                        input)
  (async/<!! (async/into [] output)))

Results in something like this:

[7 4 7 5 3 7 2 7 7 7 5 3 1 3 6 4 3 1 2 6]


 Comments   
Comment by Kevin Downey [ 17/Mar/16 2:07 PM ]

I suspect this is just a lack of clarity around what parallelism means
in the context of pipeline-async.

A parallelism of 'n' for pipeline-async means there are 'n' logical
threads spinning off async tasks, and because those tasks are async,
those logical threads spin off the task, and proceed to the next. The
other thing that 'n' effects is the size of the buffer for the
`results` channel internally in the pipeline. because the tasks are
asynchronous, the only throttle is effectively the size of the buffer
on the output channel.

If the only throttle is the buffer, which is of size 'n', why do we
seem to have n+2 logical threads running at once, well because there
are a few extra channels involved which combined with the logic for
copying from channel to channel, effectively adds another buffer of
size 2.

This behavior does seem arcane. But I would argue that the
behavior of pipeline-async is so different from pipeline and
pipeline-blocking that it really should be something distinct from
them. The other pipelines take transducers, and using the parallelism
'n' to spin up a number of logical threads for the other pipelines
actually does throttle the number of tasks run in parallel.

If you have not spent time evaluating which `pipeline` variant you
needed to use, and just grabbed `pipeline-async` because that is the
one that seemed intuitively the right choice (because it has `-async`
and this is `core.async`) you almost certainly should be using the
vanilla `pipeline` variant, its behavior is much more intuitive it is
much more likely to be what you want.

Comment by Moritz Heidkamp [ 21/Mar/16 5:30 PM ]

Hi Kevin, thanks for your reply! After reading it, I think your point
regarding clarity around what parallelism means in the context of
pipeline-async is spot on It seems to be quite a different
beast from the other pipelines, indeed.

We actually did evaluate the various options and from the
documentation it seemed to be the right tool in our case: We want to
launch a maximum number of parallel asynchronous HTTP requests. We
immediatley ruled out pipeline-blocking since we are not doing
blocking requests after all and don't want to hog up n threads
just for waiting for the requests to finish. We also figured that this
isn't really "computational parallelism" (as the documentation of
pipeline puts it), so pipeline-async seemed like the best
fit. Its API also looked well suited for this case: We just needed to
launch the HTTP request in af and then fill and close the
conveniently supplied result channel in the the request's completion
callback. This seemed to work fine – until we actually hit the limit
of the connection pool, that is.

I'm not sure how we would use pipeline to the same effect. And
moreover, it's now rather unclear to me what pipeline-async's
intended use case is. Can you perhaps enlighten me some more? That
would be much appreciated

Comment by Kevin Downey [ 21/Mar/16 7:05 PM ]

I dunno.

That does sort of sound like a thing for which you would use pipeline-async, but my experience with nio predates core.async, so I am not sure what the best way to stick the two together is. I am sort of skeptical about combining even "async" io libraries like http clients with core.async, since the async of the io system is different from the async of core.async, and when combining the two it is easy to screw up and use some blocking convenience function because async is hard.

depending on the nature of the http request and how you consume the results it would, of course, be way simpler to use a synchronous http library and pipeline-blocking. if you are bounding the number of http requests at a time to a certain number, I suspect you will run out of io capacity before the jvm has trouble with that number of threads.

if pipeline-async is working for you otherwise, besides the +2 thing, then, uh, maybe just subtract 2





[ASYNC-158] and with multiple <! does not short-circuit in go Created: 26/Jan/16  Updated: 14/Mar/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Asher Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Environment:

org.clojure/clojurescript "1.7.48", org.clojure/clojure "1.7.0-RC1", org.clojure/core.async "0.2.374"



 Description   

I have an and condition inside a go block, where the two conditions take from channels.
When the first condition returns false, I would expect the second condition not to be checked at all. But, it is.

Here is the code:

(defn cc [x]
   (go 
     (println "cc: " x)
   x))

  (defn foo [x]
    (go (when (and (= :ok (<! (cc x)))
                   (= :ok (<! (cc 2))))
          (print "after and"))))

(foo 1)
(foo :ok)

When running (foo 1), I would expect only cc: 1 to be printed, but actually cc: 2 is printed as well.

Happens only in ClojureScript, not in Clojure core.async.



 Comments   
Comment by Asher [ 26/Jan/16 6:14 AM ]

Note: This only happens in ClojureScript. In Clojure it works as expected.

Also, sorry for the poor formatting.. this is my first Jira issue.

Comment by Wang Xing [ 14/Mar/16 9:39 PM ]

Seems and works abnormal in many cases, not just with <!.
I tested this in:
clojure-1.8.0, clojurescript-1.7.228, core.async-0.2.374.
this test need nodejs with the request module.

(def request (js/require "request"))

Now I try to send request to an unaccessible site: google.
Yes, I can't use google.

(def result-chan (chan))

(request
  "http://www.google.com"
  (fn [err res body]
    (go
      (if (and (not err) (= (.-statusCode res) 200))
        (>! result-chan body)
        (>! result-chan [])))))

the result should be "[]" (the request will timeout make "err" non-nil)

(go
  (println "result:" (<! result-chan)))

but actually I get:
TypeError: Cannot read property 'statusCode' of undefined ...
I have printed err and I am sure it's a JS object in this case, not nil or false. that's why res is undefined.
And it also means the short-circuit for and didn't work.
to make things right, use "if" instead of "and"

(def result-chan (chan))

(request
  "http://www.google.com"
  (fn [err res body]
    (go
      (if (not err)
        (if (= (.-statusCode res) 200)
          (>! result-chan body)
          (>! result-chan []))
        (>! result-chan [])))))

then test it again:

(go
  (println "result:" (<! result-chan)))

work as expected now:

result: []




[ASYNC-161] `<!` inside nested `let` inside `go` block erroneously yields "<! not used inside go block" error Created: 21/Feb/16  Updated: 21/Feb/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Jonathan Leonard Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mac OS X.
Node 4.1.1.
Clojure 1.7.0
ClojureScript 1.7.170



 Description   

Title says it all.



 Comments   
Comment by Jonathan Leonard [ 21/Feb/16 3:08 PM ]

Actually, my code is a bit more complicated:

(defn func []
   (go
     (let [...]
       (for [[a b c d] e]
         (let [...]
            (when pred
               (let [val (<! ...)]  ...)))))))
Comment by Nicola Mometto [ 21/Feb/16 3:20 PM ]

You cannot user `for` inside a go block

Comment by Jonathan Leonard [ 21/Feb/16 3:39 PM ]

The `for` cannot appear anywhere inside a `go`? Or the `for` cannot contain inside it any async calls; i.e., is it ok for `for` to appear somewhere in the let binding as a sibling to a call to <!?

Comment by Nicola Mometto [ 21/Feb/16 3:48 PM ]

You can't use `<!` inside a `for` inside a `go` block, code like this OTOH should be valud:

(go 
  (let [.. (for [..] ..)] 
    (<! ..)))
Comment by Jonathan Leonard [ 21/Feb/16 4:45 PM ]

Ahh, perfect! Thx!





[ASYNC-126] Add public channel "closed?" predicate Created: 08/Jun/15  Updated: 12/Feb/16

Status: In Progress
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Unresolved Votes: 4
Labels: None

Attachments: Text File 0001-ASYNC-126-Add-public-channel-closed-predicate.patch     Text File 0002-ASYNC-126-Add-public-channel-closed-predicate.patch    
Patch: Code
Approval: Triaged

 Description   

Current patch file: 0002-ASYNC-126-Add-public-channel-closed-predicate.patch

Add a public function closed? in clojure.core.async to determine if a channel is closed without putting or taking any values.

This is a trivial wrapper around clojure.core.async.impl.protocols/closed?

There could still be a race between closed? and close! on another thread. This is just a check to avoid doing unnecessary work if a channel is already closed.

Example use case: a producer process which puts values on a channel can check if a channel is closed? before doing the work to produce the next value.



 Comments   
Comment by Alejandro [ 18/Jul/15 11:41 AM ]

The patch only adds 'closed?' to the ClojureScript version of core.async, is there a reason for not adding it to the Clojure version?

Comment by Stuart Sierra [ 19/Jul/15 12:07 PM ]

Previous patch was missing the Clojure(JVM) implementation. Fixed in new patch 0002-ASYNC-126-Add-public-channel-closed-predicate.patch.

Comment by Brandon Bloom [ 12/Feb/16 1:30 PM ]

Please do not add a closed? predicate to the public API. It's a race condition in 99.9% of cases. Go does not provide such a predicate and after growing accustom to CSP via both core.async and Go, I have never felt the need for it. Having the predicate would invite its use. I enjoy clojure.core's preference for omission of questionable functions and expect the same from core.async.

If somebody desperately feels the need for this predicate, ask in IRC or Slack, I'm sure somebody can help you fix the structuring of your channels/processes/etc.

Comment by Ghadi Shayban [ 12/Feb/16 2:16 PM ]

Echoing Brandon, I'm also unconvinced. This is a bug waiting to happen.





[ASYNC-133] Expose 'closed?' predicate Created: 18/Jul/15  Updated: 12/Feb/16  Resolved: 18/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alejandro Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Description   

It can be useful to, given a channel, use a predicate to know wheter is closed or not. The 'closed?' predicate is currently not public and remains an implementation detail.



 Comments   
Comment by Alex Miller [ 18/Jul/15 11:36 AM ]

Dupe of CLJ-126

Comment by Brandon Bloom [ 12/Feb/16 1:27 PM ]

I think you mean dupe of http://dev.clojure.org/jira/browse/ASYNC-126





[ASYNC-100] core.async with multiple catch blocks causing weird loop behaviour Created: 27/Oct/14  Updated: 26/Jan/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tom Coupland Assignee: Unassigned
Resolution: Unresolved Votes: 5
Labels: None

Attachments: Text File ASYNC-100_2.patch    
Patch: Code and Test
Approval: Triaged

 Description   

I've been seeing this weird looping behavior with some go loops over the last few days. An exception is being thrown from a function within the loop and rather than logging and looping back around to a waiting take, the loop seems to just hop back to the line before the function call.

I've managed to boil it down to the following:

(def s (chan))
(def single
  (go-loop []
    (try
      (prn "Awaiting single")
      (<! s)
      (prn "Single")
      (throw (new Throwable))
      (catch Throwable t
        (prn t)))
    (recur)))

(def d (chan))
(def double
  (go-loop []
    (try
      (prn "Awaiting double")
      (<! d)
      (prn "Double")
      (throw (new Throwable))
      (catch Exception re
        (prn re))
      (catch Throwable t
        (prn t)))
    (recur)))

Now if you (>!! s :a), you'll see the throwable printed out and the loop go back to waiting on the s channel. However, (>!! d :a) and you'll get to enjoy an infinite stream of 'Double'. In actual fact you can remove the -loop from double and get the same result.

Not sure what's going on here at all. In the macro expanded version of double '(prn t)' doesn't appear at all (it does in single's expansion), so it looks like it's not surviving the move into the state machine and instead is routing back to (prn "Double") or the take isn't really completing somehow, leaving the :a on the chan.



 Comments   
Comment by Paavo Parkkinen [ 03/Dec/14 5:30 PM ]

From my experiments, it seems what's causing the issue isn't the two catch clauses, but the fact that you are catching an Exception first, and then the Throwable. When I switched the two catch clauses (Throwable first) the issue went away.

You don't even need the two catch clauses to trigger it, a single catch clause of an Exception will do.

;; Works
(require '[clojure.core.async :as async])
(def c (async/chan))
(def st
  (async/go-loop []
    (try
      (prn "Awaiting")
      (async/<! c)
      (prn "Received")
      (throw (new Throwable))
      (catch Throwable t
             (prn t)))
    (recur)))
(async/>!! c :a)
;; Doesn't work
(require '[clojure.core.async :as async])
(def c (async/chan))
(def se
  (async/go-loop []
    (try
      (prn "Awaiting")
      (async/<! c)
      (prn "Received")
      (throw (new Throwable))
      (catch Exception t
             (prn t)))
    (recur)))
(async/>!! c :a)
Comment by Paavo Parkkinen [ 04/Dec/14 12:18 AM ]

Attaching a patch that fixes the original issue, but not the one in my comment above with an uncaught exception. All test cases pass.

Instead of just picking the first catch block, and creating a state machine block and exception frame for that one, I create blocks and frames for all of them.

I tried creating a test case too, but was unable to create one that would reproduce the error. I'll spend some more time trying to write a test case, but I wanted to submit the code patch without the test coverage first.

Comment by Paavo Parkkinen [ 04/Dec/14 7:45 PM ]

Attached patch with test case included.

Comment by Stuart Sierra [ 18/Jun/15 2:52 PM ]

Possible workaround until this is fixed: Just have one catch clause, catching all Throwable. Then examine the type of the throwable object inside the catch block to decide what to do.

Comment by Ken Allen [ 29/Oct/15 1:34 PM ]

This is even more troublesome when the exception is thrown from code that's not part of what the user wrote in the go block. You don't even need a loop for it to loop forever:

(require '[clojure.core.async :as async])
(def c (async/chan))
(doseq [_ (range 1024)] (async/put! c :test))
(async/go (try (println "BEFORE") (async/>! c :test) (catch Exception ex (println "ERR"))))

This was confusing as hell when we ran into it but catching Throwable instead does indeed work around the problem so that's what we've been doing. Was hoping the latest release of core.async would fix this.

Comment by Thomas Getgood [ 26/Jan/16 10:46 AM ]

As it is, you need to catch Throwable or nothing. Unhandled Throwables will cause this as well.

(require '[clojure.core.async :refer [go <! put! chan]])
(def ch (chan))
(go (try (print (<! ch)) (assert false) (catch Exception e nil)))
(put! ch 1)

Will print an infinite stream of 1s.

It was a lot of fun figuring out why functions failing :pre conditions caused infinite loops.





[ASYNC-155] Preserve loop binding metadata when inside a go block Created: 15/Dec/15  Updated: 04/Jan/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File 0001-ASYNC-155-preserve-loop-binding-metadata.patch    
Patch: Code
Approval: Triaged

 Description   

Description:
Currently this causes a reflection warning because core.async loses the type hint on the binding:

(go (loop* [^String foo "foo"](.substring foo 0) (<!) (recur nil)))
Reflection warning, /tmp/form-init2119477256040974630.clj:1:1 - call to method substring can't be resolved (target class is unknown).

This is a problem since macros like `doseq` expand to similar code and cause reflection warnings

The attached patch preserves that metadata and removes the reflection warning

Patch: 0001-ASYNC-155-preserve-loop-binding-metadata.patch



 Comments   
Comment by Marcus Crestani [ 04/Jan/16 3:17 AM ]

I can confirm that this patch also fixes ASYNC-157. Thanks!





[ASYNC-157] Reflection warning with doseq Created: 04/Jan/16  Updated: 04/Jan/16

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Marcus Crestani Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

OS X 10.11.2
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
[org.clojure/clojure "1.7.0"]
[org.clojure/core.async "0.2.374"] and also core.async GIT HEAD, latest commit 6e8b1aa



 Description   

A doseq within a go block that contains some async calls causes a reflection warning.
Here is a simple example:

(ns dummy
  (:require [clojure.core.async :as async]))

(defn dummy
  [ch cs]
  (async/go
    (doseq [c cs]
      (async/>! ch c))))
Reflection warning, /Users/crestani/activegroup/factorylink/dummy.clj:6:3 - call to method nth can't be resolved (target class is unknown).

How can we avoid these warnings?



 Comments   
Comment by Nicola Mometto [ 04/Jan/16 2:53 AM ]

Dupe of ASYNC-155





[ASYNC-156] (ClojureScript) go block containing letfn does not compile Created: 30/Dec/15  Updated: 30/Dec/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Gabe Johnson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: cljs, go-macro
Environment:

[org.clojure/clojure "1.8.0-RC3"]
[org.clojure/clojurescript "1.7.189"]
[org.clojure/core.async "0.2.374"]



 Description   

The following compiles and runs correctly:

(ns cljs-letfn-go-bug.core
  (:require [clojure.core.async :refer [go]]))

(go
  (letfn [(foo [x] x)] (foo 1)))

However, this fails to compile with `clojure.lang.ExceptionInfo: bindings must be vector of even number of elements...`:

(ns cljs-letfn-go-bug.core
  (:require-macros [cljs.core.async.macros :refer [go]]))

(go
  (letfn [(foo [x] x)] (foo 1)))


 Comments   
Comment by Gabe Johnson [ 30/Dec/15 11:09 AM ]

Please forgive the markdown. I haven't used JIRA in quite some time and don't appear to have permissions to edit the description.





[ASYNC-154] (require :reload-all) causes NullPointerException Created: 29/Nov/15  Updated: 22/Dec/15

Status: Reopened
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Aaron Cummings Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Both Oracle and IBM JDK (1.7) affected.


Approval: Triaged

 Description   

I'm seeing a problem with (require :reload-all ...) with clojure.core.async, where running with :reload-all causes a NullPointerException as shown below.

Curiously, this does not happen with 'lein repl'; it seems the :reload-all is being somehow inhibited there.

So, starting the REPL like this:

java -cp asm-all-4.2.jar:clojure-1.7.0.jar:core.async-0.2.374.jar:core.cache-0.6.4.jar:core.memoize-0.5.8.jar:data.priority-map-0.0.7.jar:tools.analyzer-0.6.7.jar:tools.analyzer.jvm-0.6.9.jar:tools.reader-1.0.0-alpha1.jar clojure.main

Results below:

Clojure 1.7.0
user=> (require ['clojure.core.async])
nil
user=> (require :reload-all ['clojure.core.async])
CompilerException java.lang.NullPointerException, compiling:(clojure/core/async.clj:1138:8) 
user=> *e
#error {
 :cause nil
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "java.lang.NullPointerException, compiling:(clojure/core/async.clj:1138:8)"
   :at [clojure.lang.Compiler analyzeSeq "Compiler.java" 6730]}
  {:type java.lang.NullPointerException
   :message nil
   :at [clojure.tools.analyzer.passes.jvm.warn_on_reflection$eval10493$fn__10494 invoke "warn_on_reflection.clj" 58]}]
 :trace
 [[clojure.tools.analyzer.passes.jvm.warn_on_reflection$eval10493$fn__10494 invoke "warn_on_reflection.clj" 58]
  [clojure.lang.MultiFn invoke "MultiFn.java" 229]
  [clojure.lang.Var invoke "Var.java" 379]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8436 invoke "passes.clj" 166]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.tools.analyzer.passes$compile_passes$fn__8431$fn__8438 invoke "passes.clj" 168]
  [clojure.core$partial$fn__4529 invoke "core.clj" 2500]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk$walk__8336$walk__8337 invoke "ast.clj" 96]
  [clojure.tools.analyzer.ast$_update_children$fn__8327 invoke "ast.clj" 51]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 333]
  [clojure.core$reduce invoke "core.clj" 6518]
  [clojure.tools.analyzer.ast$_update_children invoke "ast.clj" 49]
  [clojure.tools.analyzer.ast$update_children_reduced invoke "ast.clj" 64]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk$walk__8336$walk__8337 invoke "ast.clj" 96]
  [clojure.tools.analyzer.utils$mapv_SINGLEQUOTE_ invoke "utils.clj" 208]
  [clojure.tools.analyzer.ast$_update_children$fn__8327 invoke "ast.clj" 51]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 333]
  [clojure.core$reduce invoke "core.clj" 6518]
  [clojure.tools.analyzer.ast$_update_children invoke "ast.clj" 49]
  [clojure.tools.analyzer.ast$update_children_reduced invoke "ast.clj" 64]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk$walk__8336$walk__8337 invoke "ast.clj" 96]
  [clojure.tools.analyzer.ast$_update_children$fn__8327 invoke "ast.clj" 51]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 333]
  [clojure.core$reduce invoke "core.clj" 6518]
  [clojure.tools.analyzer.ast$_update_children invoke "ast.clj" 49]
  [clojure.tools.analyzer.ast$update_children_reduced invoke "ast.clj" 64]
  [clojure.tools.analyzer.ast$walk$walk__8336 invoke "ast.clj" 99]
  [clojure.tools.analyzer.ast$walk invoke "ast.clj" 95]
  [clojure.tools.analyzer.ast$walk invoke "ast.clj" 92]
  [clojure.tools.analyzer.ast$prewalk invoke "ast.clj" 108]
  [clojure.tools.analyzer.passes$compile_passes$analyze__8443 invoke "passes.clj" 170]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.core$comp$fn__4495 invoke "core.clj" 2438]
  [clojure.tools.analyzer.jvm$analyze$fn__11679 invoke "jvm.clj" 469]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invoke "core.clj" 630]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1868]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.tools.analyzer.jvm$analyze invoke "jvm.clj" 456]
  [clojure.core.async.impl.ioc_macros$state_machine invoke "ioc_macros.clj" 1109]
  [clojure.core.async$go doInvoke "async.clj" 413]
  [clojure.lang.RestFn invoke "RestFn.java" 442]
  [clojure.lang.Var invoke "Var.java" 388]
  [clojure.lang.AFn applyToHelper "AFn.java" 160]
  [clojure.lang.Var applyTo "Var.java" 700]
  [clojure.lang.Compiler macroexpand1 "Compiler.java" 6631]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6709]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyze "Compiler.java" 6485]
  [clojure.lang.Compiler$BodyExpr$Parser parse "Compiler.java" 5861]
  [clojure.lang.Compiler$LetExpr$Parser parse "Compiler.java" 6179]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6723]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6711]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyze "Compiler.java" 6485]
  [clojure.lang.Compiler$BodyExpr$Parser parse "Compiler.java" 5861]
  [clojure.lang.Compiler$FnMethod parse "Compiler.java" 5296]
  [clojure.lang.Compiler$FnExpr parse "Compiler.java" 3925]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6721]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6711]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler access$300 "Compiler.java" 38]
  [clojure.lang.Compiler$DefExpr$Parser parse "Compiler.java" 577]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 6723]
  [clojure.lang.Compiler analyze "Compiler.java" 6524]
  [clojure.lang.Compiler analyze "Compiler.java" 6485]
  [clojure.lang.Compiler eval "Compiler.java" 6786]
  [clojure.lang.Compiler load "Compiler.java" 7227]
  [clojure.lang.RT loadResourceScript "RT.java" 371]
  [clojure.lang.RT loadResourceScript "RT.java" 362]
  [clojure.lang.RT load "RT.java" 446]
  [clojure.lang.RT load "RT.java" 412]
  [clojure.core$load$fn__5448 invoke "core.clj" 5866]
  [clojure.core$load doInvoke "core.clj" 5865]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [clojure.core$load_one invoke "core.clj" 5671]
  [clojure.core$load_all$fn__5389$fn__5392 invoke "core.clj" 5688]
  [clojure.core$load_all$fn__5389 invoke "core.clj" 5687]
  [clojure.lang.AFn call "AFn.java" 18]
  [clojure.lang.LockingTransaction run "LockingTransaction.java" 273]
  [clojure.lang.LockingTransaction runInTransaction "LockingTransaction.java" 229]
  [clojure.core$load_all invoke "core.clj" 5685]
  [clojure.core$load_lib$fn__5397 invoke "core.clj" 5711]
  [clojure.core$load_lib doInvoke "core.clj" 5710]
  [clojure.lang.RestFn applyTo "RestFn.java" 142]
  [clojure.core$apply invoke "core.clj" 632]
  [clojure.core$load_libs doInvoke "core.clj" 5749]
  [clojure.lang.RestFn applyTo "RestFn.java" 137]
  [clojure.core$apply invoke "core.clj" 632]
  [clojure.core$require doInvoke "core.clj" 5832]
  [clojure.lang.RestFn invoke "RestFn.java" 421]
  [user$eval7383 invoke "NO_SOURCE_FILE" 2]
  [clojure.lang.Compiler eval "Compiler.java" 6782]
  [clojure.lang.Compiler eval "Compiler.java" 6745]
  [clojure.core$eval invoke "core.clj" 3081]
  [clojure.main$repl$read_eval_print__7099$fn__7102 invoke "main.clj" 240]
  [clojure.main$repl$read_eval_print__7099 invoke "main.clj" 240]
  [clojure.main$repl$fn__7108 invoke "main.clj" 258]
  [clojure.main$repl doInvoke "main.clj" 258]
  [clojure.lang.RestFn invoke "RestFn.java" 421]
  [clojure.main$repl_opt invoke "main.clj" 324]
  [clojure.main$main doInvoke "main.clj" 422]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [clojure.lang.Var invoke "Var.java" 375]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.Var applyTo "Var.java" 700]
  [clojure.main main "main.java" 37]]}
user=>


 Comments   
Comment by Alex Miller [ 22/Dec/15 4:09 PM ]

dupe of and fixed by ASYNC-152

Comment by Nicola Mometto [ 22/Dec/15 6:00 PM ]

Just for the record, this is neither a core.async nor a tools.analyzer bug.
It turns out that re-evaluating a defmulti expression completely nukes any metadata on that defmulti, breaking tools.analyzer's scheduler since it relies on that metadata to order the passes.

Comment by Nicola Mometto [ 22/Dec/15 6:08 PM ]

Alex Miller I'm reopening this, pending an assessment on CLJ-1870.
While it is true that the ASYNC-154 patch fixed this specific NPE, `(require :reload-all)` will still break things since as explained in the ticked description of CLJ-1870, t.a passess will be ordered randomly





[ASYNC-90] Pub/sub leaks memory Created: 13/Sep/14  Updated: 22/Dec/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ziyang Hu Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: None
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]

java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) (reproducible on OpenJDK 7 as well)

OS X 10.9.4 (reproducible on Ubuntu Linux 14.04 as well)


Approval: Triaged

 Description   

The following code will cause OOME:

(require '[clojure.core.async :refer [chan close! <! pub sub unsub go timeout]])

(def p* (chan))

(def p (pub p* :topic))

(go
  (loop []
    (let [s (chan)
          t (rand-int Integer/MAX_VALUE)]
      (sub p t s)
      (<! (timeout 10))
      (unsub p t s)
      (close! s)
      (recur))))

(It grows slowly: to see the OOME in a reasonable amount of time, either give JVM very small
memory like 64m, or remove the timeout step.)

I tried to profile the code, and the reason seems to be that even though I
unsubed the channel from the port, something is still retained which causes
the heap to be used up.



 Comments   
Comment by Ziyang Hu [ 13/Sep/14 8:50 AM ]

Here is the problem:

https://github.com/clojure/core.async/blob/96de9a47ac511d9bb4309645a3bc594a2fc0c33a/src/main/clojure/clojure/core/async.clj#L826-L828

When unsub* is called, it just untaps the channel from the mult specified by the topic. The mult still remains in the atom called mults even if the mult has no taps left.

I can't think of a clean fix for this problem, since currently the channels which are tapping a mult aren't exposed, i.e., we currently have no way of knowing if a mult has any taps on it.

Comment by Ghadi Shayban [ 15/Oct/14 11:27 PM ]

I also cannot think of a clean fix, as mults do not expose their registrants. (The notion of "current" is a concurrent system is subtle)
Besides saying (and perhaps amending the docstring) that pubs are indeed resources, their footprint grows by the # of seen topics, and that resources should almost always be bounded.

Comment by Daniel Compton [ 11/Oct/15 5:02 PM ]

This bit us too in https://github.com/apa512/clj-rethinkdb/issues/97. My suggestion for a docstring addition is:

pubs are resources, and their footprint grows by the number of seen topics. unpub does not reclaim this resource. You should not use pub to subscribe to an unbounded number of topics.

Are there any changes/suggestions? Would you like me to create a patch for this?

Comment by Daniel Compton [ 11/Oct/15 5:04 PM ]

Also, is a pub able to be garbage collected when it's no longer used? When I was doing some limited testing, it didn't seem like it was, but I may be wrong.

Comment by Daniel Compton [ 15/Oct/15 4:45 PM ]

A partial workaround (I think?) is to use

(unsub-all pub topic)
if you know you don't need a topic anymore.





[ASYNC-123] Channel operations fail in for comprehension. Created: 25/May/15  Updated: 22/Dec/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Christian Weilbach Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]



 Description   

``` clojure
(<!! (go (for [a (<! (go [1 2 3]))]
a)))
;=> (1 2 3)
```

works fine. But:

```
(go (for [a [1 2 3]
:let [b (<! (go a))]]
b))
```

1. Caused by java.lang.IllegalArgumentException
No method in multimethod '-item-to-ssa' for dispatch value: :fn

MultiFn.java: 160 clojure.lang.MultiFn/getFn
MultiFn.java: 227 clojure.lang.MultiFn/invoke
ioc_macros.clj: 492 clojure.core.async.impl.ioc-macros/item-to-ssa
ioc_macros.clj: 547 clojure.core.async.impl.ioc-macros/let-binding-to-ssa/fn
ioc_macros.clj: 122 clojure.core.async.impl.ioc-macros/all/fn/fn
ArrayChunk.java: 58 clojure.lang.ArrayChunk/reduce
protocols.clj: 98 clojure.core.protocols/fn
protocols.clj: 19 clojure.core.protocols/fn/G
protocols.clj: 31 clojure.core.protocols/seq-reduce
protocols.clj: 54 clojure.core.protocols/fn
protocols.clj: 13 clojure.core.protocols/fn/G
core.clj: 6289 clojure.core/reduce
ioc_macros.clj: 120 clojure.core.async.impl.ioc-macros/all/fn
ioc_macros.clj: 554 clojure.core.async.impl.ioc-macros/eval22522/fn/fn
ioc_macros.clj: 600 clojure.core.async.impl.ioc-macros/eval22585/fn/fn
ioc_macros.clj: 806 clojure.core.async.impl.ioc-macros/parse-to-state-machine/fn
ioc_macros.clj: 80 clojure.core.async.impl.ioc-macros/get-plan
ioc_macros.clj: 802 clojure.core.async.impl.ioc-macros/parse-to-state-machine
ioc_macros.clj: 1066 clojure.core.async.impl.ioc-macros/state-machine
async.clj: 384 clojure.core.async/go
RestFn.java: 442 clojure.lang.RestFn/invoke
Var.java: 388 clojure.lang.Var/invoke
AFn.java: 160 clojure.lang.AFn/applyToHelper
Var.java: 700 clojure.lang.Var/applyTo
Compiler.java: 6552 clojure.lang.Compiler/macroexpand1
Compiler.java: 6630 clojure.lang.Compiler/analyzeSeq
Compiler.java: 6445 clojure.lang.Compiler/analyze
Compiler.java: 6406 clojure.lang.Compiler/analyze
Compiler.java: 3719 clojure.lang.Compiler$InvokeExpr/parse
Compiler.java: 6646 clojure.lang.Compiler/analyzeSeq
Compiler.java: 6445 clojure.lang.Compiler/analyze
Compiler.java: 6406 clojure.lang.Compiler/analyze
Compiler.java: 5782 clojure.lang.Compiler$BodyExpr$Parser/parse
Compiler.java: 5217 clojure.lang.Compiler$FnMethod/parse
Compiler.java: 3846 clojure.lang.Compiler$FnExpr/parse
Compiler.java: 6642 clojure.lang.Compiler/analyzeSeq
Compiler.java: 6445 clojure.lang.Compiler/analyze
Compiler.java: 6700 clojure.lang.Compiler/eval
Compiler.java: 7130 clojure.lang.Compiler/load
REPL: 1 geschichte.p2p.hooks/eval32348
Compiler.java: 6703 clojure.lang.Compiler/eval
Compiler.java: 6666 clojure.lang.Compiler/eval
core.clj: 2927 clojure.core/eval
main.clj: 239 clojure.main/repl/read-eval-print/fn
main.clj: 239 clojure.main/repl/read-eval-print
main.clj: 257 clojure.main/repl/fn
main.clj: 257 clojure.main/repl
RestFn.java: 1523 clojure.lang.RestFn/invoke
interruptible_eval.clj: 67 clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn
AFn.java: 152 clojure.lang.AFn/applyToHelper
AFn.java: 144 clojure.lang.AFn/applyTo
core.clj: 624 clojure.core/apply
core.clj: 1862 clojure.core/with-bindings*
RestFn.java: 425 clojure.lang.RestFn/invoke
interruptible_eval.clj: 51 clojure.tools.nrepl.middleware.interruptible-eval/evaluate
interruptible_eval.clj: 183 clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
interruptible_eval.clj: 152 clojure.tools.nrepl.middleware.interruptible-eval/run-next/fn
AFn.java: 22 clojure.lang.AFn/run
ThreadPoolExecutor.java: 1145 java.util.concurrent.ThreadPoolExecutor/runWorker
ThreadPoolExecutor.java: 615 java.util.concurrent.ThreadPoolExecutor$Worker/run
Thread.java: 745 java.lang.Thread/run

Possibly this is due to the function boundaries of go-blocks, but I think it would be nice if this would work similar to the binding for `a`. I have basically one big for-comprehension to map a highly nested seq and rewriting it will make the code much more verbose. Note that I know that I cannot execute channel ops inside the body of the for-comprehension due to the function boundaries in lazy seqs. This is no problem, but I cannot fix the for-comprehension bindings easily.
I played around with extending the -item-to-ssa multimethod, but some pointers of how to fix it properly would be nice. Then I could try to implement a patch.



 Comments   
Comment by Christian Weilbach [ 14/Jun/15 9:11 AM ]

This is actually a problem for many more cases, only a take operation in the first binding works, because the for-comprehension macro in core.clj wraps every following binding in a lazy-seq. The best fix to me seemed to be a custom version of the for-comprehension macro.

I have simplified the macro to remove lazyness and rewritten it with go-loops. This works fairly well, but it is eager and channel operations are sideeffects, so channels representing constants on which iteration happens repeatedly should be put in an immutable collection before (e.g. let binding around the comprehension).

https://github.com/ghubber/geschichte/blob/c57bacdcf8e3335d17dc370941384c559ae80373/src/cljx/geschichte/go_for.cljx#L25

Comment by Christian Weilbach [ 18/Oct/15 12:40 PM ]

Can be closed, is fixed with go-for for me. The irritation came from the first binding working as a channel op, while nested ones are wrapped in lazy-seq fns.





[ASYNC-122] Parking in finally block replaces result Created: 06/May/15  Updated: 22/Dec/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Vesa Karvonen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojurescript "0.0-3211"]
[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]


Approval: Triaged

 Description   

Because

(try 1 (finally 2))

evaluates to 1, I would expect

(go (println (try 1 (finally (<! (go 2))))))

to print 1, but it prints 2 in both CLJ and CLJS. This can be worked around by replacing try-finally with try-catch.



 Comments   
Comment by Nicola Mometto [ 17/Dec/15 10:06 AM ]

This appears to be a bug caused by nesting `go` blocks





[ASYNC-152] Reflection warning when binding received values to names. Created: 20/Nov/15  Updated: 22/Dec/15  Resolved: 22/Dec/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Marcus Crestani Assignee: Nicola Mometto
Resolution: Completed Votes: 0
Labels: None
Environment:

OS X 10.11.1
java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
[org.clojure/clojure "1.7.0"]
[org.clojure/core.async "0.2.374"]


Attachments: Text File 0001-ASYNC-152-disable-t.a.jvm-s-warn-on-reflection-pass.patch    
Approval: Vetted

 Description   

After upgrading core.async in our project to 0.2.374 (we came
from 0.1.346.0-17112a-alpha), we now see many reflection warnings
that we did not see with the old version.

Here is a simple example code to reproduce the typical reason for
the warning:

(ns dummy
  (:require [clojure.core.async :as async]))

(defn dummy-1
  [ch]
  (async/go-loop []
    (let [msg (async/<! ch)]
      (if (= :close msg)
        (async/close! ch)
        (recur)))))

We see:

Reflection warning: dummy.clj:8:11 - call to static method equiv on clojure.lang.Util cannot be resolved

The warning is about (= :close msg).

Note that there is no reflection warning when we are not binding
the received value to a name, i.e. (= :close (async/<! ch))
works.

Why do we see these reflection warnings after upgrading?
How can we fix these warning?

Patch:
0001-ASYNC-152-disable-t.a.jvm-s-warn-on-reflection-pass.patch



 Comments   
Comment by Alex Miller [ 20/Nov/15 8:24 AM ]

I have seen some of these as well and I suspect it's most likely due to the more recent tools.analyzer versions. I think there are some useful notes here too, thanks.

I do not currently have any suggestion on how to stop these from happening.

Comment by Nicola Mometto [ 15/Dec/15 5:55 PM ]

Minimal case:

(fn [ch] (go (let* [msg (<! ch)] (= :foo msg)))
Comment by Nicola Mometto [ 15/Dec/15 6:02 PM ]

tools.analyzer.jvm's method matcher seems to be bugged, the reflection warning comes from t.a rather than from clojure, clojure doesn't actually require reflection for this.

This patch simply disables tools.analyzer's warn-on-reflection pass, since the clojure compiler will warn itself if something will reflect

Comment by Alex Miller [ 22/Dec/15 3:43 PM ]

applied





[ASYNC-32] onto-chan retains head of input sequence causing OutOfMemoryError Created: 06/Nov/13  Updated: 15/Dec/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Brian Lubeski Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: memory
Environment:

org.clojure/core.async 0.1.242.0-44b1e3-alpha


Attachments: File patch.clj    
Patch: Code
Approval: Triaged

 Description   
(require '[clojure.core.async :as a])
(let [c (a/chan)]
  (a/onto-chan c (iterate inc 0))
  (while true
    (a/<!! c)))

onto-chan is holding on to the head of the input sequence as it is unfolded, resulting in an (eventual) OutOfMemoryError.

I've attached a diff showing changes I made to onto-chan that fixed the problem for me.



 Comments   
Comment by Colin Taylor [ 07/Oct/14 7:13 PM ]

Just to note, to-chan uses onto-chan so is similarly affected.
We ran into this, Brian's patch worked fine.

Comment by Alex Miller [ 10/Aug/15 3:22 PM ]

This seems like a viable change, but the patch needs a better test (even if not in the patch) and to be properly formatted for git apply (per http://dev.clojure.org/display/community/Developing+Patches).

Comment by Brian Lubeski [ 11/Aug/15 12:00 AM ]

This issue might be caused by ASYNC-138.

Comment by Nicola Mometto [ 15/Dec/15 6:12 PM ]

The fix for ASYNC-138 should fix this issue aswell and is more general





[ASYNC-153] Return value of pipe is not documented Created: 29/Nov/15  Updated: 29/Nov/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Viktor Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring

Approval: Triaged

 Description   

pipe explicitly returns its "to" channel, but it's not documented.

If that's how it should work, this should be documented.






[ASYNC-151] Accept optional transducer in `to-chan` Created: 18/Nov/15  Updated: 18/Nov/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Karlis Lauva Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File to-chan-xducer.patch    
Patch: Code and Test

 Description   

Allow `clojure.core.async/to-chan` accept an optional transducer that gets passed to the newly created channel.
Why? With transducer passed directly in `to-chan` there is no need to use `pipe` or `onto-chan` with an explicitly created a channel that has transducer attached.

This is my first patch for a Clojure project. Please let me know if something was wrong / missing when creating the ticket.






[ASYNC-99] go block with <! inside protocol method invocation fails to compile (dispatch :protocol-invoke on '-item-to-ssa') Created: 23/Oct/14  Updated: 11/Nov/15  Resolved: 11/Nov/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Valentin Waeselynck Assignee: Unassigned
Resolution: Not Reproducible Votes: 4
Labels: go-macro, protocols
Environment:

[org.clojure/core.async "0.1.346.0-17112a-alpha"]
happened on both [org.clojure/clojure "1.7.0-alpha1"] and [org.clojure/clojure "1.6.0"]

java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)



 Description   

I was programming a function involving a core.async go block, when I stumbled on a strange compilation error :

CompilerException java.lang.IllegalArgumentException: No method in multimethod '-item-to-ssa' for dispatch value: :protocol-invoke, compiling:(NO_SOURCE_PATH:2:3)

I experimented a little to try and strip down the problem, and found it was very generic. Say I have any protocol MyProtocol :

(defprotocol MyProtocol
  (do-something [this param] "some method"))

The following code will not compile, failing with the exception I showed you above :

(defn uncompilable! [me ch] 
  (go 
    (do-something me (<! ch)) ;; apparently, it hurts to use <! in a protocol method invocation 
    ))

However, the following 2 will compile without any problem :

(defn compilable! [me ch] 
  (let [call-it #(do-something me %)] ; wrapping the protocol call in a function
    (go 
     (call-it (<! ch))
     )))

(defn compilable-2! [me ch] 
  (go 
    (let [my-value (<! ch)] ; taking out the <! call
      (do-something me my-value))
    ))

It seems to me the '<! inside protocol method invocation form' is a situation which the go macro fails to handle.



 Comments   
Comment by Alex Miller [ 11/Nov/15 5:13 PM ]

NR in latest core.async 0.2.374.





[ASYNC-149] Cannot compile a recur in a case statement when in a go-loop with a <! Created: 29/Oct/15  Updated: 11/Nov/15  Resolved: 11/Nov/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Josh Comer Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

When compiling the following code causes a Caused by: java.lang.UnsupportedOperationException: Can only recur from tail position exception to be thrown (from the 1 line of the case):

(defn foo
  [x]
  (let [c (a/chan)]
    (a/go-loop [o 1]
      (when-let [r (a/<! c)]
        (case o
          1 (recur 2)
          2 (recur 3)
          3 false)))))

Interestingly this code does not create the exception:

(defn foo-2
  [x]
  (let [c (a/chan)]
    (a/go-loop [o 1]
      (when-let [r true]
        (case o
          1 (recur 2)
          2 (recur 3)
          3 false)))))

The error causing code compiles fine using 0.1.346.0-17112a-alpha. My first guess would be this has to do with the updating of tools.analyzer.jvm



 Comments   
Comment by Alex Miller [ 11/Nov/15 4:19 PM ]

Fixed by upgrade to latest tools.analyzer.jvm in https://github.com/clojure/core.async/commit/f01c280b68f0aa013e6aa08e08f25354f78ae343





[ASYNC-140] transducer not applied with unbuffered channels Created: 17/Aug/15  Updated: 04/Nov/15  Resolved: 18/Aug/15

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Alex Miller Assignee: Alex Miller
Resolution: Declined Votes: 0
Labels: None

Approval: Triaged

 Description   

Found this while testing some stuff on ASYNC-124.

(require '[clojure.core.async :as a :refer (>!! <!! chan onto-chan)])
(def c (chan 1 (mapcat #(range %))))
(onto-chan c [1 2 3])
(<!! (a/reduce conj [] c))
;; [0 0 1 0 1 2]

(def c (chan 0 (mapcat #(range %))))
(onto-chan c [1 2 3])
(<!! (a/reduce conj [] c))
;; [1 2 3]

Cause: For an unbuffered channel, a take will try to pair up with a put, in which case the two are satisfied directly without ever involving the buffer (since there isn't one). Because of this, add! is never called and the xf will not be applied. The example above uses an expanding transducer, but problem exists for any transducer and could perhaps happen even on buffered channels?

Approach: one approach would be to not satisfy the put unless there were enough pending takes to satisfy all the transduced values, which seems consistent with an unbuffered channel.



 Comments   
Comment by Alex Miller [ 17/Aug/15 5:13 PM ]

I suspect the problem is here:
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/impl/channels.clj#L115-L133

where a take can't get a value from the buffer (there is none) and instead satisfies itself by matching up directly with a put. In this case, add! is never called so the transducer is not applied.

In the case of an expanding transducer, a buffer is needed at this point where none exists.

Comment by Alex Miller [ 18/Aug/15 8:17 AM ]

I'm a dope and unbuffered channels (or 0-sized fixed buffer) channels do not support transducers per the chan docstring.

Comment by Alex Nixon [ 04/Nov/15 5:10 PM ]

I vote to reopen this.

As a user this feels like a very subtle bug - the docstring says "if a transducer is supplied, a buffer must be specified". In your example you have specified a buffer – one of size zero – and so it's very surprising when you find out (at some later stage in your program) that your transducer was silently ignored. If a fixed buffer of size zero is not a valid thing to construct then I would expect (buffer 0) to throw an IllegalArgumentException or similar.

And philosophically I don't understand why the combination of an unbuffered channel and a transducer should be forbidden. What's the argument here?

Comment by Alex Miller [ 04/Nov/15 9:17 PM ]

After talking to Rich, it is considered an error to created a fixed buffer of size 0 and that should throw. The ticket ASYNC-143 covers that check.

On the last question, a transducer needs to be (potentially) be able to produce multiple values with something like a mapcat transducer. Without a buffer, there is no place for those buffers to go, so an unbuffered channel is insufficient.





[ASYNC-150] Unordered pipeline Created: 01/Nov/15  Updated: 01/Nov/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Corin Lawson Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I have a use case for clojure.core.async/pipeline whereby the order of the outputs is unimportant and it is desirable to have the nth output present on the to channel without having to wait for the first output to be ready.

I don't have a great deal of experience with clojure; I'm wondering if this is something that belongs in the core library or whether there's some idiomatic way of achieving an unordered pipeline without rewriting the pipeline function. In any case, I have made an attempt at rewriting pipeline, and learnt a thing or two in the process

Obviously, this code (below) cannot be merged as it completely breaks the existing behaviour (not to mention I've barely tested it), but I would be appreciative if you could provide some feedback and/or consider adding this functionality. Perhaps a new function, unordered-pipeline?

(defn- pipeline*
  ([n to xf from close? ex-handler type]
     (assert (pos? n))
     (let [ex-handler (or ex-handler (fn [ex]
                                       (-> (Thread/currentThread)
                                           .getUncaughtExceptionHandler
                                           (.uncaughtException (Thread/currentThread) ex))
                                       nil))
           jobs (chan n)
           results (chan n)
           process (fn [[v p :as job]]
                     (if (nil? job)
                       (comment closing results here would be too early)
                       (let [res (chan 1 xf ex-handler)]
                         (>!! res v)
                         (close! res)
                         (put! p res)
                         true)))
           async (fn [[v p :as job]]
                   (if (nil? job)
                     (comment closing results here would be too early)
                     (let [res (chan 1)]
                       (xf v res)
                       (put! p res)
                       true)))]
       (go-loop []
                (let [v (<! from)]
                  (if (nil? v)
                    (close! jobs)
                    (do
                      (comment removed the indirection and pass results as part of the job)
                      (>! jobs [v results])
                      (recur)))))
       (go-loop []
                (let [res (<! results)]
                  (if (nil? res)
                    (when close? (close! to))
                    (do (loop []
                          (let [v (<! res)]
                            (when (and (not (nil? v)) (>! to v))
                              (recur))))
                        (recur)))))
       (go
         (comment ensure results is closed after all jobs have been processed)
         (<! (async/merge
               (map #(case %
                       :blocking (thread
                                   (let [job (<!! jobs)]
                                     (when (process job)
                                       (recur))))
                       :compute (go-loop []
                                         (let [job (<! jobs)]
                                           (when (process job)
                                             (recur))))
                       :async (go-loop []
                                       (let [job (<! jobs)]
                                         (when (async job)
                                           (recur)))))
                    (repeat n type))))
         (close! results)))))


 Comments   
Comment by Corin Lawson [ 01/Nov/15 1:51 AM ]

I've only tried the :compute scenario...

(snooze [] (Thread/sleep (+ 5 (rand-int 10))))
(defn slowly [f] (fn
  ([] (snooze) (let [out (f)] (snooze) out)) 
  ([r] (snooze) (let [out (f r)] (snooze) out))
  ([r i] (snooze) (let [out (f r i)] (snooze) out))))
(def cout (chan))
(pipeline 3 cout (comp slowly (map inc)) (async/to-chan (range 18)))
(<!! (async/into [] cout))
-> [1 3 2 4 5 6 8 9 7 10 12 11 13 14 15 16 17 18]

Each group of 3 (n) is outputted in a random order





[ASYNC-124] put! into channel that has mapcat transducer fails to dispatch more than one waiting takes Created: 30/May/15  Updated: 28/Oct/15  Resolved: 28/Oct/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Critical
Reporter: Jozef Wagner Assignee: Alex Miller
Resolution: Completed Votes: 1
Labels: None
Environment:

Clojure 1.7.0-RC1, latest core.async head (May 2015)


Attachments: Text File async-124-2.patch     Text File async-124-3.patch     Text File async-124-4.patch     Text File async-124-5.patch     Text File async-124-6.patch     Text File async-124-7.patch     Text File async-124-8.patch     Text File async-124.patch    
Patch: Code and Test
Approval: Ok

 Description   

This issue is similar to one that has caused the revert of ASYNC-103. When channel has multiple pending takes, one put will dispatch only one take, even if transducer attached to the channel produces multiple values.

(require '[clojure.core.async :refer (chan put! take!)])
(let [xf (mapcat #(range % (+ % 5)))
        c (chan 10 xf)]
    (dotimes [x 5]
      (take! c #(println (str x %))))
    (put! c 10)
    (put! c 15))

The output of the above code is

0 10
1 11

but it should be

0 10
1 11
2 12
3 13
4 14

Note that if we change the order and execute puts before takes, the result will be as expected.

Cause: When executing a write on a channel, no more than one pending take will be satisfied. Other pending takes will wait until the next put. When a put could release at most one take, this was sufficient but with expanding transducers, a put can potentially satisfy many pending takers.

Approach: Once a put succeeds into the buffer (through the transducer), instead of finding a single taker, do the prior work in a loop, pairing up callbacks with values removed from the buffer. Once the loop is complete, release the mutex and dispatch all paired takes.

Patch: async-124-8.patch

Screened by:



 Comments   
Comment by Jozef Wagner [ 30/May/15 10:30 AM ]

patch with test

Comment by Jozef Wagner [ 30/May/15 10:32 AM ]

Note that with attached patch the actual dispatch/run on takes happens before (.unlock mutex) and before (when done? (abort this)) are called.

Also note that added test only works with 1.7.0 version of clojure

Comment by Jozef Wagner [ 30/May/15 12:46 PM ]

Added new patch async-124-2.patch that dispatches after unlocking and aborting. Test was also changed as it had race condition.

Comment by Alex Miller [ 06/Jul/15 9:23 AM ]

We can't add a test that is 1.7-only. See https://github.com/clojure/core.async/blob/master/src/test/clojure/clojure/core/pipeline_test.clj#L7 for an example of how to patch in a transducer impl that will work in pre-1.7.

Comment by Jozef Wagner [ 06/Jul/15 10:41 AM ]

Added async-124-3.patch that works with Clojure <1.7

Comment by Alex Miller [ 10/Aug/15 3:53 PM ]

Jozef, I don't see any reason why the independent callbacks in your test will necessarily conj in order as they contend for the atom. When I run the test I usually get a failure. If you agree, could wrap expected and actual in a set. Or if that seems wrong, then let's decide why.

Comment by Jozef Wagner [ 11/Aug/15 10:49 AM ]

Attached async-124-4.patch that uses unordered comparison in test

Comment by Alex Miller [ 14/Aug/15 1:01 PM ]

Added async-124-5.patch which should be semantically identical as -4 but retains the structure of a single dispatch point.

Comment by Alex Miller [ 14/Aug/15 3:03 PM ]

New -6 patch tries to retain more of the original algorithm and leverages the original loop instead of adding a new one.

Comment by Alex Miller [ 18/Aug/15 8:25 AM ]

New -7 patch addresses some bugs in the prior version and cleans up the loop code, also adds better tests.

Comment by Alex Miller [ 28/Aug/15 11:53 AM ]

test tweaks added in -8, per Stuart Sierra

Comment by Stuart Sierra [ 28/Aug/15 12:01 PM ]

Screened. Clojure version good. Still needs corresponding fix to ClojureScript version.





[ASYNC-103] promise-chan Created: 05/Nov/14  Updated: 28/Oct/15  Resolved: 28/Oct/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Alex Miller
Resolution: Completed Votes: 3
Labels: None

Attachments: Text File async-103-2.patch     Text File async-103-3.patch     Text File async-103-4.patch     Text File async-103-5.patch     Text File async-103-6.patch     Text File async-103-cljs.patch     Text File async-103.patch    
Patch: Code and Test
Approval: Ok
Waiting On: Alex Miller

 Description   

A promise-chan is a new kind of core.async channel that gives promise semantics to a channel.

  • Takes block prior to put or close!
  • After a put, all pending takes are notified, and any subsequent takes return the put value immediately
  • After a close (and no put), all pending takes are notified with nil, and any subsequent takes return nil immediately

In practice, a promise-chan is a normal channel that uses a promise-buf:

  • buf can only take on one value (or nil if closed) ever
  • buf is never emptied: N consumers can all get the same, one and only value by reading from the channel

In order to make close! sensible with this new kind of buf

  • all buffers need to be closeable
  • closing a channel closes its buffer
  • all basic buffer types do nothing when closed
  • a promise-buf make a one-time transition to having no value when closed (if not previously delivered)
  • closing a buffer is implementation detail not exposed to end users

Approach:

  • Buffer protocol now has a close-buf! function (close! would have overlapped the Channel function close!). close-buf! is invoked on a buffer when the channel is closed, allowing it to update itself if necessary.
  • Existing buffers implement close-buf! and do nothing (buffer still available for taking)
  • New promise-buffer implementation. Makes a one-time transition when value is supplied or buffer is closed. value is held in an unsynchronized-mutable deftype field - updates via add!* or close-buf! always happen under the containing channel mutex.
  • New api functions: promise-chan creates a channel with a promise-buffer and promise-buffer.

Patch: async-103-6.patch, async-103-cljs.patch (catches up cljs to diffs in async-103-6)



 Comments   
Comment by Ghadi Shayban [ 07/Nov/14 4:06 PM ]

My initial gut reaction was that this is more related to semantics of the channel, not the buffer, and I'm wary of significant changes to esp. the impl protocols. But after seeing an impl, it looks straightforward and the changes aren't too significant. (Another possibility is to make another simpler implementation of a channel, with just slots for the value, lock and pending takers before the value is delivered. No slots for xfn or putters or buffer handling would be needed.)

Note an atom is not needed in the PromiseBuffer, just a set! on a mutable field to be inline with the other buffer flavors. If the patch continues using an atom, maybe guard val to not swap unnecessarily.

Comment by Alex Miller [ 07/Nov/14 11:09 PM ]

Good call on the atom - backed off to just using existing clojure.lang.Box. If that's too weird, will just deftype it.

Comment by Alex Miller [ 07/Nov/14 11:18 PM ]

Dur, just need the val field itself.

Comment by Fogus [ 09/Jan/15 4:27 PM ]

I'd really love to see the reason for the current impl over a more pointed promise channel (perhaps as described by Ghadi). This is a clear implementation, but with the addition of close-buf! some complexity is added for implementations in the future. Specifically, I'd like to at least see a dcostring in the protocol describing the meaning and desired semantics around close-buf! and what it should return and when. Like I said, this is a quality patch, but I'm just concerned that there are unstated assumptions around close-buf! that escape me.

Comment by Alex Miller [ 28/Jan/15 12:43 PM ]

Fogus, I looked into implementing the promise-chan a bit today but it requires replicating a significant portion of the existing chan implementation. I believe that's the primary reason to do it this way, just leveraging the majority chan code.

New -5 patch has two changes - I commented the Buffer protocol fns, and I removed promise-buffer from the api, as I think users should only use promise-chan.

Comment by Fogus [ 30/Jan/15 10:06 AM ]

The patch looks good. My only (minor) reservation is that the Buffer docstrings are written under the assumption that people will use instances only in the context of a guarded channel. I understand why of course, so I think I might be too pedantic here. Because of this I see no reason not to +1 this patch.

Comment by Alex Miller [ 23/Feb/15 8:32 AM ]

Applied patch.

Comment by Alex Miller [ 03/Apr/15 3:57 PM ]

Reopening (and likely reverting commit).

This example demonstrates that only one of the waiting blocks will be notified, instead of all, as we'd like:

(let [c (promise-chan)] 
  ;; takers
  (dotimes [_ 3] 
    (go (println (<! c)))) 

  ;; put, should get to all takers
  (go (>! c 1)))
Comment by Jozef Wagner [ 30/May/15 10:35 AM ]

ASYNC-124 fixes the issue mentioned by Alex

Comment by Leon Grapenthin [ 11/Jun/15 11:44 AM ]

Quote: "buf is never emptied: N consumers can all get the same, one and only value by reading from the channel"

This is not true with the new closing semantics introduced here, correct? If I deref a Clojure promise, I get the same value everytime. But what is the purpose in closing promise-chans after the promise has been delivered?

When is a good time to close a promise-chan? If I close after the put, which is what many core.async processes do, takers must have taken before the close which means it will often be a race condition unless explicit care is taken.

Is there a benefit or am I simply missing something here?

Comment by Herwig Hochleitner [ 08/Jul/15 2:23 PM ]

Here is my cljs implementation of a promise channel: https://github.com/webnf/webnf/blob/master/cljs/src/webnf/promise.cljs
I'm the sole author and you are free to use any of it.

Comment by Alex Miller [ 12/Aug/15 3:00 PM ]

I believe that this patch can be re-evaluated after ASYNC-124 is applied as that patch addresses the reason this was retracted.

Comment by Leon Grapenthin [ 23/Aug/15 6:46 AM ]

It seems that there are two different approaches to how a promise-chan is supposed to or can be used.

1. All consumers are supposed to be takers before the promise is delivered. Taking after delivery is not intended, hence producers should close the channel after delivery.
2. " can take at any time and either have to wait for the promise (take blocks) or they get its value right away.

(2) models the behavior of real promises. (1) rather models that of a mutable reference.

It seems that the current patch explicitly makes room for (1) by giving producers the possibility to close.

Now if you want to use a promise-chan like (1) as a producer you can't create and return a promise-chan. Instead you need to be passed a promise-chan which has (a) taker(s) already. On the consumer side, it seems like an annoying quirk to enqueue takes before being able to pass the promise-chan to the producer (since he could close the promise immediately). I can't think of a scenario where one would want to do such thing. So my new question is:

Given the lengths taken here to enable (1), are there practical usecases for that? Why is closing a promise-chan not just going to be a no op?

Comment by Alex Miller [ 24/Aug/15 12:56 PM ]

Taking after delivery is intended. It is hard sometimes to know whether a take will happen before or after delivery in asynchronous systems so both must work. close! will deliver nil to all pending or future takers - in this case (as with all channels) nil indicates a drained channel, meaning that no delivery will ever happen.

However, one good question here is - what happens when you close a promise-chan that has already been delivered?

Currently:

  • (def pc (promise-chan))
  • take from promise-chan (blocks)
  • (>!! pc 5) - delivers 5 to blocked taker(s)
  • take from promise-chan (immediately returns 5) ;; for any number of takes
  • (close! pc 5)
  • take from promise-chan (immediately returns nil) ;; ???

I could easily be persuaded that the last answer is wrong and that it should always deliver 5 regardless of whether the pc gets closed (that is closing has no effect on delivered promise-chans).

Comment by Alex Miller [ 24/Aug/15 1:03 PM ]

I just confirmed with Rich that once delivered, a promise-chan should always deliver that value to takers, regardless of whether it is subsequently closed.

Comment by Stuart Sierra [ 28/Aug/15 4:02 PM ]

Screened. Clojure version good, assuming ASYNC-124 is applied first. ClojureScript version may or may not need additional work.

I agree that the addition of close-buf! and the no-op implementations feel like working around some missing abstraction. (What does it even mean to "close" a buffer?) But the Buffer protocol is not part of the public API, it's buried down in clojure.core.async.impl.protocols. The Channel and Buffer implementations are already tightly coupled, so adding another point of collaboration does not seem likely to cause problems. Adding close-buf! is certainly preferable to duplicating the implementation of ManyToManyChannel.





[ASYNC-119] ClojureScript: combine cljs.core.async.macros and cljs.core.async Created: 15/Mar/15  Updated: 14/Oct/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: J. S. Choi Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: enhancement


 Description   

The macros in cljs.core.async.macros can now be moved into the cljs.core.async namespace, now that combined .cljs/.clj namespaces are possible in ClojureScript. This, along with ClojureScript 0.0-2755's recent improvements, would enable users to use `(require [cljs.core.async :refer [go go-loop ...])` without having to distinguish between functions and macros, and without having to use `require-macros` at all.



 Comments   
Comment by J. S. Choi [ 14/Oct/15 9:19 PM ]

ASYNC-142 (http://dev.clojure.org/jira/browse/ASYNC-142) proposes renaming the ClojureScript namespace from cljs.core.async to clojure.core.async, which would supersede this improvement.

Personally, I would love for ASYNC-142 to happen, though if it ends up being held back for a major change across all clojure.core projects, as Alex Miller suggested on its ticket, then this improvement may still be worth doing in the meantime.





[ASYNC-6] alts! assumes nth works on ports argument Created: 15/Jul/13  Updated: 14/Oct/15  Resolved: 27/Sep/13

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Brandon Bloom Assignee: Timothy Baldridge
Resolution: Completed Votes: 0
Labels: None


 Description   

The docstring for alts! reads:

ports is a set of channel endpoints, which
can be either a channel to take from or a vector of
[channel-to-put-to val-to-put], in any combination.

However, trying to use an actual set yields the following exception:

java.lang.UnsupportedOperationException: nth not supported on this type: PersistentHashSet



 Comments   
Comment by Brandon Bloom [ 07/Aug/13 2:32 PM ]

Discussed at length in IRC on Aug 7th: http://www.raynes.me/logs/irc.freenode.net/clojure/2013-08-07.txt

Comment by Rich Hickey [ 07/Aug/13 9:28 PM ]

The documentation should be changed. alts requires an efficiently nth-able collection (e.g. a vector), else it will be slow.

https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L183

If CLJS supports nth on sets it is broken.

Comment by Brandon Bloom [ 07/Aug/13 9:52 PM ]

> alts requires an efficiently nth-able collection

Is that the case? Or does alts actually only require efficiently sampling a single element from a collection?

Efficient nth is a reasonable proxy, since the only choice we have for randomness is rand-int. However, one could imagine a CollSample protocol that would allow efficient random choice leveraging data-structure-specific knowledge, much as CollReduce works. Given the world as it exists now, I agree that this is a doc bug.

> If CLJS supports nth on sets it is broken.

That's the conclusion we came to in IRC. David quickly fixed this:
https://github.com/clojure/clojurescript/commit/a113b08a8c2811b0590cc6a36b2e9e5adc1c4c1e

Comment by Timothy Baldridge [ 27/Sep/13 8:47 AM ]

fixed in master

Comment by Daniel Compton [ 14/Oct/15 5:30 PM ]

For those reading this in the future, this was fixed by changing the docstring to use vector instead of set. https://github.com/clojure/core.async/commit/ca633479f51481f4aee40b4d60120484839d20c6





[ASYNC-142] Rename ClojureScript namespace from cljs.core.async to clojure.core.async Created: 18/Aug/15  Updated: 03/Oct/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None


 Description   

I'm not sure what the original reasoning for using separate namespaces for the Clojure and ClojureScript versions of core.async were, but now with Reader Conditionals people are able to write cross platform code that targets both at the same time. For that reason, it would be nice to only need to require clojure.core.async so that the require didn't need to be split into two platform specific requires for the same library.

The biggest downside to this is that it would break backwards compatibility. However core.async is still in alpha, and updating code to reference the new namespace should be simple and able to be done quickly. It may be possible to leave a shim namespace in place for some time to allow people to migrate their code.

Thoughts?



 Comments   
Comment by Alex Miller [ 18/Aug/15 8:36 AM ]

I think we should do this, but it will likely be in tandem with a bigger set of changes (build, moving away from Clojure 1.6, etc).

Comment by Thomas Heller [ 03/Oct/15 5:48 AM ]

The biggest problem here are macros. The CLJS implementation (and cljs.core.async.macros) is very different from the CLJ one. Given that CLJS still requires :require-macros to use macros I doubt we can come up with a solution that does NOT require Reader Conditionals in their NS form.

CLJS can easily do away with macros requiring the extra :include-macros or :refer-macros (instead of just :refer) though. David Nolen rejected my initial implementation but was open to addressing the issue somewhere else [1]. I never went to pursue this via JIRA but it is probably something that should be solved prior to renaming things from cljs.core.async.

[1] http://dev.clojure.org/jira/browse/CLJS-948?focusedCommentId=37666&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-37666





[ASYNC-148] Shorten core.async go block variable names Created: 28/Sep/15  Updated: 28/Sep/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: PNG File Screenshot of Google Chrome.png    

 Description   

When using core.async go blocks in ClojureScript, code is transformed into asynchronous functions. When an error occurs inside one of these go blocks, the name is printed as part of the stack trace. These names can get very long, for example one recent stack trace I saw was:

day8.mwi_enhancer.steps.load_mwi.handlers.load_previous_steps_state.cljs.core.async.impl.dispatch.run.call.day8$mwi_enhancer$steps$load_mwi$handlers$load_previous_steps_state_$_state_machine__24889__auto____1

This name is so long that it doesn't even fit on the display of a 13" Macbook in the Chrome console, and part of the name and location of the code is wrapped.

Can anything be done about making the name shorter? In this case there is a lot of duplication in the name, but there is probably a good reason why it was done this way.






[ASYNC-147] subscriber blocks other subscriber of a topic Created: 24/Sep/15  Updated: 28/Sep/15  Resolved: 28/Sep/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Sonny To Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

Here is code to reproduce the problem:

code
(require-macros '[cljs.core.async.macros :refer [go go-loop]])
(require '[cljs.core.async :refer [put! <! >! chan pub sub unsub]])

(defonce message-bus (chan))
(defonce message-publication (pub message-bus (fn [msg]
(if (vector? msg)
(first msg)
:no-topic))))

(def s1 (chan))
(def s2 (chan))

(sub message-publication :foo s1)
(go (while true (println "s1 " (<! s1))))
(put! message-bus [:foo "bar"]) ;works as expected

(sub message-publication :foo s2) ;blocks s1 from recieving messages
(put! message-bus [:foo "bar"]) ;s1 cannot recieve this message
(go (while true (println "s2 " (<! s2)))) ; unblocks s1; both s1 and s2 can recieve messages

code

This problem happens on both clojure and clojurescript



 Comments   
Comment by Alex Miller [ 28/Sep/15 12:30 PM ]

This is the expected behavior. Docstring for pub states:

"Each item is distributed to all subs in parallel and synchronously, i.e. each sub must accept before the next item is distributed. Use buffering/windowing to prevent slow subs from holding up the pub."

In this code, you are using an unbuffered channel as a sub and (according to the above), it will block delivery.





[ASYNC-146] poll! still not being supported Created: 18/Sep/15  Updated: 18/Sep/15  Resolved: 18/Sep/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Miro Kubicek Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: maven, poll!, release


 Description   

In the last releases (0.1.346.0-17112a-alpha and older) poll! is still not supported even though it has been implemented for awhile (I can see it in the trunk and it seemed to be implemented as a part of ASYNC-104 in Feb).
If I try to use poll! with the latest version from maven repo I get "java.lang.IllegalAccessError: poll! does not exist".
Further checking maven repo history it even seems that the latest release is from 22-Sep-2014 - i.e. a year ago? Or is it a glitch? I understand this is an alfa, but especially for that reason it would be great if there were newer "nightly builds" / versions available in maven/leiningen.
Would getting the trunk (master) code directly from git be a suggested approach? (as opposed to using maven/leiningen)



 Comments   
Comment by Alex Miller [ 18/Sep/15 10:32 AM ]

I'm closing this as it's really a question not a ticket.

We are working towards a release in the near future that will include this functionality. Unfortunately, the existing build setup for this particular library does not make it easy to release interim snapshots - you are welcome to build your own from master if you like.





[ASYNC-145] mix throws assertion error when many channels are added to a mix. Created: 08/Sep/15  Updated: 08/Sep/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Angus Fletcher Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Found in core.async version 0.1.346.0-17112a-alpha


Attachments: Text File 0001-Change-channel-to-sliding-buffer-090815.patch     Text File 0001-Change-channel-to-sliding-buffer.patch    
Patch: Code

 Description   

When a large number of channels is added to a mix simultaneously, this error is thrown:

java.lang.AssertionError: Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.

We can reproduce the issue with the following in a REPL:

user> (require '[clojure.core.async :as a])
nil
user> (defn mixtest []
(let [out (a/chan 1024)
mix (a/mix out)]
(dotimes [i 2048]
(let [chan (a/chan)]
(a/admix mix chan)
(a/put! chan i)))))

(mixtest)
#'user/mixtestAssertionError Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.
(< (.size puts) impl/MAX-QUEUE-SIZE) clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:150)
user>

This is a consequence of the use of an unbuffered channel in the mix implementation. Since the channel's function is just to update the mix's internal data structure, using a windowed buffered channel appears to clear up this issue. Patch 0001-Change-channel-to-sliding-buffer.patch contains the proposed solution.



 Comments   
Comment by Angus Fletcher [ 08/Sep/15 2:32 PM ]

Update patch to include cljs fix as well.





[ASYNC-144] pipeline-async docstring correction Created: 27/Aug/15  Updated: 27/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Derek Troy-West Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

Currently says:

"af must close! the channel before returning."

more correctly:

"af is responsible for closing the channel.", or similar.

af is asynchronous so should return immediately, the thread or go-block it spawns should close the channel later.






[ASYNC-137] Make (<! (timeout 0)) be closer to 0 Created: 03/Aug/15  Updated: 25/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Mike Thompson Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: None


 Description   

Currently `(<! (timeout 0))` will take about 4ms on a modern browser because it ends up as `(js/setTimeout f 0)`. On a modern browser, setTimeout won't come back in under 4ms, even if given a 0. And 4ms is too long if you are just wanting to hand back control to the browser for the minimum about of time.

See:
https://github.com/clojure/core.async/blob/d073896192fa55fab992eb4c9ea57b86ec5cf076/src/main/clojure/cljs/core/async/impl/timers.cljs#L161-L164

followed by:
https://github.com/clojure/core.async/blob/d073896192fa55fab992eb4c9ea57b86ec5cf076/src/main/clojure/cljs/core/async/impl/dispatch.cljs#L35-L36

I was imagining this tiny rewrite:

(defn queue-delay [f delay]
(if (= 0 delay)
(goog.async.nextTick f) ;; <---- new path for 0
(js/setTimeout f delay)) ;; <---- existing path



 Comments   
Comment by Patrick O'Brien [ 03/Aug/15 10:31 PM ]

I think the use of goog.Timer.callOnce should work for any amount of time and would still solve the problem posed by the use of js/setTimeout.

Comment by Mike Thompson [ 04/Aug/15 2:38 AM ]

From what I can see, goog.Timer.callOnce use setTimeout. But that's specifically what we're trying to avoid in the case of a 0ms duration. So I don't see callOnce as a solution.

Comment by Patrick O'Brien [ 04/Aug/15 7:27 AM ]

I think you are right, Mike. Hard as it is to believe, since obviously Google knows about the problem. But now that I'm rereading the goog.Timer code I can't see that they've fixed it here. Looks like goog.async.nextTick is the only solution.

Comment by Patrick O'Brien [ 04/Aug/15 7:33 AM ]

I just wanted to mention that this issue is for the ClojureScript version of core.async, and that it appears to be fairly idiomatic to use (<! (timeout 0)) as a way to temporarily pass control back to the js event processing. So I think it is very important to make this efficient.

Comment by Ghadi Shayban [ 06/Aug/15 1:56 PM ]

In reading this "bug", it seems that the actual problem is having more varied goblock dispatch in CLJS. It could/should be improved, but making (<! (timeout 0)) – non-sensical code – behave differently is not addressing the problem in a meaningful way.

We should eliminate the need for this hack at the root, rather than make undefined behavior more specific. ASYNC-131 and ASYNC-43 are related and probably better starting points.

Comment by Mike Thompson [ 06/Aug/15 10:32 PM ]

@Ghadi

  1. I can't see how issue 131 is closely enough related to act as a "better starting point". It has a different thrust, represents different needs, and will likely have a different solution (as evidenced by the current suggestions).
  1. It almost looks to me as if Issue 43 could be closed with the switch to Google Closure dispatch noted by David Nolen. BTW, you'll note that my suggestion is to indeed use goog.async.nextTick (so I'm adhering to the use Google Closure approach).

Also, I would disagree with your assertion that (<! (timeout 0)) is "non-sensical code". In a single threaded environment, like the browser, there is sometimes a need to explicitly "hand back control" to the browser from within go loops (if that go loop is, for a time, hogging the CPU). At the moment (<! (timeout 0)) is the only way I know to do that. It could reworked to be (<! (yield-cpu)) in order to make the intent clearer, and that might be nice, but what I care about is having a way of yielding which doesn't always cost 4ms. I believe this need is genuine, and very "sensical"

Comment by Mike Thompson [ 06/Aug/15 11:06 PM ]

@Ghadi
Just to be clear ... you refer to this as a "bug" in double quotes. Almost as if it isn't a bug.

But for me this is a sufficiently real "bug" that I'll have to either work with a fork OR move away from using core.async entirely.

I need (<! (timeout 0)) to mean 0, or as close to it as possible. I need it to work as advertised. The current implementation does not deliver on 0 (or as close to it as possible), and instead delivers the same as (<! (timeout 4)) and 4ms is FOREVER compared to 0ms (or close to it). At 4ms each time around, a goloop can only iterate 250 times a second.

Anyone who has a goloop listening to the output of a bursty websocket will have this problem. They need to process like crazy (no 4ms delays) BUT also hand back control so the browser can do what it needs to, every now and again. All of this get exacerbated when a browser page loses "focus", goes into the background, and the js gets throttled, animation frames slow down, etc.

Comment by Patrick O'Brien [ 15/Aug/15 9:59 PM ]

In case it helps anyone who might be reading this, I have begun using the following as a temporary solution:

(defn yield []
  (let [ch (chan)]
    (goog.async.nextTick #(close! ch))
    ch))

(go
  ; Some code...
  (<! (yield))  ;; Instead of (<! (timeout 0))
  ...)
Comment by Mike Thompson [ 25/Aug/15 5:04 AM ]


That's nice! Certainly provides a placeholder solution.





[ASYNC-143] Assert that fixed buffers must have size > 0 Created: 18/Aug/15  Updated: 18/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None

Attachments: Text File async-143.patch    
Patch: Code
Approval: Triaged

 Description   

(chan 0) is invalid (and is not the same thing as an unbuffered (chan)).

Approach: Add an assertion to throw on (chan 0). I found based on some failing tests that one place this can come up is in to-chan, which creates a buffer sized to the collection (0). The patch also changes to-chan to return an empty, closed chan if the collection is of size 0.

Patch: async-143.patch






[ASYNC-141] Convert project to use Reader Conditionals Created: 18/Aug/15  Updated: 18/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

There is a fair amount of duplicated code between the Clojure and ClojureScript versions of core.async. Converting the library to use Reader Conditionals would get rid of a lot of code volume and reduce the chances of changes in one library not being correctly implemented in the other. It may also make maintenance and source reading easier by making the platform specific code stick out more, where the 'essence' of core.async stays in pure Clojure. This could also be an opportunity to bring cljs.core.async.macros into the main async namespace using reader conditionals.

Despite these benefits there are a few downsides to this:

  • Currently the two ports are able to exist fairly separate from each other and the domain experts in each can work on their own part without affecting the other. If this was a concern, more tests may be able to be added to ensure there aren't breakages
  • Combining the two ports may make it harder to read either version in the same source file.

As this is a fairly large change, I wanted to get feedback on this proposal before going any further.



 Comments   
Comment by Alex Miller [ 18/Aug/15 8:34 AM ]

For the time being, we are still supporting Clojure 1.6 with core.async so this is not an option. At some point, that will change, not sure when.

Another aspect here is that reader conditionals are designed for code that is very similar across platforms but the code for clj and cljs is substantially different in many cases, so it is not an obvious candidate. Rather, I would expect to see alternate versions of the same namespace in many cases.

Changing namespaces would also of course break existing users, so we would need to plan ahead for this change.

While I think there is an endpoint with more consolidated code base (and a better build system, which I have been working on), I do not think it's wise for anyone outside the core team to work on it.





[ASYNC-139] inconsistent thread pool in async/pipeline-blocking function Created: 15/Aug/15  Updated: 16/Aug/15  Resolved: 15/Aug/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Miro Kubicek Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

platform non-specific


Attachments: PNG File pipeline diff screenshot .png    
Patch: Code

 Description   

Hi there,
in pipeline* function for the :blocking processing there is no loop within each thread - while for the async processing there is always a go-loop [] which can be re-written as (go (loop []..., for sync processing there is only "thread" macro with no loop in it and with a recur call at the end. I would assume recur without a loop jumps back to a given defn. Why is there no loop? If a thread is instantiated then it should continue doing its work.

Apologies if I am missing something and the recur really jumps back to the thread macro.

To illustrate this in the code - basically I would expect that the following:

pipeline*
:blocking (thread
                           (let [job (<!! jobs)]
                             (when (process job)
                              (recur))))

Should look more like this:

pipeline*
:blocking (thread (loop[]
                           (let [job (<!! jobs)]
                             (when (process job)
                              (recur)))))

Even if the recur without loop jumps back to the thread macro I would wonder whether it might lead to some unexpected behavior (each thread being recreated after every iteration) and if it jumps elsewhere it might lead to the thread halting after only one iteration is performed.

My (closed) pull request for this fix:
https://github.com/clojure/core.async/pull/47/commits

Cheers
Miro



 Comments   
Comment by Alex Miller [ 15/Aug/15 4:09 PM ]

thread is a macro that nests the body inside an anonymous fn, which serves as a recur target. Thus the code is working as expected.

Comment by Miro Kubicek [ 16/Aug/15 1:25 AM ]

Thanks for the explanation Alex!





[ASYNC-102] implement IDeref, IBlockingDeref for channels Created: 05/Nov/14  Updated: 12/Aug/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File async-102-2.patch     Text File async-102.patch    
Patch: Code and Test
Approval: Vetted
Waiting On: Alex Miller

 Description   
  • each deref will take a value from the channel
  • deref from a closed channel returns nil
  • deref with timeout must either timeout or take value from the channel (not both!). Might just want to call some variant of alt!! through the var
  • use internals efficiently
  • do NOT implement IPending/realized, neither interpretation is that great

Patch: async-102-2.patch

The implementation of deref and timed deref is conceptually straightforward, but is difficult to implement due to circular namespace dependencies. Suggestions welcome.



 Comments   
Comment by Stuart Halloway [ 09/Nov/14 8:24 AM ]

The async-102.patch does not match the description "a channel becomes realized when it is closed" – instead, a channel becomes realized when a value is currently available. Both of these interpretations seem problematic to me, and I think the definition of realized? needs clarification to support this new case. The other uses of realized? in Clojure all guarantee that subsequent derefs will be filled immediately, but that will not be true for channels if another consumer takes the value.

Once that is solved, I need better doc strings for the core.async implementation protocols before I can screen this. What guarantees are made about use of threads? Where does the work enqueue, and what happens if the queue is full?

Comment by Alex Miller [ 09/Nov/14 10:32 PM ]

My implementation of realized? is based on Rich's response to your question (from internal chat): "open+empty = false, else true" and I believe that is what's implemented and reflected in the tests. This also makes more sense to me than only being realized on close.

Your comment vs other cases of realized? seems accurate, so I agree that's a question.

Can you be more specific on which implementation protocol? I'm guessing you specifically mean Channel and Buffer. Any thread could be calling into the Channel. The M2MC protects its internal state with a mutex and will forward calls down to the Buffer. All calls into the buffer are protected by the channel mutex. M2MC enqueues pending puts and takes in the puts and takes lists. Those are bounded by the fixed limit clojure.core.async.impl.protocols/MAX-QUEUE-SIZE (1024), at which point an exception is thrown.

Comment by Alex Miller [ 05/Jan/15 12:33 PM ]

The conceptual implementations of IDeref and IBlockingDeref on ManyToManyChannel using existing async constructs is relatively straightforward:

(deftype ManyToManyChannel
  #_existing_code...
  IDeref
  (deref [this] (<!! this))

  IBlockingDeref
  (deref [this ms timeoutValue]
    (alt!!
      this ([val _] val)
      (timeout ms) timeoutValue)))

However, M2MC is defined in clojure.core.async.impl.channels. <!!, alt!!, and timeout are all defined in clojure.core.async, which depends on clojure.core.async.impl.channels, so there is a cyclic dependency problem here. The <!! and timeouts are pretty easy to deal with looking up the var behind delay like this:

(def ^:private <!!' (delay (find-var 'clojure.core.async/<!!)))

;; then:
  IDeref
  (deref [this] (@<!!' this))

However, I'm a little stumped on how to do the equivalent with alt!!, which is a macro around do-alt and alts!!.

Comment by Ghadi Shayban [ 05/Jan/15 1:14 PM ]

Don't rely on resolving the circular dependency. Instead make a shared alt-flag and enqueue the two handlers [1]. fret will be delivering a promise, and then deref the promise.

[1] https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L230-L231

Comment by Alex Miller [ 06/Jan/15 11:42 AM ]

New patch goes back to prior implementation of blocking deref (basically guts of <!!) and adds new alts-based version of timed deref.

Comment by Fogus [ 09/Jan/15 1:11 PM ]

> However, I'm a little stumped on how to do the equivalent with
> alt!Unable to render embedded object: File (, which is a macro around do-alt and alts) not found.!.

I wonder if the following will solve the alts!! problem:

(def ^:private ^:macro alts!!' (deref (find-var 'clojure.core.async/alts!!)))

And then using it in the deref impl behind IBlockingDeref. As it stands it's difficult to grok the code as written given that it's using some core.async implementation details (indeed, the same code). Whereas your implementation of IBlockingDeref.deref is crystal clear, the implementation prompted by the cyclic dependency is... less so. Not to belabor the point (which I seem to be), but alts!! is a clear implementation for this, but using the guts muddies the water. If I had my druthers, I'd prefer clarity.

That said, being a human macro-expander I was able to reason through the implementation eventually.

Comment by Alex Miller [ 28/Jan/15 2:23 PM ]

That doesn't seem to work to me. If you have some formulation that would, that would be great though!





[ASYNC-43] queue-dispatcher should fall back to process.nextTick if setImmediate is not available Created: 07/Dec/13  Updated: 07/Aug/15  Resolved: 07/Aug/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Travis Vachon Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: cljs

Attachments: Text File async_43.patch     Text File nextTick.patch    
Approval: Triaged

 Description   

discussion here:

https://groups.google.com/forum/#!searchin/clojurescript/nextTick/clojurescript/RW1FMv0UoPE/hsMHI4SLKXYJ

discussion of the differences between setImmediate and nextTick here:

http://stackoverflow.com/questions/15349733/setimmediate-vs-nexttick

it sounds to me like nextTick should be ok, but I'm not familiar with the design decisions in the current implementation

I'm happy to create a patch - will do that shortly.



 Comments   
Comment by Travis Vachon [ 07/Dec/13 7:28 PM ]

patch here: https://github.com/clojure/core.async/pull/40

Comment by Travis Vachon [ 08/Dec/13 8:52 PM ]

just the code, not entirely sure how to test this, but have tested manually in the parse.com cloud code environment

Comment by Travis Vachon [ 10/Dec/13 9:53 AM ]

add git am able patch

Comment by Travis Vachon [ 05/Mar/14 5:21 PM ]

just want to ping this issue - anything else it needs to be mergable? I'm using a core.async fork at the moment and would love to get back on the main line!

Comment by David Nolen [ 05/Nov/14 6:24 AM ]

The patch probably needs updating - we've since switched to Google Closure for choosing the underlying dispatch functionality for the JavaScript environment.

Comment by David Nolen [ 07/Aug/15 2:10 PM ]

we use Google Closure to make this decision.





[ASYNC-136] Add webinar to presentations section in README.md Created: 30/Jul/15  Updated: 30/Jul/15  Resolved: 30/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Michiel Borkent Assignee: David Nolen
Resolution: Completed Votes: 0
Labels: None


 Description   

Add a link to the core.async webinar presented by David Nolen to the presentations section in README.md.
The link is: http://go.cognitect.com/core_async_webinar_recording



 Comments   
Comment by Alex Miller [ 30/Jul/15 8:21 AM ]

Added





[ASYNC-135] alt! with a closed channel will still return results from the closed channel Created: 27/Jul/15  Updated: 30/Jul/15  Resolved: 30/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

I'm not sure if this is a documentation bug, or an implementation bug, but alt! will always return nil from chan1 in the following code.

(let [chan1 (async/chan 10)
      chan2 (async/chan 10)]
  (async/close! chan1)
  (async/>!! chan2 "run")
  (async/go (async/alt! :priority true

               chan1 ([val] (println "chan1" val))

               chan2 ([val] (println "chan2" val)))))
;; => chan1 nil

The intent of my code was to alt! over the two channels, preferring chan1, but running chan2 if chan1 has nothing to take. However changing this would interfere with using timeout channels in an alt!, so it seems like documenting this might be the best fix?



 Comments   
Comment by Leon Grapenthin [ 30/Jul/15 3:25 AM ]

This is expected. Takes from closed channels return nil. The behavior is documented in the docstring of close!





[ASYNC-134] async/alts!! with an empty vector throws ArrayIndexOutOfBoundsException Created: 27/Jul/15  Updated: 28/Jul/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I'm not sure if this is intended, but this code throws an exception.

(async/alts!! [])
;; => ArrayIndexOutOfBoundsException 1  clojure.core.async/random-array (async.clj:182)

Is this desired behaviour, or is there another way we could represent that this collection is empty? I'm thinking of the pattern where we take from any channel, and recur with that channel removed from the collection until the collection is empty. Perhaps this usage should be discouraged. If it should be discouraged, should it be documented in the docstring?



 Comments   
Comment by Alex Miller [ 28/Jul/15 9:08 AM ]

What would you expect this to do?

Comment by Daniel Compton [ 28/Jul/15 5:20 PM ]

You could return [nil nil]? That's not great either though. Documentation is probably a better fix.





[ASYNC-113] alt!! should not evaluate the :default expr if a non-default clause is selected Created: 13/Jan/15  Updated: 27/Jul/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Caleb Spare Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Clojure 1.5.0, core.async 0.1.346.0-17112a-alpha


Approval: Triaged

 Description   

alt!! will always evaluate the expr given in the :default clause. For instance, consider this code:

(require '[clojure.core.async :as async])
(let [c1 (async/chan 1)
      c2 (async/chan 1)]
  (async/alt!!
    [[c1 "a"]] (println "first")
    [[c2 "a"]] (println "second")
    :default (println "default")))

It will print either "first" or "second", but also (always) "default". Reading the documentation, technically it doesn't say whether the non-selected clauses are evaluated (only that the value of the selected expr is returned) but it's certainly not the behavior I would expect.

I haven't checked whether alt! does the same thing as well.



 Comments   
Comment by Daniel Compton [ 27/Jul/15 7:37 PM ]

It looks to me like :default is expected to be a value, not a function, and println is being run as part of evaluating the alt!! expression, not as the return value.





[ASYNC-94] Allow user supplied executor/thread-pool (and potentially other options) to be used in go, thread macros and chan Created: 09/Oct/14  Updated: 17/Jul/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Max Penet Assignee: Unassigned
Resolution: Unresolved Votes: 11
Labels: None

Attachments: File ASYNC-94e.diff    
Patch: Code
Approval: Triaged

 Description   

Right now thread pools used by core.async are defined as globals, they are shared by all go or thread macro on their respective pools:

see https://github.com/clojure/core.async/blob/ac0f1bfb40237a18dc0f03c0db5df41657cd23a6/src/main/clojure/clojure/core/async.clj#L391-L411
and https://github.com/clojure/core.async/blob/ac0f1bfb40237a18dc0f03c0db5df41657cd23a6/src/main/clojure/clojure/core/async/impl/dispatch.clj#L16

It would be very useful to have more control over this, hopefully not via a simple setter, like the one we have in core for agents (set-agent-send-executor!), since it only allows one pool to be used at a time.
One possible solution would be to do that via an option map parameter to thread-call, and/or some other mechanism for go blocks (go-call with the same kind of signature). Making this parameter a map would also allow to further extend internals in the future, if necessary, without requiring heavy changes to the API.



 Comments   
Comment by Max Penet [ 15/Mar/15 9:35 AM ]

I have a patch here https://github.com/mpenet/core.async/commit/1c2b0a9af9a5e891af0f1631b8debd337a73999d that adds this features.

It adds 2 macros: go* and thread*. These are identical to go/thread, but takes first an option map, then the body. The option map can receive an :executor key with an clojure.core.async.impl.protocols/Executor implementation.

I took the liberty to move the `thread` executor in the same namespace as the ioc/go one, and have it support the same interface instead of using a raw j.u.c Executor instance. Both instances declared in this namespace are now lazy (in a delay block), preventing useless thread pool initialisations/leaks.

Lastly I also modified thread-call to support that same option map.

I'll attach the .patch file to the ticket shortly.

Comment by Max Penet [ 15/Mar/15 12:22 PM ]

well actually this patch is incomplete, I am just discovering that ioc-macros rewriting leads to calls on the global thread pool (via put!, take! callbacks in particular).

We could allow to pass an executor at channel creation to force ops on this channel to run there relatively easily (partial example here https://github.com/mpenet/core.async/commit/1449b3842052033cdf917ae92259ad9789722fdb).

I believe this is how manifold handles it as well, but there might be more unknowns for me at this point.

Comment by Max Penet [ 15/Mar/15 2:51 PM ]

patch with `chan` modifications included. chan now can take an additional parameter that would be an clojure.core.async.impl.protocols/Executor compatible executor, this executor will then be used instead of the global default pool when supplied.

Comment by Max Penet [ 17/Mar/15 9:52 AM ]

provide patch as single commit (more readable). You can also view the changes here: https://github.com/mpenet/core.async/commit/e7dea04553935863d1abb1880d84bbdc273854ec

Comment by Ghadi Shayban [ 17/Mar/15 1:32 PM ]

Hi Max. I think that having some better control over this is necessary.

There are basically two needs:
1) callbacks need to wake up parked go-blocks.
2) running heavier `thread`s (which may block and thus cannot share the same pool)

Right now channel callbacks don't know what they are waking up. I think that's a good design. If they are waking up a thread, it's only indirectly doing so by delivering a promise [1]
I think there should be no mention of a specific executor in channels.clj Having multiple different places to run go-blocks is probably an antipattern as go-blocks should not be blocking or doing I/O.

I very much welcome some control over the `thread` block's executors though.

[1] https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L137

Comment by Max Penet [ 18/Mar/15 3:18 AM ]

Hi Ghadi,

Thanks for the feedback, but I don't think you read my patch correctly.

I do not change the case you mention (promise), the only thing that's changed is that for the few cases where clojure.core.async.impl.dispatch/run is called, meaning the function is run in a globally defined threadpool used by everything in core.async with the exception of thread, it can *optionally* be done via a function argument supplied threadpool. The default is unchanged, if you don't pass an executor you get the current master core.async behavior without any change at all.

Few people mentioned that need, some publicly, some in private, myself included, and I know of a team who expressed the same concerns and just switched one of their systems from core.async to manifold because the latter offer more knobs of that sort.

In the end it's about control, it changes nothing to core.async and it's current execution model if you don't care about it, but it's important to people who needs that kind of fine grained tuning.

It came up again a couple of days ago on twitter: https://twitter.com/puredanger/status/576378306062262272 and you could certainly find reference of it in the irc logs as well.

I would have released this as a lib if that was possible, but the code implicated is sometimes very deeply rooted and make it nearly impossible to do without forking the whole library, hence that patch.

Comment by Max Penet [ 18/Mar/15 4:02 AM ]

same as previous patch, added missing arity to promise-chan

Comment by Stuart Halloway [ 11/Jul/15 7:38 AM ]

Consider my upvote to be for the problem, not necessarily for the approach taken here.

It seems to me that go and thread are separate cases, deserving of separate consideration. It isn't clear to me why chan needs consideration at all, and I don't want to have to read a patch (or learn about impl namespaces) to discover why. Can somebody explain the case for chan without reference to implementation?

Is there a dev list discussion on this?

Comment by Max Penet [ 17/Jul/15 12:42 AM ]

Sure, as long as this is improved/fixed.

If you would read the patch, or the source for channels you would see
that channels run their callbacks on a threadpool.

When this patch was submitted I spent time discussing it with Timothy
Baldridge on IRC and made some changes at his suggestion. Since you
probably share offices with him I suggest you discuss the current
approach with him as he is more familiar with the codebase than both
of us.





[ASYNC-132] Can't close over locals in #js in go Created: 09/Jul/15  Updated: 09/Jul/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

CLJS 3308



 Description   
(go
  (let [bar 42]
    #js{:foo bar}))

Use of undeclared Var my.core/bar






[ASYNC-131] go! or "go-now" for CLJS Created: 29/Jun/15  Updated: 06/Jul/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Thomas Heller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I'd like to see a variant of the "go" macro that executes immediately until the first parkable operation instead of dispatching (effectively calling "setTimeout" twice). This is primarily for CLJS since CLJ can get away with blocking operations but it still might be useful there as well.

My Use-Case:

I currently use core.async to orchestrate CSS animations together and while it works perfectly fine with "go" it does require careful attention to detail on the order of operations since "go" will yield control back to the browser which may decide to reflow/repaint the screen although the animation wasn't properly started yet.

Example:

(defn show-toast [message]
  (let [toast (create-toast-dom message)]
    (dom/append js/document.body toast)
    ;; X - PROBLEM: REPAINT HAPPENS NOW
    (go (<! (anim/slide-in-from-bottom animation-time toast))
        (<! (timeout 3000))
        (<! (anim/fade-out animation-time toast))
        (dom/remove toast)
        )))

This example tries to create a "Toast" [1] DOM node which is animated in from the bottom and removed after 3 seconds. A real implementation would require handling clicks and so forth but the example is easy enough to describe the problem. The (anim/...) functions set the appropriate CSS and return a (async/timeout time) channel which lets us wait for the "completion".

The problem at X is that the go is dispatched and control is given back to the browser. Therefore the "setup" part of the animation is delayed and the toast dom node is rendered by the browser as is. Usually this means that the browser will do a repaint. This will have a "flicker" effect as the node appears and then disappears again and animates (depending on the default CSS).

The simple solution is to move the dom/append call into the go block, which is easy enough in this example but hard for more complex. Another solution is to start the animation outside the go:

(defn show-toast [message]
  (let [toast (create-toast-dom message)]
    (dom/append js/document.body toast)
    (let [slide-in (anim/slide-in-from-bottom animation-time toast)]
      ;; REPAINT, but slide-in animation already started so no "flicker"
      (go (<! slide-in)
          (<! (timeout 3000))
          (<! (anim/fade-out animation-time toast))
          (dom/remove toast)
          ))))

Again, simple enough here, hard for more complex things.

What I would prefer to see is a "go!" variant which instead of dispatching the go block just starts executing it.

(defn show-toast [message]
  (let [toast (create-toast-dom message)]
    (dom/append js/document.body toast)
    (go! (<! (anim/slide-in-from-bottom animation-time toast)) ;; REPAINT ON <!
         (<! (timeout 3000))
         (<! (anim/fade-out animation-time toast))
         (dom/remove toast)
         )))

The subtle difference is that the browser does not get a chance to do anything before we are ready and would wait anyway. After fine-tuning a few of those animation effects myself it always requires moving stuff out of or into go blocks as it stands now.

I'm aware that this is a rather specific use-case and core.async might not even be the best way to do this at all but so far I had great success with it and the code is very easy to reason about.

I have not looked too much into the internals of core.async but go! should be a pretty simple addition. If this change is accepted I would start looking into things and create a patch, just wanted to make sure there were no objections before doing so.

Hope my intent is clear. CSS animations can be a bit counter-intuitive which might not make it obvious where the problem actually lies.

[1] http://www.google.com/design/spec/components/snackbars-toasts.html#snackbars-toasts-specs



 Comments   
Comment by Thomas Heller [ 06/Jul/15 6:06 AM ]

I looked into the issue to find out what needed to be done on the core.async side to facilitate this change, turns out nothing.

(defmacro go!
  "just like go, just executes immediately until the first put/take"
  [& body]
  `(let [c# (cljs.core.async/chan 1)
         f# ~(ioc/state-machine body 1 &env ioc/async-custom-terminators)
         state# (-> (f#)
                    (ioc/aset-all! cljs.core.async.impl.ioc-helpers/USER-START-IDX c#))]
     (cljs.core.async.impl.ioc-helpers/run-state-machine state#)
     c#))

Works perfectly fine so far and can live perfectly well inside a library, I'm beginning to think that this might be a better "default" for CLJS since there are no "threads" that could start running the "go".

With the normal "go" the flow of execution is:

code before go
setTimeout/nextTick
code inside go until first take/put
setTimeout/nextTick

With go! it is:

code before go!
code inside go! until first take/put
setTimeout/nextTick

Given the non-blocking nature of Javascript the extra setTimeout does not hurt it doesn't provide much benefit either.

Since no changes to core.async are required I can simply keep this macro in my library. My use-case is rather specific so feel free to close the issue if there is no interest to make this "common".

Comment by Leon Grapenthin [ 06/Jul/15 5:46 PM ]

As a default this would violate the async execution guarantee of go which would be a breaking change. Some observations from my user perspective: For readability I'd prefer both examples you proposed for usage of go over go-now because go conveys async execution of its entire body whereas in go-now I'd have to scan the body for a parkable operation to determine the part which executes synchronously. I find it difficult to convince myself that there could really be problems so complex that go-now would be preferable over refactoring go. (I don't think you can safely rely on impl namespaces in library code )





[ASYNC-106] A unit test in ioc_macros_test.clj tests nothing Created: 10/Nov/14  Updated: 06/Jul/15  Resolved: 06/Jul/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Andy Fingerhut Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Approval: Ok

 Description   

I am not sure what it should be doing, but this test in file ioc_macros_test.clj is always true, because it is of the form (is (= x)), which always passes:

(testing "lazy-seqs in bodies"
    (is (= (runner
            (loop []
              (when-let [x (pause 10)]
                (pause (vec (for [i (range x)]
                              i)))
                (if-not x
                  (recur))))))))


 Comments   
Comment by Alex Miller [ 06/Jul/15 4:08 PM ]

Committed a fix.





[ASYNC-23] Support channel buffers of unlimited size Created: 10/Sep/13  Updated: 29/Jun/15  Resolved: 15/Sep/13

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Constantine Vetoshev Assignee: Timothy Baldridge
Resolution: Declined Votes: 0
Labels: None
Environment:

Clojure
ClojureScript


Attachments: Text File 0001-Add-unlimited-size-buffer-implementation.patch    
Patch: Code and Test

 Description   

It is sometimes useful to use buffered async channels and not worry about either pre-allocating their size, or the way sliding and dropping buffers can lose potentially critical data passing through the channel.

The attached patch adds support for clojure.core.async/unlimited-buffer and cljs.core.async/unlimited-buffer for this purpose.



 Comments   
Comment by Timothy Baldridge [ 15/Sep/13 3:55 PM ]

One of the key aspects of core.async is that back-pressure and sliding/dropping buffers allow for decoupling of processes. A unbounded queue introduces many problems. Now a incorrectly designed system can crash, but not for some time. With the other buffer types a bug in the system will either a) allow producers to continue, or b) backup the producers allowing for easier debugging.

This patch doesn't really fit with CSP theory. And one of the main goals of core.async is to make users recognize where there are unbounded buffers, and allow them to remove them. Closing as this doesn't fit with the goals of the library.

Comment by Daniel Compton [ 29/Jun/15 4:38 PM ]

While I agree with Tim that this doesn't fit the goals of core.async, from a quick read through the patch, I don't think theres anything stopping you from creating this as a library (with proper warnings about the implications of using unbounded queues).





[ASYNC-130] Improve put! docstring to mention no more than 1024 pending puts are allowed Created: 21/Jun/15  Updated: 21/Jun/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Daniel Compton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: docstring


 Description   

The put! docstring says:

> "Asynchronously puts a val into port, calling fn0 (if supplied) when
complete. nil values are not allowed. Will throw if closed. If
on-caller? (default true) is true, and the put is immediately
accepted, will call fn0 on calling thread. Returns nil."

It would be good if it also mentioned that it can throw if more than 1024 pending puts are already on the channel. It could be something like:

> "Asynchronously puts a val into port, calling fn0 (if supplied) when
complete. nil values are not allowed. Will throw if closed or if there are more than 1024 pending puts. If
on-caller? (default true) is true, and the put is immediately
accepted, will call fn0 on calling thread. Returns nil."






[ASYNC-129] Channels with transducer using reduced don't work as intended Created: 09/Jun/15  Updated: 11/Jun/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Ghadi Shayban
Resolution: Unresolved Votes: 0
Labels: None
Environment:

CLJ 1.7.0-RC1, (CLJS not tested)



 Description   
(def ch (chan 1 (take 2)))

(put! ch 1)
(put! ch 2)
(put! ch 3)

(take! ch identity)

ClassCastException clojure.lang.PersistentVector cannot be cast to java.util.concurrent.locks.Lock clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:55)

This does not happen if a take is made before the first put so that the buffer size fits. The problem doesn't seem to be related only to the buffer size though.

(def ch (chan 1 (take-while true?)))

(put! ch true)
(put! ch false)
(put! ch false)

(take! ch identity)

ClassCastException clojure.lang.PersistentVector cannot be cast to java.util.concurrent.locks.Lock clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:55)






[ASYNC-128] or evaluation not stopped when exp nests take Created: 09/Jun/15  Updated: 09/Jun/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

CLJS



 Description   
(go (or true
        (= (<! (js/console.log "this should not happen"))
           :doesnt-matter)))





[ASYNC-127] mult distribution behavior doesn't work as intended Created: 08/Jun/15  Updated: 08/Jun/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: mult

Attachments: Text File async-127.patch    
Approval: Triaged

 Description   

Quote from docstring: "[...] each tap must accept before the next item is distributed."

(def ch (chan))

(def m (mult ch))

(def t-1 (chan))
(def t-2 (chan))
(def t-3 (chan))

(def t-1-takes (atom []))

(defn log [l] (partial swap! l conj))

(tap m t-1)
(tap m t-2)
(tap m t-3)

(close! t-3)

(take! t-1 (log t-1-takes))

(take! t-1 (log t-1-takes)) ;; this take shouldn't be happening before
                            ;; a take on t-2

(put! ch true)

(put! ch true)

@t-1-takes

;-> [true true] ;; but it does.

The reason is that the internal atom dctr is decreased twice when a tapped channel is already closed.



 Comments   
Comment by Leon Grapenthin [ 08/Jun/15 1:53 PM ]

Fixing this for clj/cljs





[ASYNC-125] Closing a tap with a pending item blocks the mult input channel Created: 07/Jun/15  Updated: 07/Jun/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Klaus Wuestefeld Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

org.clojure:clojure:1.7.0-alpha5
org.clojure:core.async:0.1.346.0-17112a-alpha


Approval: Triaged

 Description   

Closing a tap without a pending item is OK but closing a tap with a pending item blocks the mult input channel:

(require '[clojure.core.async :refer :all])
(def c (chan))
(def m (mult c))
(def t (chan))
(tap m t)
(>!! c :a)
(close! t)
(>!! c :b)  ; BLOCKS


 Comments   
Comment by Klaus Wuestefeld [ 07/Jun/15 6:22 PM ]

A more general case:

Doing this:

(go (println (>! c 42)))

and then closing c will cause the >! to block, instead of returning false.

If c is closed before that, >! will return false.

Is this race condition the intended behavior?





[ASYNC-49] cljs IOC macro issue with double-dot form Created: 12/Jan/14  Updated: 05/Jun/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Paul Butcher Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

ClojureScript 0.0-2138
core.async 0.1.267.0-0d7780-alpha



 Description   

This ClojureScript compiles and runs as expected:

(let [circle (dom/getElement "circle")]
  (go-loop [x 20]
    (<! (timeout 10))
    (set! (.-value (.-baseVal (.-cx circle))) x)
    (recur (inc x))))

But if I change the set! line to:

(set! (.. circle -cx -baseVal -value) x)

I get the following error:

Wrong number of args (3) passed to: core$-DOT

For further discussion see https://groups.google.com/d/topic/clojurescript/ONMaEho4K0c/discussion



 Comments   
Comment by Andrew S [ 05/Jun/15 8:49 AM ]

I too noticed this, but only when using the .. operator in a go loop. The linked thread further suggests this is a core.async problem with this operator.





[ASYNC-121] compilation warning when calling 'satisfies?' inside a go block Created: 21/Apr/15  Updated: 21/Apr/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Yehonathan Sharvit Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: compiler
Environment:

[org.clojure/core.async "0.1.346.0-17112a-alpha"]
[org.clojure/clojurescript "0.0-3196"]



 Description   

This piece of code is causing the following compilation warning:

(defprotocol a)
(defrecord b [])
(go (satisfies? a (b.)))

WARNING: Use of undeclared Var cljs-explore.me/bit_4884auto_ at line 12 src/cljs_explore/me.cljs






[ASYNC-120] cljs port of promise-chan|buffer / offer! / poll! Created: 15/Mar/15  Updated: 03/Apr/15  Resolved: 03/Apr/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Max Penet Assignee: David Nolen
Resolution: Completed Votes: 1
Labels: None

Attachments: File promise-chan+offerpoll-cljs.diff    

 Description   

Giving a try on this, but I am not really familiar with cljs anymore. It seems fairly straightforward. I would add tests but I am still trying to figure out how to run the current cljs tests at this point.



 Comments   
Comment by Max Penet [ 16/Mar/15 5:16 AM ]

fixed fn-handler calls in offer/poll

Comment by Max Penet [ 16/Mar/15 7:19 AM ]

latest version, with passing tests

Comment by David Nolen [ 17/Mar/15 3:16 PM ]

Will review, thanks.

Comment by David Nolen [ 03/Apr/15 7:08 PM ]

fixed by

https://github.com/clojure/core.async/commit/1afb09d766cce8cfc386d3204bed6f32b483472b
https://github.com/clojure/core.async/commit/d8047c0b0ec13788c1092f579f03733ee635c493





[ASYNC-86] Update tools.analyzer.jvm dep Created: 14/Aug/14  Updated: 30/Mar/15  Resolved: 30/Mar/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Completed Votes: 4
Labels: None

Attachments: Text File 0001-update-to-tools.analyzer.jvm-0.6.6.patch    
Patch: Code
Approval: Ok
Waiting On: Timothy Baldridge

 Comments   
Comment by Nicola Mometto [ 18/Aug/14 1:17 PM ]

This patch also adds a bunch of :op nodes that -item-to-ssa might hit but were not implemented, such as :set! and :host-interop
It also improves/simplifies the passes done by core.async by using info already collected by t.a.jvm passes

Comment by Darrick Wiebe [ 22/Aug/14 7:27 PM ]

The old version of tools.analyzer.jvm seems to be causing a problem with the lein-figwheel plugin if I update it to the current version of core.async.

I've reported that at https://github.com/bhauman/lein-figwheel/issues/32 and am unsure if an update to this dependency would fix the issue, but thought I should point it out here.

Comment by Nicola Mometto [ 22/Aug/14 8:23 PM ]

No, this is not related to core.async.
tools.analyzer.jvm depends on core.cache and if a leiningen plugin pulls an older version of core.cache, than plugins that depend tools.analyzer.jvm will not work, assuming they are :eval-in-leiningen true

Comment by Ambrose Bonnaire-Sergeant [ 12/Oct/14 12:50 PM ]

This issue is stopping core.typed from upgrading past tools.analyzer.jvm 0.3.0.

Comment by Andy Fingerhut [ 12/Oct/14 4:33 PM ]

Ambrose, this comment is directed to you, with no intended bearing on this ticket ASYNC-86.

Just an FYI regarding core.typed that I have found it not exactly fun, but it has eliminated some problems by having Eastwood copy the source code of tools.analyzer(.jvm) and its dependencies, with renamed namespaces, into its own source tree. This enables Eastwood to analyze+eval source code of any version of those libraries without causing conflicts. I have a pre-alpha version of a tool to assist in automating parts of this copy&rename called Dolly [1]. If Dolly was polished some more, it might become useful to you for this purpose in core.typed.

Benedek Fazekas has created a tools called MrAnderson [2] for the same purpose (independently developed from Dolly). I haven't looked into the details of how it works yet, but it may do a better job than Dolly 0.1.0.

[1] https://github.com/jafingerhut/dolly for the library, https://github.com/jonase/eastwood/blob/master/copy-deps-scripts/README.md#using-dolly-to-copy-dependencies-into-eastwood-source-code for copy-and-paste-into-a-REPL forms that I use to copy in updated versions of Eastwood dependencies into its source tree. The polishing I mention above would cut that down significantly.

[2] https://github.com/benedekfazekas/mranderson I learned about this lib from a Github issue on Eastwood, where there is a little bit of discussion: https://github.com/jonase/eastwood/issues/87

Comment by Nicola Mometto [ 15/Oct/14 12:35 PM ]

Added patch to update to tools.analyzer.jvm 0.6.1.
This makes the core.async passes use the new tools.analyzer pass scheduler and the :passes-opts mechanism to configure the passes.

Comment by Timothy Baldridge [ 27/Oct/14 9:17 AM ]

Took a look at the latest patch, it applies cleanly and the tests all pass. Marking it as screened.

Comment by Nicola Mometto [ 27/Oct/14 11:18 AM ]

Updated patch to use version 0.6.3 of t.a.jvm which has a faster scheduler and significant performance enhancements

Comment by Alex Miller [ 30/Oct/14 3:56 PM ]

Tim, can you re-screen latest?

Comment by Nicola Mometto [ 03/Nov/14 1:59 PM ]

Attached another patch updating to the recently released t.a.jvm 0.6.4, identical to the previous patch except for the dependency.

Comment by Nicola Mometto [ 10/Nov/14 3:08 PM ]

Updated lastest patch adding a -item-to-ssa impl for :with-meta nodes, discussed with Tim Baldridge on irc

Comment by Timothy Baldridge [ 27/Feb/15 3:13 PM ]

Screened, looks good.





[ASYNC-12] Add chan? or channel? function to identify a channel Created: 19/Jul/13  Updated: 24/Mar/15  Resolved: 12/Dec/13

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Brenton Ashworth Assignee: Timothy Baldridge
Resolution: Declined Votes: 0
Labels: None


 Description   

There are cases where one may be handed a value which could possibly be a channel. It would be nice to have a function which returns true if a value is a channel. This can currently be done from user code but requires reaching into the clojure.core.async.impl.channels namespace and depending on an implementation detail.

(defn channel? [c]
(satisfies? channels/MMC c))



 Comments   
Comment by Hugo Duncan [ 26/Sep/13 9:20 AM ]

I ran into the need for this when writing a schema to validate input arguments.

Comment by Timothy Baldridge [ 12/Dec/13 10:37 PM ]

We're rejecting this feature for now. If you need to check if an object implements the correct interfaces, feel free to dive into clojure.core.async.impl.protocols and check for satisfies? against ReadPort and WritePort. Not only do these allow you to accept ReadPorts that aren't channels, but it also keeps the main interface from getting cluttered.

At least this was the argument Rich proposed last time we talked. If this won't work, feel free to submit a new ticket or start a clojure-dev mailing list thread.

Comment by Kaspars Dancis [ 24/Mar/15 3:42 PM ]

Would it make sense to move or at least alias the protocols in clojure.core.async? .impl. seem to imply that it's not public interface and is subject to change.





[ASYNC-118] (ClojureScript) A let-binding named 'arguments' not bound in go block on nodejs Created: 14/Mar/15  Updated: 14/Mar/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Sean Doig Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: binding, cljs, nodejs
Environment:

OS X 10.10, node 0.10.35, Clojure 1.6.0, ClojureScript 0.0-3030, core.async 0.1.346.0-17112a-alpha



 Description   

Discovered this while using tools.cli 0.3.1 where it's handy to destructure one of their returned maps which has a key :arguments, but you can reproduce it like so:

(let [arguments [:a :b]]
    (println "Args Sync:" arguments)
    (go (println "Args Async:" arguments)))

which prints

Args Sync: [:a :b]
Args Async: #js {:0 #js [#<function (state_12638){
switch(arguments.length){
case 0:
return dls$rip$core$go_BANG__$_state_machine__6252__auto____0.call(this);
case 1:
return dls$rip$core$go_BANG__$_state_machine__6252__auto____1.call(this,state_12638);
}
throw(new Error('Invalid arity: ' + arguments.length));
}> 1 nil nil nil nil #<[object Object]>]}

I haven't tested this in the browser yet, so it might not be limited to node.



 Comments   
Comment by Sean Doig [ 14/Mar/15 12:34 PM ]

Just tried, can't reproduce in Chrome.





[ASYNC-104] offer! and poll! Created: 10/Nov/14  Updated: 23/Feb/15  Resolved: 23/Feb/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Stuart Halloway Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File async-104.patch    
Patch: Code and Test
Approval: Ok

 Description   

offer! and poll! facilitate subsystems that continue to progress even when their channel partners are not keeping up. This functionality could be done today as a wrapper on alts!!, but can be done more efficiently at the channel level.

  • never blocks
    • usable in all contexts
  • performs a channel operation if possible to do so immediately
    • returns value or nil for poll!
    • returns true or false for offer!
  • initial implementation could use alt!! plus a default
    • this is inefficient
  • better implementation would work directly on channels
    • API that provides a new exit point when enqueue attempt fails


 Comments   
Comment by Alex Miller [ 11/Nov/14 5:00 PM ]

Added offer! and poll! to api per specs above. They call through impl/put! and impl/take! just like >!! and <!!, however I added a new function to the handler protocol: blockable? that is true for all prior handlers (meaning that it is ok for the channel to block them and respond later).

The offer! and poll! calls pass handlers with blockable?=false and do not use a promise. Instead they expect to always receive the value as the result of the impl/put! or impl/take! call.

Inside the channel, the only change made is that when blockable? is false, the put or take is not queued and nil is returned instead. All other cases where the operation can succeed, still succeed.

Because offer! and poll! never block, they are usable both in and out of go blocks. There are tests for both.

The patch does not include the CLJS-equivalent changes.

Comment by Fogus [ 09/Jan/15 4:02 PM ]

Read, tested, REPL'd and understood. This patch looks good to my eyes.

Comment by Alex Miller [ 23/Feb/15 8:59 AM ]

Applied patch





[ASYNC-101] clojure.core.async/reduce doesn't respect reduced Created: 04/Nov/14  Updated: 23/Feb/15  Resolved: 23/Feb/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File async-101.patch    
Patch: Code and Test
Approval: Ok

 Description   

clojure.core.async/reduce does not currently look for or check whether the result of f is reduced.

Patch: async-101.patch



 Comments   
Comment by Fogus [ 09/Jan/15 1:27 PM ]

Works as expected and is a clean impl.

Comment by Alex Miller [ 23/Feb/15 8:30 AM ]

Applied patch.





[ASYNC-114] Allow a second arity on channel ex-handler which takes val as well as throwable Created: 27/Jan/15  Updated: 19/Feb/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Derek Troy-West Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None

Attachments: Text File 0001-Pass-the-val-along-with-the-exception-to-the-channel.patch    

 Description   

When a transducer is applied to a channel and an exception occurs during transformation the ex-handler will be called with
the Throwable as an argument.

Can we also send the val which caused the transformation exception to the ex-handler? When dealing with transducer errors it might be useful to have the full picture.

If that's agreeable, and the solution is a second-arity on ex-handler that takes val and throwable, and the call applied is:

clojure.core.async.impl.channels becomes:

(defn chan
...
(try
(add! buf val)
(catch Throwable t
(handle buf exh t val)))))))))

I would be happy to supply a patch.

Apologies if I have missed something obvious.
Derek



 Comments   
Comment by Nivedita Priyadarshini [ 19/Feb/15 5:56 AM ]

Found the same need when using core.async/pipeline in my code. Wrote a patch of sorts (demonstrative) which allows you to have val in the exh for clj. Happy to write a full patch if this looks okay.





[ASYNC-117] let-bindings unavailable in #js literals in go-blocks Created: 13/Feb/15  Updated: 13/Feb/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Rasmus Erik Voel Jensen Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Environment:

[org.clojure/clojure "1.7.0-alpha5"]
[org.clojure/clojurescript "0.0-2760"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]



 Description   

As far as I can see, let bindings are unavailable in #js-literals within go-blocks, ie:

(go (let [a 1] (js/console.log #js[a])))

prints `[ undefined ]` instead of `[1]`

It happens both with `#js[..]` and `#js{..}`

To make it easier to reproduce the bug, I've created a minimal repository github.com/rasmuserik/cljs-bug with project.clj etc.






[ASYNC-110] 0.1.346, using >! crashes with no method 'setTimeout', when targeting node Created: 23/Dec/14  Updated: 12/Feb/15  Resolved: 12/Feb/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: William Assignee: Unassigned
Resolution: Completed Votes: 3
Labels: bug, nodejs
Environment:

[[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.346.0-17112a-alpha"]
[org.clojure/clojurescript "0.0-2511"]]


Attachments: Text File async-110-exportSymbol.patch     Text File br.log    

 Description   

When targeting node, running a simple >! crashes immediately:

(let [a (chan)]
  (go (>! a :hi))
  (go (println (<! a))))

Crashes with:

goog.global.setTimeout(cb, 0);
   ^
TypeError: Object #<Object> has no method 'setTimeout'

This works fine with 0.1.338.0-5c5012-alpha.



 Comments   
Comment by Joel Wilsson [ 01/Feb/15 3:34 PM ]

The problem is really in the ClojureScript compiler. It does not export those NodeJS global functions. I have a fix that works for me and seems sound, so I submitted a pull request: https://github.com/clojure/clojurescript/pull/45

Comment by David Nolen [ 01/Feb/15 3:47 PM ]

I'm not convinced yet that there are any issues with the ClojureScript compiler nor that your PR represents the solution we should pursue.

Before we even begin assessing any solution first someone needs to identify precisely why the goog.async.nextTick detection doesn't pick up the Node.js globals. The first thing that Google Closure library does is map the global environment to goog.global. I'm curious as to why this isn't working in this case.

Comment by Joel Wilsson [ 01/Feb/15 3:47 PM ]

Patch from the pull request attached.

Comment by Joel Wilsson [ 03/Feb/15 3:49 PM ]

Yes, the first thing the Google Closure library is set up a reference to the global environment with goog.global = this;

However, this is not the global environment in Node.js. From http://nodejs.org/api/vm.html#vm_sandboxes:

the this expression within the global scope of the context evaluates to the empty object ({}) instead of to your sandbox.

If you start a Node.js REPL and enter console.log(this) you will see that this is in fact the global environment. However, if you put that statement in a file and run it with nodejs <filename>, the output will be "{}".

Some possible solutions:

  1. Use goog.exportSymbol to get these functions into goog.global, as in the patch I attached. A drawback of this approach is that it is using goog to fix goog, so it must be done after goog has been loaded.
  2. Explicitly set properties on this before doing anything else. This is the simplest solution. Preamble:
    this.setTimeout = setTimeout;
    this.clearTimeout = clearTimeout;
    this.setInterval = setInterval;
    this.clearInterval = clearInterval;
  3. Use vm.Script.runInThisContext, as bootstrap_node.js does to load base.js with this referencing the global environment found in the variable global.
    A bit tricky in this case, since the file we're executing needs to read itself, but it can be done with this preamble:
    if (!this.__nodeFilename) {
        var vm = require('vm');
        var fs = require('fs');
        global.__nodeFilename = __filename;
        vm.runInThisContext.call(global, fs.readFileSync(__nodeFilename), __nodeFilename);
        process.exit(0);
    }

    For this to work the hashbang must be removed, since #!/usr/bin/env node is not valid JavaScript. Since node is a symlink to ax25-node on Ubuntu, and the Node.js binary is called "nodejs", this might be a good idea either way.
    While this solution is the closest to bootstrap_node.js, it is complex and may be a bit too clever.

  4. Convince the Closure library maintainers to check if base.js is being loaded in Node.js and do goog.global = global; if that is the case.

With 3 and 4, goog.global will contain a lot of Node.js stuff. That may or may not be a good thing. Considering the effort in bootstrap node to hide module and exports, I'm guessing it's a bad thing.

From what I can tell, programs using the Closure library, and which have been compiled by the Closure compiler into a single file, have never worked properly if they have been using anything that needs setTimeout and friends.

Comment by David Nolen [ 03/Feb/15 3:53 PM ]

Joel, many thanks for all the extra information. Will sort through this and think about it.

Comment by David Nolen [ 10/Feb/15 12:00 PM ]

Issue is confirmed on this end. Will look into.

Comment by David Nolen [ 10/Feb/15 12:41 PM ]

So I looked into this a bit, this does not actually appear to be a real issue, at least not yet. The Node.js environment must be bootstrapped for ClojureScript to work. However this process could be simpler and we will address the usability bits in ClojureScript. When it's resolved there and we can confirm via automated testing in core.async we will close this ticket.

Comment by Stuart Mitchell [ 11/Feb/15 8:16 PM ]

Not quite sure what you mean by 'not a real problem' as I have a clojurescript project that targets atom-shell that worked with 0.1.338.0-5c5012-alpha but now doesn't.

Do you mean that its actually a clojurescript issue, if so is there a bug report/ work around?

Comment by Joel Wilsson [ 12/Feb/15 7:58 AM ]

Stuart, could you please try the four lines from option 2 in my previous comment as a preamble and let me know if that solves the problem for you? Ie. put them in a file preamble.js on your project's classpath (I use resources/) and add :preamble "preamble.js" to your :compiler settings. Assuming that you are using lein-cljsbuild, that is.

This is a real issue for me as well, because it stops me from using core.async on AWS Lambda.

Comment by David Nolen [ 12/Feb/15 8:10 AM ]

Sorry Stuart I'm not saying it's not real a issue but rather that I'm trying to determine the scope and precise nature of the issue. I noticed that the problem does not manifest when I use :optimizations :none. Are you and Joel using :simple, :advanced? Thanks.

Comment by Joel Wilsson [ 12/Feb/15 8:23 AM ]

I use :simple to get a single JavaScript file, which I can then use with AWS Lambda with this in main.js:

var r = require("./clojurescript-output.js");
exports.handler = r.hello_lambda.core.handler;

where clojurescript-output.js is generated from a file like

(ns hello_lambda.core)
(enable-console-print!)
(defn ^:export handler [event context]
  (println "Hello AWS Lambda")
  (.done context))

:advanced doesn't work for me: it just creates a JavaScript file with a single (function () {...})(), which makes it useless for creating a module (the symbols are not exported properly), but that's another issue.

Comment by David Nolen [ 12/Feb/15 9:11 AM ]

this was in fact an upstream ClojureScript issue, fixed as of this commit https://github.com/clojure/clojurescript/commit/e8ae2c8a9c3f2429555e136cc17c1669be5e993a





[ASYNC-116] Convert core.async ClojureScript tests to cljs.test Created: 12/Feb/15  Updated: 12/Feb/15

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: David Nolen Assignee: David Nolen
Resolution: Unresolved Votes: 1
Labels: None


 Description   

This is a good dog-fooding opportunity for cljs.test's async testing functionality.






[ASYNC-115] Node.js target should support :main Created: 10/Feb/15  Updated: 10/Feb/15  Resolved: 10/Feb/15

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: David Nolen Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Comments   
Comment by David Nolen [ 10/Feb/15 12:46 PM ]

wrong place.





[ASYNC-112] Change wording from transformer to transducer in documentation of deprecated functions Created: 30/Dec/14  Updated: 01/Jan/15  Resolved: 01/Jan/15

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Erik Assum Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Approval: Ok

 Description   

In the documentation for the deprecated functions in core.async, the wording "transformer" is used instead of "transducer", this should be fixed.

A fix is implemented in https://github.com/clojure/core.async/pull/46



 Comments   
Comment by Alex Miller [ 01/Jan/15 10:07 AM ]

Updated messages to s/transformer/transducer/





[ASYNC-109] Clarify timeout doc to mention that close! should not be called on a timeout channel Created: 11/Dec/14  Updated: 30/Dec/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Ryan Neufeld Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: documentation

Attachments: Text File clarify-timeout-doc.patch    
Patch: Code

 Description   

After running into a race-condition involving closed timeout channels, it seems like it would be appropriate to mention that `close!` should never be called on a timeout channel in its docstring. The attached patch tweaks the doc string to that effect. Please advise if you'd like the wording changed a bit.



 Comments   
Comment by Howard Lewis Ship [ 12/Dec/14 11:03 AM ]

Alternately/additionally, it would be nice if close! on a timeout channel would throw an exception.

Comment by Erik Assum [ 30/Dec/14 8:25 AM ]

or alternately, make it a no-op?





[ASYNC-111] maps are represented different inside and outside of go blocks Created: 27/Dec/14  Updated: 27/Dec/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Sven Richter Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

clojure 1.6.0, clojurescript "0.0-2371", W7, Oracle JDK 8



 Description   

When initializing a state atom with a set "conjuring" a map behaves differently depending of the scope where it happens:

(def state (atom {:transformations []}))
(reset! state {:transformations #{:foo "bar"}})

; throws: Uncaught Error: compare on non-nil objects of different types
(...(go ;some async stuff
(swap! state conj {:name "baz"}))...)

; works
(...(go ;some async stuff)
(swap! state conj {:name "baz"}))

I am not sure if this is a defect at all as maps are not comparable as Tom in the thread stated. However, the fact that it works in one case and does not in the other is weird.
You can find the complete thread here: https://groups.google.com/forum/#!topic/clojurescript/lmvltpa3k8s






[ASYNC-108] cljs to-chan hangs on infinite lazy sequences Created: 20/Nov/14  Updated: 20/Nov/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Felix Andrews Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: cljs
Environment:

[org.clojure/core.async "0.1.346.0-17112a-alpha"]


Patch: Code

 Description   

e.g. this hangs the browser:

(go-loop [ch (to-chan (range))]
(println (<! ch))
(<! (timeout 1000))
(recur ch))

The problem is in
https://github.com/clojure/core.async/blob/53bf7866f195e6ba247ff7122b99784e66e9f1bb/src/main/clojure/cljs/core/async.cljs#L362
where the order of arguments to bounded-count is wrong.






[ASYNC-70] documentation of thread macro should include behavior of nil (closes the channel) Created: 15/May/14  Updated: 14/Nov/14

Status: Reopened
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: reader
Environment:

0.1.278.0


Attachments: Text File async-70-patch.1.txt    
Patch: Code

 Description   

I needed the ability to invoke some code in a thread, and have the channel close if nil was returned.

Digging though the code, I discovered it already does this, but it is no documented in the docstring.

I'll supply a patch shortly.



 Comments   
Comment by Alex Miller [ 23/Jun/14 11:15 AM ]

The thread docstring says: "Returns a channel which will receive the result of the body when completed." The special case of a null return value is actually handled by ignoring it (because you are not allowed to explicitly put nils on a channel). The channel is closed on completion regardless. I'm not sure I get what needs to be added here and no patch, so closing. Reopen if there is a concrete suggestion to evaluate.

Comment by Howard Lewis Ship [ 14/Nov/14 10:41 AM ]

It's a tiny little patch. DRY is great for code, less so for docs!





[ASYNC-107] Improved docstring for alt! Created: 10/Nov/14  Updated: 10/Nov/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: docstring

Attachments: Text File core-async-alt-doc-1.patch    
Patch: Code

 Description   

I had a little remembering exactly how alt! is used, and put together some better documentation that more clearly identifies the differences between taking from and putting to channels.






[ASYNC-105] Remove or correct incorrect type tag on MAX-QUEUE-SIZE Created: 10/Nov/14  Updated: 10/Nov/14  Resolved: 10/Nov/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Andy Fingerhut Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None


 Description   

In this line in file src/main/clojure/clojure/core/async/impl/protocols.clj:

(def ^:const ^int MAX-QUEUE-SIZE 1024)

the int type tag is interpreted by the Clojure compiler as the function clojure.core/int, and thus does not help in avoiding reflection. Suggest removing it, or changing that line to have a correct type tag as follows:

(def ^:const ^{:tag 'int} MAX-QUEUE-SIZE 1024)

Found using Eastwood linter (latest released version 0.1.5 looks for incorrect type tags like this).



 Comments   
Comment by Alex Miller [ 10/Nov/14 12:38 PM ]

Dupe of ASYNC-54





[ASYNC-68] Improve error handling for map< and map> Created: 09/May/14  Updated: 04/Nov/14  Resolved: 04/Nov/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Timothy Baldridge Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Approval: Vetted

 Description   

How can we deal with error handling better. For example, map< will throw an exception in the taker's thread if the function throws an exception. map> throws an exception on the put!. So this can crash an entire chain of ops. For example a pipe into map> will crash the pipe go to crash if the map function throws an error. We can add try/catch to all of this, but what do we do with the exceptions, and how do we recover?



 Comments   
Comment by Timothy Baldridge [ 14/May/14 12:25 PM ]

<Comment Removed> I misunderstood. But the original problem still stands.

Comment by Alex Miller [ 04/Nov/14 4:08 PM ]

Won't fix - these are deprecated and going away.





[ASYNC-57] reify in go macro compile error Created: 20/Feb/14  Updated: 04/Nov/14  Resolved: 04/Nov/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Lijin Liu Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.5.1"]
[org.clojure/core.async "0.1.267.0-0d7780-alpha"]



 Description   

(go
(reify java.util.Collection
(add [this o]
(println o)
(println this)
true)))

clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: Unable to resolve symbol: this in this context



 Comments   
Comment by Ghadi Shayban [ 22/Apr/14 9:55 AM ]

Reassigning to minor. The go macro will obviously not rewrite parking ops inside the reify. Lifting the reify into a var is a decent workaround.

Comment by Alex Miller [ 04/Nov/14 4:02 PM ]

Not supported.





[ASYNC-83] Taking from channels in anonymous functions inside go-loops does not work Created: 05/Aug/14  Updated: 04/Nov/14  Resolved: 04/Nov/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Joel Knighton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

latest core.async release, latest core.async built from source



 Description   

This seems to be related/comparable to the issue posted on this mailing-list thread:
https://groups.google.com/forum/#!topic/clojure/vtvk7warLE4

Minimal testcases to reproduce:

(def ch (chan))

(go-loop []
(>! ch "test"))

(go-loop []
(println (<! ch))) ;;prints "test"

(future (println (<!! ch))) ;;prints "test"

(future (println (#(<!! ch)))) ;;prints "test"

(go-loop []
(println (#(<! ch)))) ;;CompilerExpection java.lang.IllegalArgumentException:
;; No method in multimethod '-item-to-ssa' for dispatch
;; value :fn



 Comments   
Comment by Joel Knighton [ 05/Aug/14 12:41 PM ]

It appears the go-loop isn't even necessary -
(go (println (#(<! ch)))) reproduces the issue.

Comment by Kevin Downey [ 05/Aug/14 1:39 PM ]

I think this is expected behaviour, the go transformation macro doesn't attempt to descending in to called functions to transform their bodies, since their bodies are not transformed the <! and >! operators, which do not work in untransformed code, do not work.

the following which explicitly transforms the body of the inner function using the go macro should work:

(go (println (#(go (<! ch)))))

it will print out a channel though, because the result of (go ...) is a channel that will contain the result of ...

Comment by Joel Knighton [ 05/Aug/14 1:45 PM ]

Okay - thanks for the feedback. This doesn't cause an issue for me, and your explanation makes sense.

Comment by Alex Miller [ 04/Nov/14 4:01 PM ]

Agreed that go macro does not descend into function creation (fn or #() forms). The error could be better in this case but that's logged in a separate ticket.





[ASYNC-92] go macro removes binding forms that are intialized with logical false value Created: 03/Oct/14  Updated: 30/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Oleh Palianytsia Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Environment:

org.clojure/core.async "0.1.346.0-17112a-alpha"


Attachments: File fix-async-92.diff    
Approval: Triaged

 Description   
(require '[clojure.core.async :as a])

(a/go (let [a nil] (a/alts! (if a <whatever> <whatever>)))) // Unable to resolve a
(a/go (let [a nil] (a/<! (if a <whatever> <whatever>))) // Unable to resolve a

Seems that 'go' macro removes falsely initialized symbols that are used as channels, because
in both cases there's exception, that says " Unable to resolve symbol: a in this context".



 Comments   
Comment by Willy Blandin [ 17/Oct/14 12:19 PM ]

Confirmed.
Bug was introduced between 0.1.278.0-76b25b-alpha and 0.1.295.0-9ea6ef-alpha.

Comment by Willy Blandin [ 17/Oct/14 12:27 PM ]

Worked around with:

(defmacro workaround-async-92
  "Hack to workaround core.async bug
   cf. http://dev.clojure.org/jira/browse/ASYNC-92"
  []
  ;; has to be a list
  `(do nil))

(let [a (workaround-async-92)]
  ...)
Comment by Leon Grapenthin [ 23/Oct/14 11:55 AM ]

modifies two methods of the RawCode inst so that they check:collected-locals in locals via contains? before ignoring them

Comment by Ghadi Shayban [ 23/Oct/14 5:19 PM ]

Hi Leon, thanks for the patch. Can you fill out a Contributor Agreement? http://clojure.org/contributing

Comment by Leon Grapenthin [ 24/Oct/14 7:17 AM ]

I did, yesterday. Got an automatic confirmation email saying Rich Hickey signed it. Anything else I should do with it?





[ASYNC-64] Race condition when closing mults Created: 29/Apr/14  Updated: 16/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: James Reeves Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: mult

Approval: Triaged

 Description   

When a mult is tapped at around the same time as the source channel is closed, the tapped channel may not be closed.

(require '[clojure.core.async :refer (chan mult tap close!)])
(let [s (chan)
      m (mult s)
      c (chan)]
  (tap m c)
  (close! s)
  (impl/closed? c))

The above code will sometimes return true, and sometimes return false.

Cause: This is caused by the following code in the mult function:

(if (nil? val)
  (doseq [[c close?] @cs]
    (when close? (close! c)))

Any channels tapped after cs is dereferenced will not be closed.

Approach: A possible solution to this could be to always close channels tapped to a closed source. i.e.

(let [s (chan)
      m (mult s)
      c (chan)]
  (close! s)
  (tap m c))  ;; will always close c

This could be achieved by adding a flag to the cs atom to denote whether the mult is open or closed. If it's closed, any tapped channel is closed automatically.



 Comments   
Comment by James Reeves [ 30/Apr/14 6:05 AM ]

For reference, below is the custom fix for mult I'm using:

(defn mult [ch]
  (let [state (atom [true {}])
        m (reify
            Mux
            (muxch* [_] ch)
            Mult
            (tap* [_ ch close?]
              (let [add-ch    (fn [[o? cs]] [o? (if o? (assoc cs ch close?) cs)])
                    [open? _] (swap! state add-ch)]
                (when-not open? (close! ch))
                nil))
            (untap* [_ ch]
              (swap! state (fn [[open? cs]] [open? (dissoc cs ch)]))
              nil)
            (untap-all* [_]
              (swap! state (fn [[open? _]] [open? {}]))))
        dchan (chan 1)
        dctr (atom nil)
        done (fn [_] (when (zero? (swap! dctr dec))
                       (put! dchan true)))]
    (go-loop []
      (let [val (<! ch)]
        (if (nil? val)
          (let [[_ cs] (swap! state (fn [[_ cs]] [false cs]))]
            (doseq [[c close?] cs]
              (when close? (close! c))))
          (let [chs (keys (second @state))]
            (reset! dctr (count chs))
            (doseq [c chs]
              (when-not (put! c val done)
                (swap! dctr dec)
                (untap* m c)))
            (when (seq chs)
              (<! dchan))
            (recur)))))
    m))
Comment by David Nolen [ 14/Oct/14 6:10 AM ]

Is this also fixed in master? Thanks.

Comment by Ghadi Shayban [ 15/Oct/14 11:09 PM ]

I understand the scenario, but honestly I'm not sure this is a bug in mult or the usage. A channel shouldn't be expected to always yield a take. The consumer of the "late tap" can guard against it with alts or some other mechanism, and also you can enforce a no-late-taps through a policy on the "production" side of things.

Rich Hickey can you weigh in?

Comment by James Reeves [ 16/Oct/14 3:51 AM ]

The "tap" function currently has an explicit "close?" flag, and if a tapped channel isn't guaranteed to close when the source channel closes, that argument probably shouldn't exist. Also, if auto-closing taps is taken out, should we remove the "close?" argument on "sub" as well?

Comment by Ghadi Shayban [ 16/Oct/14 11:34 AM ]

It's more than respecting the flag. Related to the close behavior, channels can tap and untap without receiving anything while the mult process happily distributes a value to another set of channels (like the ABA problem). Could also make it an error to tap after the close is distributed to the last deref'ed set of channels. That is different than the familiar permanent nil receive, but mults already differ from simple channels.





[ASYNC-95] Tranducers and unbuffered channels Created: 12/Oct/14  Updated: 16/Oct/14  Resolved: 16/Oct/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Nguyễn Tuấn Anh Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

Currently buffered channels are required to use transducers.

Is this a limitation of the current implementation, or was it a design decision?

In the Javascript port of core.async (js-csp), we have been experimenting with a different approach:

  • The step function takes the whole channel as its "result" parameter, instead of just the buffer. It also handles committing pending active takes. This allows values to go straight from puts to takes when needed (subjecting to the transformation), allowing unbuffered channels to use transducers.
  • To handle "expanding" transducers like "cat", a separate overflow buffer is used instead of allowing the fixed buffer to grow on overflow.

In short, "add" would do this:

  • Commit the first pending active taker it finds, giving it the value
  • Put the value into the normal buffer if there is one, and if it is not full
  • Put the value into the overflow buffer

We haven't thoroughly tested this approach yet, but it seems like it would work well, except for the overhead of an overflow buffer for every channel, which can be fixed by creating it on-demand.

This is the discussion we had: https://github.com/ubolonton/js-csp/issues/7#issuecomment-57940050
Is this approach reasonable? Are there other potential pitfalls we missed?



 Comments   
Comment by David Nolen [ 14/Oct/14 5:54 AM ]

As far as I know it was a design decision. Given that I'm not sure what the purpose of this ticket is. Questions about implementation are probably best discussed on the clojure-dev Google Group before creating tickets. I'm inclined to close this unless a real problem is specified.

Comment by Nguyễn Tuấn Anh [ 16/Oct/14 11:19 AM ]

Thanks. I'm unfamiliar with the workflow. I'm going to post this to that group.





[ASYNC-93] can't use fn inside go macro Created: 05/Oct/14  Updated: 16/Oct/14  Resolved: 15/Oct/14

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alexei Guevara Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

osx 10.9.2 / java 1.7.0_51-b13 / clojure 1.6



 Description   

Can't compile this expression

(ns example
  (:require [clojure.core.async :refer [chan go >! put!]]))

(let [ch (chan)]
  (go (fn [] (>! ch 1))))

I'm getting this error

clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No method in multimethod '-item-to-ssa' for dispatch value: :fn, compiling:(.../example.clj:5:1)

Whereas this expression compiles with no issues

(let [ch (chan)]
  (go (fn [] (put! ch 1))))

I reduced the original code where I found the issue to the first expression.



 Comments   
Comment by Kevin Downey [ 14/Oct/14 3:37 PM ]

the go macro does not descend in to and transform functions. >! only works inside the go macro, so regardless, the first example is bad code, so it should throw some kind of error. I am not sure that error shown is the correct error it should throw, or why the second example doesn't throw. the ticket doesn't mention what version of core.async is being used

Comment by Alexei Guevara [ 14/Oct/14 3:41 PM ]

I was using core.async v0.1.346.0-17112a-alpha.

Comment by Nicola Mometto [ 14/Oct/14 5:37 PM ]

The second example doesn't throw because `go` will only transform forms that are, or contain in their sub-forms, channel operations for go blocks (>!, <!, alts!), put! is not one of those so the go macro won't transform the expression.

Comment by Ghadi Shayban [ 15/Oct/14 10:43 PM ]

Kevin and Nicola are correct, the go macro stops transformation at function boundaries, and and only transform alts! >! and <!

Comment by Alexei Guevara [ 15/Oct/14 11:11 PM ]

Not transforming function definitions inside the go macro makes perfect sense. But, it will also make sense to fail with a message more meaningful than "No method in multimethod '-item-to-ssa' for dispatch value".

Comment by Ghadi Shayban [ 15/Oct/14 11:20 PM ]

Agreed, created ASYNC-98.





[ASYNC-97] CLJS: alts! sporadically failing to unblock despite channel activity (Safari 7) Created: 15/Oct/14  Updated: 16/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Oliver Charles Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File gistfile1.txt     File main.cljs     File socket.cljs    

 Description   

Hi all, bit of a tricky bug to report here... We're seeing some problems with using core.async in ClojureScript on Safari 7. Our application is built around a large event loop that blocks on a message from one of many channels that correspond to user activity or API calls. The problem seems to lie within this event loop - we are using alts! to pull a message out of any available channel, but sometimes logging shows that we reach alts! and never unblock. However, with a little more logging, I can see that there are subsequent writes to one of the channels in the list of channels passed to alts!, so I'm not really sure what's going on.

That's the high level overview, now on to some code.

Our main event loop is as follows:

(log "Entering main event loop.")
  (go
    (while true
      (log "alts! channel hashes: " (map hash (:channels @app)))
      (let [[message channel] (alts! (seq (:channels @app)))]
        (log "alts! unblocked, calling our process-message")
        (swap! app process-message message channel)
        (log "process-message completed, looping"))))

process-message here is our a function internal to our application, but I don't think it's details are necessarily important. In the scenario where Safari gets stuck, the log looks like:

[Log] process-message completed, looping (main.js, line 62)
[Log] alts! channel hashes:  (16 12 19 33) (main.js, line 82)
[Log] Socket connected. (socket.js, line 309)
[Log] put! to channel with hash  19 (socket.js, line 86)
[Log] The message is [:metronome [:staff [{:description nil, :deletable true, :email nil, :isAdmin true, :isTrainer false, :telephone nil, :name "Fynder Admin", :picture nil, :userId 1} {:description nil, :deletable fa... (socket.js, line 87)
[Log] put! callback gave us true (socket.js, line 89)
[Debug] Metronome: staff data decoded. put! complete.: 12.282ms (socket.js, line 93)
[Log] put! to channel with hash  19 (socket.js, line 86)
[Log] The message is [:metronome [:class-types [{:deletable false, :picture nil, :name "CycleCore", :id 2, :description "CycleCore is a 55-minute dual workout concept that combines 30 minutes of intense cardiovascular ... (socket.js, line 87)
[Log] put! callback gave us true (socket.js, line 89)
[Debug] Metronome: class-types data decoded. put! complete.: 1.288ms (socket.js, line 93)
[Log] put! to channel with hash  19 (socket.js, line 86)
[Log] The message is [:metronome [:locations [{:studios [{:deletable false, :name "Kensington", :id 1, :locationId 1, :description "Studio (11a) sits just off Stratford Road in Stratford Studios. To find us, just pass ... (socket.js, line 87)
[Debug] Metronome: locations data decoded. put! complete.: 0.884ms (socket.js, line 93)

Note that we see a log entry for "alts! channel hashes", but we never seen "alts! unblocked". However, note the list of hashes passed to alts!. Channel 19 is mentioned, but subsequently we put! to channel 19... yet we still don't get unblocked. Something that also strikes me as suspicious, is that while we're blocked at alts!, two calls to put! have succeeded immediately, for a channel that is bounded to contain only one element at a time. Maybe I'm misunderstanding something, but I wouldn't expect the immediate-put callback to be invoked more than once. Interestingly that last put! doesn't invoke the callback.

Unfortunately, reproduction of this bug is reasonably difficult. I can somewhat reliably reproduce it by quitting Safari, re-opening it, and navigating to the dev server. About 1 in 15 attempts get stuck in this fashion. I wondered if it was something to do with Safari's MessageChannel implementation - you can see in the log entries where nexttick.js calls its callback, which seems to be how dispatch is working in my browser.

I'd be very happy to help provide any more information that's useful, but this problem is now outside my ability to debug it. While the code is proprietary, I'd be happy to temporarily add people to the Github project in order to try and get this fixed. We have development APIs servers that you can point at, so it should be just a case of running lein cljs.

I've attached our code for our Socket.io wrapper and our main event loop. Sadly I do not yet have a minimal test-case - I wouldn't really know where to begin.



 Comments   
Comment by Oliver Charles [ 15/Oct/14 7:37 AM ]

I went deep into the guts of the Google Closure library and changed getSetImmediateEmulator_ to:

goog.async.nextTick.getSetImmediateEmulator_ = function() {
  // Fall back to setTimeout with 0. In browsers this creates a delay of 5ms
  // or more.
  return function(cb) {
    goog.global.setTimeout(cb, 0);
  };
};

and I haven't been able to get it stuck. So maybe MessageChannel has problems in Safari...

Comment by Ghadi Shayban [ 15/Oct/14 10:40 PM ]

Hi Oliver, seems like a race, and we'll figure this out.

Would you mind compare running upon 0.1.319.0-6b1aca-alpha vs 0.1.346.0-17112a-alpha ?

alts! should be passed an indexed collection/vector btw. Passing a seq wouldn't cause this bug, just something to note.

Comment by Oliver Charles [ 16/Oct/14 6:53 AM ]

Hi Ghadi,

0.1.319.0-6b1aca-alpha is what the initial report was against - I should have mentioned that. So 0.1.319.0-6b1aca-alpha does get stuck.

0.1.346.0-17112a-alpha however does not get stuck, which is odd - as I'm sure I tried upgrading to this! I've tried on two Macs that are normally problematic, and they didn't get stuck once. I'm pushing this out to more of our testers and will see what happens.

Comment by Oliver Charles [ 16/Oct/14 7:16 AM ]

Aha, I knew it wouldn't be that easy! Upon releasing this to production, it immediately froze again. The dev server runs with very different optimisations though, so I'm going to build a production release and serve that locally - will see what happens there.

Comment by Oliver Charles [ 16/Oct/14 7:28 AM ]

Yep, definitely a problem with optimisations. Here is my Shadow Build configuration

(ns fynder.shadowbuild
  (:require [clojure.java.io :as io]
            [shadow.cljs.build :as cljs]))

(defn define-modules [state]
  (-> state
      (cljs/step-configure-module :cljs '[cljs.core clojure.walk clojure.string cljs.reader cljs.core.async] #{})
      (cljs/step-configure-module :test-support '[inflections.core no.en.core enfocus.bind fynder.winchan] #{:cljs})
      (cljs/step-configure-module :devel '[fynder.devel] #{:cljs})
      (cljs/step-configure-module :admin '[fynder-admin.main] #{:cljs})
      (cljs/step-configure-module :trainer '[fynder-trainer.main] #{:cljs})
      (cljs/step-configure-module :mobile '[fynder-mobile.main] #{:cljs})
      (cljs/step-configure-module :sweatybetty '[fynder-sweatybetty.main] #{:cljs})
      (cljs/step-configure-module :loader '[fynder-loader.loader] #{:cljs})))

(defn dev
  "build the project, wait for file changes, repeat"
  [& args]
  (let [state (-> (cljs/init-state)
                  (cljs/enable-source-maps)
                  (assoc :optimizations :advanced
                         :pretty-print false
                         :work-dir (io/file "target/cljs-work")
                         :public-dir (io/file "resources/dev")
                         :public-path "")
                  (cljs/step-find-resources-in-jars)
                  (cljs/step-find-resources "src/cljs/")
                  (cljs/step-finalize-config)
                  (cljs/step-compile-core)
                  (define-modules))]
    ;; compile, flush, reload, repeat
    (loop [state state]
      (let [new-state (try
                        (-> state
                            (cljs/step-compile-modules)
                            (cljs/flush-unoptimized)
                            (cljs/wait-and-reload!))
                        (catch Throwable t
                          (prn [:failed-to-compile t])
                          (.printStackTrace t)
                          (cljs/wait-and-reload! state)))]
        (recur new-state)))))

(defn prod
  "build the project, wait for file changes, repeat"
  [& args]
  (-> (cljs/init-state)
      (cljs/enable-emit-constants)
      (cljs/enable-source-maps)
      (assoc :optimizations :advanced
             :pretty-print false
             :work-dir (io/file "target/cljs-work")
             :public-dir (io/file "resources/prod")
             :externs ["react/externs/react.js"
                       "externs/base32.js"
                       "externs/jquery.js"
                       "externs/react_addons.js"
                       "externs/fastclick.js"
                       "externs/socket.io.js"
                       "externs/moment.js"
                       "externs/papa.js"
                       "externs/markdown.js"
                       "externs/xss.js"
                       "externs/facebook.js"
                       "externs/checkout.js"])
      (cljs/step-find-resources-in-jars)
      (cljs/step-find-resources "src/cljs/")
      (cljs/step-finalize-config)
      (cljs/step-compile-core)
      (define-modules)
      (cljs/step-compile-modules)
      (cljs/closure-optimize)
      (cljs/flush-to-disk)
      (cljs/flush-modules-to-disk)))

If I run in the dev profile, then I can't get it stuck. If I switch over to the production profile (and serve the result of lein publish with python's SimpleHTTPServer), then Safari does get stuck.





[ASYNC-98] Less hostile message for #'go stopping at (fn [] ) boundaries Created: 15/Oct/14  Updated: 15/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Ghadi Shayban Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: errormsgs


 Description   

ASYNC-93 has an example of a not nice message when #'go attempts to shred through a (fn []) form. Maybe we can improve this, but then again creating a (fn []) in a block should be permitted, though the inner contents of it will not be transformed.






[ASYNC-51] Core.async memory leak in Safari and Mobile Safari Created: 21/Jan/14  Updated: 15/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Bruce Hauman Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Mobile Safari on iOS 7.0; Safari 6.1 on OSX Lion



 Description   

Chaining together channels apparently leaks memory in Safari.

A statement such as

(def test [input-chan]
  (map< identity (map< identity (map< identity input-chan))))

will leak memory when the channels are used. The longer the chain the more memory.

I have created an example repository and an example page.

The repo and the example code is here:
https://github.com/bhauman/checkmemleak/blob/master/src/checking_safari_leak/core.cljs

The demonstration page is here:
This link will leak
http://rigsomelight.com/checkmemleak/?leak=true
This link will not
http://rigsomelight.com/checkmemleak

The leak is pretty darn severe.



 Comments   
Comment by Bruce Hauman [ 21/Jan/14 4:55 PM ]

I have just confirmed this is also a problem in Safari Version 7.0.1 (9537.73.11) on Mavericks.

Comment by Bruce Hauman [ 23/Jan/14 7:13 PM ]

I updated the example page.

http://rigsomelight.com/checkmemleak/index.html

This link wont leak:
http://rigsomelight.com/checkmemleak/index.html#comp-partial

This link will leak:
http://rigsomelight.com/checkmemleak/index.html#map<

There is navigation to try the different implementations and optimization modes.

To be clear map<-chain, custom-map, map<, and map> all exhibit the leak in :none and simple optimization modes.

Comment by David Nolen [ 14/Oct/14 6:00 AM ]

To make sure this ticket has all the necessary information - if I remember correctly even advanced optimization is affected? Am I correct Bruce?

Comment by Bruce Hauman [ 15/Oct/14 12:53 PM ]

Yes, it affected advanced optimization. I haven't checked if this is still an issue.





[ASYNC-42] (try (catch :default)) inside a cljs go block raises an exception Created: 07/Dec/13  Updated: 14/Oct/14  Resolved: 14/Oct/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Travis Vachon Assignee: Unassigned
Resolution: Completed Votes: 2
Labels: cljs

Attachments: Text File async_42.patch     Text File async_42.v2.patch     Text File try-catch-default.patch    

 Description   

test demonstrating issue and patch fixing it here: https://github.com/clojure/core.async/pull/39



 Comments   
Comment by Travis Vachon [ 08/Dec/13 8:51 PM ]

code + test

Comment by Travis Vachon [ 10/Dec/13 9:53 AM ]

add {{git am}}able patch

Comment by Yehonathan Sharvit [ 24/Aug/14 12:56 AM ]

What is the status of this issue?
Why isn't it merged into core.async code?

Comment by David Nolen [ 14/Oct/14 6:13 AM ]

Can we get an updated patch? Thanks.

Comment by Travis Vachon [ 14/Oct/14 2:41 PM ]

updated patch

Comment by David Nolen [ 14/Oct/14 7:33 PM ]

fixed https://github.com/clojure/core.async/commit/bd71ba73b43ea09e242d37a54eaaa0fed761504c





[ASYNC-72] CLJS: mult becomes blocked after putting to a closed channel Created: 15/May/14  Updated: 14/Oct/14  Resolved: 14/Oct/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Logan Linn Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: Text File cljs_mult_dec.patch    
Patch: Code

 Description   

A mult can become blocked after a putting to a closed tap channel due to an off-by-one error in triggering the end condition.

It appears the issue was recently fixed in the Clojure version of mult, but not the ClojureScript version.
https://github.com/clojure/core.async/commit/8864214ed9fa9aa8dc12297c6cf99ad0be65a6ae

The issue occurs because the counter that indicates the mult has finished a putting to all taps, dctr is decremented out of band and the zero? triggering condition in done function is missed.
https://github.com/clojure/core.async/blob/65b2d6c81350ba8b4bb3a8b82ad45ba984c0037c/src/main/clojure/cljs/core/async.cljs#L457

The done function passed to put! appears to be called regardless of whether the channel is closed. Because of this, we don't need to (swap! dctr dec) or necessarily call (done nil) (the CLJ fix) to fix this issue.

Demonstration: https://gist.github.com/loganlinn/e840139a9ab5503970fd



 Comments   
Comment by Gijs Stuurman [ 06/Jun/14 11:03 AM ]

The title of this issue says it's CLJS only, but the double done/counter-decrement applies to the CLJ version as well. As is briefly mentioned at the end of this issue.

For closed channels the done function is called twice in mult, because put! returns false for closed channels and calls the callback:
https://github.com/clojure/core.async/blob/65b2d6c81350ba8b4bb3a8b82ad45ba984c0037c/src/main/clojure/clojure/core/async.clj#L684

In the CLJ version this will not lead to blocking but it violates the behavior in the docstring of "each tap must accept before the next item is distributed" when a tap is closed.

Comment by David Nolen [ 14/Oct/14 6:09 AM ]

As far as I can tell the original issue has been fixed in master?

Comment by Logan Linn [ 14/Oct/14 12:02 PM ]

Yeah. It appears https://github.com/clojure/core.async/commit/40ab7edacac747af1108c80223039e8a0b1c2ca9 resolved the issue. Thanks!





[ASYNC-96] FixedBuffer's full checking in cljs Created: 13/Oct/14  Updated: 13/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Nguyễn Tuấn Anh Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File cljs-fixed-buffer-overflow.patch    
Patch: Code and Test

 Description   

In the cljs implementation of fixed buffer, "full?" uses "=" instead of ">=". This makes it incorrect after an overflow.






[ASYNC-67] Can we get a generic sink operation? Created: 09/May/14  Updated: 07/Oct/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Ghadi Shayban Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None

Approval: Triaged

 Description   

Can we get a generic sink operation?

(defn sink
 "Fully consumes and discards a channel, applying f to each val. Returns a channel that
  closes when ch is fully drained"
 [f ch]
 (go-loop []
   (when-some [v (<! ch))]
     (f v)
     (recur))))





[ASYNC-91] 'and' does not short circuit within go block in clojurescript Created: 14/Sep/14  Updated: 25/Sep/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jan Krueger Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

core.async 0.1.338.0-5c5012-alpha



 Description   

I have the following piece of ClojureScript code within a go block:

(cond
(and (vector? x) (= (first x) :some-key)) ...)

This generates the following piece of JavaScript code for the cond case:

if (40 === f) { return d = b[9], f = cljs.core.vector_QMARK_.call(null, d), d = cljs.core.first.call(null, d), d = cljs.core._EQ_.call(null, d, new cljs.core.Keyword(null, "some-key", "some-key", -978969032)), cljs.core.truth_(f && d) ? b[1] = 42 : b[1] = 43, new cljs.core.Keyword(null, "recur", "recur", -437573268); }

This looks to me like both and arguments would actually get evaluated. As a result my code crashes whenever it hits this cond case and 'x' is not seqable.



 Comments   
Comment by Francis Avila [ 25/Sep/14 2:52 AM ]

CLJS-864 is likely the same issue. (It also has a small test project to reproduce.)





[ASYNC-59] Channel returned by cljs.core.async/map> is missing protocol method Channel.closed? Created: 08/Mar/14  Updated: 10/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Kevin Neaton Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: bug, patch
Environment:

[org.clojure/clojure "1.5.1"]
[org.clojure/clojurescript "0.0-2173"]
[org.clojure/core.async "0.1.278.0-76b25b-alpha"]


Attachments: Text File 0001-Fixed-map-by-including-impl.-for-closed.patch    
Patch: Code

 Description   

E.g.

(let [ch (->> (chan) (map> inc) (filter> even?))]
  (doseq [i (range 10)] (put! ch i)))

When filter> checks to see if the channel returned by map> is closed?, this code fails because the channel returned by map> does not implement the Channel.closed? protocol method.



 Comments   
Comment by Alex Miller [ 02/Sep/14 9:54 AM ]

map> and filter> have been deprecated and will be removed in a future release. They have been replaced with applying transducers to a channel which is now available.

Comment by Kevin Neaton [ 10/Sep/14 9:08 AM ]

Great, thanks for the update.





[ASYNC-88] Add Sonatype repository info to README for unreleased library Created: 27/Aug/14  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Bridget Hillyer Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: documentation


 Description   

It could be confusing for those not very familiar with Leiningen, Maven, etc. that there is a separate way to obtain SNAPSHOT versions of libraries vs. Released versions of libraries.

I believe an easy fix would be to have the following information directly in the README of unreleased contrib libraries (of which core.async is one):

To use Clojure and contrib library SNAPSHOTS in your Leiningen project, add the following to your project.clj:

:repositories {"sonatype-oss-public" "https://oss.sonatype.org/content/groups/public/"}



 Comments   
Comment by Alex Miller [ 02/Sep/14 9:50 AM ]

Done.

Comment by Alex Miller [ 02/Sep/14 10:02 AM ]

Actually, I have un-done this for core.async as we don't ever release SNAPSHOT versions of core.async, so it's thus a bit confusing. The canonical place we point people for this repo setup info is:

http://dev.clojure.org/display/community/Maven+Settings+and+Repositories

Comment by Bridget Hillyer [ 02/Sep/14 10:11 AM ]

I realized that this does not affect core.async after I made this ticket. Maybe something was down that day so I couldn't get the canonical release? So, right, this is not valid for core.async. But it would be for other (contrib?) libraries that do get SNAPSHOTs released.

The setup info that you pointed out:
http://dev.clojure.org/display/community/Maven+Settings+and+Repositories
would probably be useful from a usability standpoint in the READMEs themselves, including core.async.





[ASYNC-50] Add bang(!) to the names of channel ops with side effects Created: 16/Jan/14  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Jozef Wagner Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

Some channel ops immediatelly change the given channel, thus they have side effects. This include async/map, filter<, remove<, mapcat<, pipe, split, reduce, onto-chan, etc. All 'mutating' basic channel operations have bang at the end of their name (<Unable to render embedded object: File (, >) not found., alt!, ...). If we want to be consistent with this, bang should be added to every channel function which has side effects.



 Comments   
Comment by Alex Miller [ 02/Sep/14 9:56 AM ]

Rich named these and there are no plans to change them.





[ASYNC-38] keep</> instead of map</> Created: 18/Nov/13  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Leon Grapenthin Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: documentation, enhancement, errormsgs


 Description   

One problem with using map< is when the supplied function returns nil. In such case (using the latest implementation from core.async [org.clojure/core.async "0.1.256.0-1bf8cf-alpha"]) one can take nil from a channel created with map<. This is otherwise only possible with closed channels. Putting nil on a channel normally throws an IllegalArgumentException.

With the current implementation of map< it is not possible to determine whether the source-channel was closed or the supplied function returned nil.

Notice that putting onto a channel created with map> throws an IllegalArgumentException when the supplied function returns nil as if you would put nil onto a channel.

My proposal is to add keep</> (where nil values returned from the supplied function are ignored and nothing is put) to the core library or to implement map</> as keep</> since having a real map</> instead of keep</> hardly makes sense when nil is not permitted anyways.



 Comments   
Comment by Leon Grapenthin [ 24/Apr/14 3:44 AM ]

It is still an issue in "0.1.278.0-76b25b-alpha" that you can only use impl.protocols/closed? to consistently determine whether a channel created with map</> was closed - if nil is one of your predicates return values.

Of course, you could use a predicate that never returns nil. But what should be the benefit of map</> being able to magically pass nil while everything else isn't?

Comment by Alex Miller [ 02/Sep/14 9:43 AM ]

All of the transformation functions (like map<) are deprecated and will go away to be replaced with applying transducers to channels.





[ASYNC-84] Issue with (require [clojure.core.async :as async]) Created: 08/Aug/14  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Adam Krieg Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None


 Description   

I was attempting to try out the new transducer stuff with core.async, but was thwarted by namespace issues.

(ns my-ns
(require [clojure.core.async :as async]))

Yields:

java.lang.IllegalArgumentException: No single method: add_BANG_STAR of interface: clojure.core.async.impl.protocols.Buffer found for function: add!* of protocol: Buffer

My dependencies are as follows:

<dependency>
<groupId>org.clojure</groupId>
<artifactId>clojure</artifactId>
<version>1.7.0-alpha1</version>
</dependency>

<dependency>
<groupId>org.clojure</groupId>
<artifactId>core.async</artifactId>
<version>0.1.319.0-6b1aca-alpha</version>
</dependency>



 Comments   
Comment by Ghadi Shayban [ 11/Aug/14 11:30 PM ]

This sounds like a code reloading issue. Have you tried removing your target/ directory and restarting the REPL?

Totally unrelated, but ns declarations take keyword :require instead of symbol.

Comment by Alex Miller [ 02/Sep/14 9:41 AM ]

Can't reproduce





[ASYNC-31] put! does not throw on closed channel Created: 02/Nov/13  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Chris Perkins Assignee: Unassigned
Resolution: Duplicate Votes: 1
Labels: None
Environment:

rev 1bf8cf4



 Description   

The docstring of put! says "Will throw if closed". However, put!-ing to a closed channel appears to succeed, and even calls the supplied callback.

I'm not sure which is incorrect - the behavior or the docstring (or my intepretation of it) - but it does appear that the implementation of mult is expecting an exception from put!, in order to do cleanup.



 Comments   
Comment by Justin Balthrop [ 10/Feb/14 4:53 PM ]

The doc string for put! no longer says "Will throw if closed" for clojure.core.async, though it does still say this for cljs.core.async (and doesn't seem to work).

It looks like Rich started switching put! to return true unless the channel is closed in cf8dc1bf207e646c14b2bf44763737fcb5f08c78. The docstring reflects this change, but from my testing, put! still always returns nil.

Comment by Sung Pae [ 02/Apr/14 3:25 PM ]

Since this is likely a docstring mismatch, I'd like to add that the docstring for clojure.core.async/put states:

Asynchronously puts a val into port, calling fn1 (if supplied) when complete, passing true iff port is already closed.

fn1 is passed `false` when the port is closed, not `true`.

Comment by Alex Miller [ 02/Sep/14 9:39 AM ]

Docstring addressed with ASYNC-89





[ASYNC-89] mistake in core.async put! docstring Created: 02/Sep/14  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Bruce Adams Assignee: Bruce Adams
Resolution: Completed Votes: 0
Labels: None

Attachments: File docstring.diff    
Patch: Code
Approval: Vetted

 Description   

In [1], correct statement should be "passing false iff port is already closed" instead of current "passing true iff port is already closed".

Current behavior is that put callback fn gets true if a channel is not already closed:

user=> (require '[clojure.core.async :as async])
nil
user=> (def c (async/chan 1))
#'user/c
user=> (async/put! c :foo #(println "got " %))
got  true
true
user=> (async/close! c)
nil
user=> (async/put! c :foo #(println "got " %))
got  false
false

From Jozef Wagner in https://groups.google.com/forum/#!topic/clojure-dev/zduW2jMndZ4

[1] https://github.com/clojure/core.async/blob/422f8b25f0c4d5e3aea5113436c735e553422be7/src/main/clojure/clojure/core/async.clj#L140






[ASYNC-87] Add variant of pipe which accepts mapping fn Created: 24/Aug/14  Updated: 02/Sep/14  Resolved: 02/Sep/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Trivial
Reporter: Brandon van Beekum Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

I'd propose something like:

(pipe from to mapping-fn ex-handler)

Currently I am doing the following to patch two channels together with a transform fn:

(defn my-pipe [from to f exh]
(let [map-ch (chan 1 f exh)]
(async/pipe from map-ch)
(async/pipe map-ch to)))



 Comments   
Comment by Alex Miller [ 02/Sep/14 9:16 AM ]

You can do this now with async/pipeline (with concurrent execution, blocking, async, etc):

(async/pipeline 1 to f from)





[ASYNC-85] Add transducer support for ClojureScript Created: 13/Aug/14  Updated: 13/Aug/14  Resolved: 13/Aug/14

Status: Resolved
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Luke VanderHart Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None

Attachments: File cljs-transducers.diff    
Approval: Ok

 Description   

The attached patch adds transducer support, to keep ClojureScript APIs in sync with Clojure's (as far as possible).

Tests pass and the code is correct as far as I can discern but review is appreciated.



 Comments   
Comment by Luke VanderHart [ 13/Aug/14 12:58 PM ]

I pushed this patch directly after getting push access from Rich & Stu.





[ASYNC-80] Fix documentation (or code) - related to "transformer" and deprecated counterparts Created: 28/Jul/14  Updated: 07/Aug/14  Resolved: 07/Aug/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major
Reporter: Max Penet Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

When looking at the documentation now there are a bunch of functions marked as deprecated with the following message:
"Deprecated - this function will be removed. Use transformer instead"

However the `transformer` doesn't exist yet in the currently released version, or master for that matter...
It would be welcome to fix the doc for those who are using the various transformer functions already, even if they are deprecated.






[ASYNC-61] Exceptions thrown inside go/thread blocks propagate up and out of ThreadPoolExcecutor Created: 24/Mar/14  Updated: 06/Aug/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: exceptions
Environment:

core.async 0.1.267.0-0d7780-alpha on JDK 1.7



 Description   

If a go or thread blocks throws an exception, there is nothing in Clojure to catch and handle (or report) the exception. Instead, it propagates up to the ThreadExcecutor which invokes its NO-OP afterExecute() method and is re-thrown, ultimately being displayed on the System.err:

Exception in thread "async-dispatch-32" java.lang.IllegalStateException: Fall down, go boom!
at flashiz.resources.orders$index.invoke(orders.clj:26)
at clojure.lang.Var.invoke(Var.java:411)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.core$apply.invoke(core.clj:617)
at io.aviso.rook$rook_dispatcher.invoke(rook.clj:225)
at flashiz.async$wrap_sync_handler$fn__9005.invoke(async.clj:34)
at flashiz.resources$authorize_async_rook_middleware$fn_9356$fn9402$state_machine3245auto__9403$fn_9405.invoke(resources.clj:21)
at flashiz.resources$authorize_async_rook_middleware$fn_9356$fn9402$state_machine3245auto___9403.invoke(resources.clj:21)
at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:945)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:949)
at clojure.core.async.impl.ioc_macros$take_BANG_$fn__3261.invoke(ioc_macros.clj:958)
at clojure.core.async.impl.channels.ManyToManyChannel$fn__2256.invoke(channels.clj:80)
at clojure.lang.AFn.run(AFn.java:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

It would be nice if execution of the go/thread block was wrapped in an exception handler that delegated to a default function to report the exception. My goal is to be able to alter that function to report it more nicely and/or write it to a persistent log file.



 Comments   
Comment by Alex Miller [ 06/Aug/14 10:30 AM ]

Patches on ASYNC-76 have been applied and exceptions will now flow up to the top of the thread, where they can be caught by the standard Thread uncaught exception handler mechanism or ultimately by the default uncaught exception handler, which can be set for the application. There may still be further changes to support exception handling in thread/go.





[ASYNC-71] exception behavior for thread macro is hard coded Created: 15/May/14  Updated: 06/Aug/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: exceptions
Environment:

0.1.278.0-76b25b-alpha



 Description   

Currently, the thread macro's behavior when an exception occurs is to print the exception (with println), and swallow it.

Although I make a habit of wrapping a try around code inside a thread form, I still find this a bit limited; it would be nice if there was a function that could be dynamically bound, that handled the case of an exception inside a thread.



 Comments   
Comment by Alex Miller [ 06/Aug/14 10:29 AM ]

Patches on ASYNC-76 have been applied and exceptions will now flow up to the top of the thread, where they can be caught by the standard Thread uncaught exception handler mechanism or ultimately by the default uncaught exception handler, which can be set for the application. There may still be further changes to support exception handling in thread/go.





[ASYNC-76] Exceptions cannot be captured by default uncaught exception handler Created: 02/Jul/14  Updated: 06/Aug/14  Resolved: 06/Aug/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Stuart Sierra Assignee: Stuart Sierra
Resolution: Completed Votes: 6
Labels: None

Attachments: Text File 0001-ASYNC-76-do-not-catch-exceptions-on-thread-pool.patch     Text File 0002-ASYNC-76-propagate-exceptions-with-tests.patch    
Patch: Code and Test
Approval: Ok

 Description   

Both core.async thread pools have a try/catch which catches and prints exceptions thrown in callbacks. This prevents application code from using the built-in mechanisms of the JVM to handle exceptions on arbitrary threads, namely Thread.setDefaultUncaughtExceptionHandler

In addition, the pool for 'thread' and 'thread-call' does not close the channel when an exception is thrown, which may lead to deadlocks.

Current Patch: 0002-ASYNC-76-propagate-exceptions-with-tests.patch

Approach: For the callback/go thread pool, do not catch exceptions at all. For the 'thread' pool, remove the 'catch' and close the channel in a 'finally'.

Background: Printing exceptions was added shortly after changing the arity to the 'put!' callback, to aid in debugging, see commit 9fcae995. Prior to this, the 'thread-call' function has always dropped exceptions silently; see commit 0b8e6171.

Duplicate of ASYNC-71 and similar to ASYNC-61, both of which request that the thread pools dispatch errors to a handler function Var which application code can alter.

An earlier issue (no JIRA ticket) occurred when tasks were submitted to the ThreadPoolExecutor with .submit, which returns a Future, instead of .execute.



 Comments   
Comment by Timothy Baldridge [ 03/Jul/14 8:13 AM ]

These threads are created in a fixed size thread pool. Won't you need to set the exception handler on the Executor constructor instead of on the thread itself?

Comment by Stuart Sierra [ 03/Jul/14 8:18 AM ]

No, the default uncaught exception handler is global for the whole JVM. Thread.setDefaultUncaughtExceptionHandler is static.

The exception handler can be overridden on a per-thread or per-thread-group basis.

Comment by Alex Miller [ 06/Aug/14 10:23 AM ]

applied to master





[ASYNC-81] Spurious Closure warnings when using new ClojureScript dependency Created: 01/Aug/14  Updated: 04/Aug/14  Resolved: 04/Aug/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: David Nolen Assignee: Unassigned
Resolution: Completed Votes: 0
Labels: None


 Description   

Closure now emits a warning if core.async generates code with dead branches.



 Comments   
Comment by Ghadi Shayban [ 03/Aug/14 10:10 PM ]

David, do you have a minimal reproducing example?

Comment by David Nolen [ 04/Aug/14 7:57 AM ]

Sorry for not looking into this more closely first. It turns out the pattern of (assert nil "text") is the culprit - it compiles to a JavaScript if statement with null as the test and null as the success statement. We should drop this pattern for a simple throw in my opinion.

Comment by David Nolen [ 04/Aug/14 12:37 PM ]

this is fixed in master https://github.com/clojure/core.async/commit/41262e26b4d2b3070d23f927c25f585d9dc48951. The other dead code warnings from Closure needed to addressed in the ClojureScript compiler itself which is also resolved https://github.com/clojure/clojurescript/commit/8013f0fd0a7803c01efca90e757d316c0a1e1ec0





[ASYNC-82] Channels should accept Metadata Created: 02/Aug/14  Updated: 03/Aug/14  Resolved: 03/Aug/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Alexander Kiel Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None


 Description   

Currently channels do not implement IObj and so do not accept metadata. My use case is no annotate channels with a name so that I can print out its name after an alts!.



 Comments   
Comment by Ghadi Shayban [ 03/Aug/14 10:09 PM ]

For efficiency reasons, channels are plain deftypes, which don't accept metadata.

Since channels are identities, they can be used in a map or set, and you can store the names or info out-of-band. May I suggest the following pattern:

(let [in (chan)
      out (chan)

      names {in :request
             out :response}
      val (rand-int 50)
      [v ch] (alts! [in [out val]])]
  (names ch))




[ASYNC-79] (ClojureScript) go macro not correctly transforming (case) within a macro Created: 23/Jul/14  Updated: 23/Jul/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Tom Locke Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: cljs, macro
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/clojurescript "0.0-2234"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]


Attachments: File core_async_bug.tgz    

 Description   

(case-let) is a macro to handle messages of the form [:message-tag, arg1, arg2, ...] with the arguments bound to local variables. It fails to work correctly when used within a go block. Note that a simple macro with a case, e.g. (defmacro my-case [expr & cases] `(case ~expr ~@cases)) does work.

(Sample project attached)

(case-let) definition:

(ns core-async-bug.macros)

(defmacro case-let [expr & cases]
  (let [msg (gensym)]
    `(let [~msg ~expr]
       (case (first ~msg)
         ~@(apply concat (for [[key [args & body]] (partition 2 cases)]
                    [key `(let [~args (rest ~msg)] ~@body)]))))))

ClojureScript test code:

(ns core-async-bug.core
  (:require-macros [cljs.core.async.macros :refer [go]]
                   [core-async-bug.macros :refer [case-let]])
  (:require [cljs.core.async :refer[<! put! chan]]))

(enable-console-print!)

; go block with manual case + lets - works
(let [c (chan)]
  (go
    (let [msg (<! c)]
      (case (first msg)
        :a (let [[x] (rest msg)] (println "First :a" x))
        :b (let [[y] (rest msg)] (println "First :b" y)))))
  (put! c [:b 123]))

; case-let outside of go - works
(case-let [:b 123]
  :a ([x] (println "Second :a" x))
  :b ([y] (println "Second :b" y)))

; case-let inside go - broken
(let [c (chan)]
  (go
    (case-let (<! c)
      :a ([x] (println "Third :a" x))
      :b ([y] (println "Third :b" y))))
  (put! c [:b 123]))

Browser console output:

Second :b 123
First :b 123
Third :a 123          <-- Should not be there!
Third :b 123


 Comments   
Comment by Tom Locke [ 23/Jul/14 3:46 AM ]

More discussion here: https://groups.google.com/forum/#!topic/clojurescript/w21nNWkKR-c

Comment by Tom Locke [ 23/Jul/14 1:58 PM ]

I've discovered an easy workaround for this problem. During macro-expansion core names like case become fully qualified, i.e. cljs.core/case, and it seems that the go macro then fails to recognise the case as such. Replacing case with ~'case in the definition of let-case fixes the problem.

I would hope this leads to an easy fix for someone who knows the core.async codebase.

The working macro is:

(defmacro case-let [expr & cases]
  (let [msg (gensym)]
    `(let [~msg ~expr]
       (~'case (first ~msg)
         ~@(apply concat (for [[key [args & body]] (partition 2 cases)]
                    [key `(let [~args (rest ~msg)] ~@body)]))))))




[ASYNC-78] deadlock in multi catch Created: 05/Jul/14  Updated: 05/Jul/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Lars Bohl Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

ubuntu 14.04
Leiningen 2.4.2 on Java 1.7.0_60 Java HotSpot(TM) 64-Bit Server VM
[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]



 Description   

The following apparently never returns:

(defn thread-death []
  (<!! (go
        (<!
         (let [ch (chan 1)]
           (>! ch 0)
           (>! ch (try (<! ch)
                       (assert false)
                       (catch Exception _ 5)
                       (catch AssertionError _ 4)))
           ch)))))

After removing the (catch Exception _ 5) catch block it returns 4 as expected.






[ASYNC-77] StackOverflowError in go macro with cemerick.cljs.test (CLJS) Created: 04/Jul/14  Updated: 04/Jul/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Daniel Skarda Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.6.0"]
[org.clojure/core.async "0.1.303.0-886421-alpha"]
[org.clojure/clojurescript "0.0-2261"]
[com.cemerick/clojurescript.test "0.3.1"]



 Description   

I got an StackOverflowError during compilation of CLJS tests using clojurescript.test from Chas Emerick.

I narrowed the problem to following code:

(ns async.overflow.cljs
  (:require-macros [cljs.core.async.macros :refer [go]]
                   [cemerick.cljs.test :refer [deftest is]])
  (:require [cemerick.cljs.test]))

(deftest foobar
  (go
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))
    (is (= 1 1))))

If you do not get the exception, try to increase number of repetitions of 'is' call.

I was not able to replicate the issue in CLJ. cemerick.cljs.test/is macro is probably far more complex than original CLJ version in clojure.test/is.






[ASYNC-74] Provide a chan? function to determine if a given var is a channel Created: 23/Jun/14  Updated: 30/Jun/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Major
Reporter: Reno Reckling Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None

Attachments: File drupp-async-74.diff    

 Description   

When using channels, for example for streaming http bodies, it would be really helpful if there would be a chan? function to check whether a given var is a channel or not.

Aleph already does it like that in it's http-server/client implementation using lamina channels and it feels quite consistent.

It would of course also be more consistent with the rest of clojure to provide a type checking function for basic objects.



 Comments   
Comment by David Rupp [ 30/Jun/14 9:58 AM ]

Implement chan? predicate.

Comment by Timothy Baldridge [ 30/Jun/14 10:50 AM ]

I'm in favor of this, but last time I asked Rich about it his quote was "do you want a predicate for every single interface?".

Due to the implementation of core.async you'd probably need two additional predicates. One for read-port? and write-port?. You can use (satisfies? clojure.core.async.impl.protocols/ReadPort ...) but that's an internal implementation, so I'd rather have a new predicate than to tell people to touch the innards of core.async. But this call is up to Rich.

Comment by Reno Reckling [ 30/Jun/14 11:07 AM ]

I understand that reasoning. But then we would have to step up on documentation and provide a way to easily determine which interfaces are implemented by the return values of for example (chan). For me, going into the innards of core.async and trying to determine which interfaces I want, especially on very basic things like a channel, is not a user friendly approach at all.





[ASYNC-75] alts!! not working on a channel which is subscribed to a pub-channel Created: 23/Jun/14  Updated: 24/Jun/14  Resolved: 24/Jun/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Daniel Ziltener Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None
Environment:

Clojure 1.6.0, core.async 0.1.303.0-886421-alpha



 Description   

Error message: Exception in thread "main" java.lang.UnsupportedOperationException: count not supported on this type: ManyToManyChannel

Minimal failing case:

(require '[clojure.core.async :refer (chan pub sub put! alts!!)])
(def out (chan))
(def publisher (pub out #(:topic %)))
(def s (chan))
(sub publisher :1 s)

(put! out {:topic :1 :content "Yes!"})
(alts!! s :default "Oh no.")


 Comments   
Comment by Alex Miller [ 24/Jun/14 6:51 AM ]

Your use of alts!! has a bug - the ports passed to it is expected to be a vector, not a single operation:

(alts!! [s] :default "Oh no.")





[ASYNC-53] Definition of UnblockingBuffer Protocol makes core.async incompatible with clojure 1.4 Created: 03/Feb/14  Updated: 23/Jun/14  Resolved: 23/Jun/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Minor
Reporter: Kuldeep Assignee: Unassigned
Resolution: Declined Votes: 0
Labels: None

Attachments: Text File 0001-Compatibility-with-1.4.patch    
Patch: Code

 Description   

An empty protocol could not be compiled with clojure 1.4. I have attached a patch to show the problem locations.

Thanks



 Comments   
Comment by Alex Miller [ 23/Jun/14 8:53 PM ]

core.async is only intended to be compatible with Clojure 1.5.1+.





[ASYNC-54] MAX-QUEUE-SIZE has a wrong type-hint Created: 11/Feb/14  Updated: 23/Jun/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Trivial
Reporter: Nicola Mometto Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File 0001-Fix-MAX-QUEUE-SIZE-type-hint-def-evaluates-the-metad.patch    

 Description   

Since `def` evaluated the metadata on the symbol by design, primitive type hints needs to be quoted to avoid resolving to the homonimous clojure.core function



 Comments   
Comment by Alex Miller [ 23/Jun/14 3:22 PM ]

Actually, I think no hint is needed here at all and it should just be removed. Clojure will just inline a primitive long for the MAX-QUEUE-SIZE where it is used.





[ASYNC-28] try/catch inside go block may cause a reflection warning - 0.1.222.0-83d0c2-alpha Created: 08/Oct/13  Updated: 23/Jun/14  Resolved: 23/Jun/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None


 Description   

I have this code:

(go
  (try
     ...
    (catch Throwable t
          (l/errorf t "Acceptor %s exception: %s" channel-key (.getMessage t))
          (ctx/trigger-error-handlers t))))

I get a reflection warning: reference to field getMessage can't be resolved.

This does not occur if the same code is moved outside the go block.

My assumption is that an implicit type hint (t is of type Throwable) is lost in the process of converting the block to a state machine.

Adding an explicit type hint does not fix the problem.

Further, the warning is associated with the line containing the go symbol, not the catch symbol. See ASYNC-27.



 Comments   
Comment by Alex Miller [ 23/Jun/14 11:04 AM ]

I tried this on the latest (0.1.303.0-886421-alpha) and saw no reflection warnings.

(require '[clojure.core.async :refer (go)])
(set! *warn-on-reflection* true)
(go 
  (try 1 
    (catch Throwable t (println "Got ex: " (.getMessage t)))))

If you still see this, please provide an example that is reproducible, thanks!





[ASYNC-62] recur across try possible, leaks ExceptionFrames Created: 27/Mar/14  Updated: 23/Jun/14  Resolved: 23/Jun/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Philip Lewis Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None
Environment:

[org.clojure/clojure "1.5.1"]
[org.clojure/core.async "0.1.278.0-76b25b-alpha"]



 Description   

This shouldn't compile, but it does and generates nested ExceptionFrames until it crashes:

(go-loop [] (try (if 1 (recur))))

Without the if, it throws java.lang.IllegalArgumentException: No implementation of method: :emit-instruction of protocol: #'clojure.core.async.impl.ioc-macros/IEmittableInstruction found for class: clojure.core.async.impl.ioc_macros.Jmp.

Outside of a go block it throws an UnsupportedOperationException (either "can only recur from tail" or "cannot recur across try").



 Comments   
Comment by Alex Miller [ 23/Jun/14 10:59 AM ]

On latest CLJ version (0.1.303.0-886421-alpha), I see "java.lang.UnsupportedOperationException: Can only recur from tail position" from this code.





[ASYNC-58] mult channel deadlocks when untapping a consuming channel whilst messages are being queued/blocked Created: 20/Feb/14  Updated: 23/Jun/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Mathieu Gauthron Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: deadlock, mult, untap
Environment:

Mac 10.7.5; java version "1.7.0_40"; [org.clojure/clojure "1.5.1"]; [org.clojure/core.async "0.1.267.0-0d7780-alpha"]; Tested with cider and emacs 24.3


Approval: Triaged

 Description   

I have two (or more) listeners tapped onto a mult channel. I want to use them all then have one (or more) of them to leave at will without blocking the other consumer(s) or the publisher. Initially they work fine until one of them wants to stop listening. I thought the listener which drops out needs to (be a good citizen and) untap its channel from mult (otherwise a deadlock is systematic). However if messages are put into the mult before the leaving listener has had a chance to untap its channel, it creates a deadlock on the main thread (which is putting more messages simultaneously). I do not find a way to guarantee that I can untap the channel in time to avoid this race condition.

Once I have reproduced the deadlock, the repl is frozen until I interrupt with ctrl-c.
I have also tried to close the tapped channel before untapping it but the result was the same.

In the following snippet, the last (println "I'm done. You will never see this") is never reached. The publisher and the remaining consumer (consumer 1) are deadlocked even though consumer 2 was trying to leave in good terms.

(require '[clojure.core.async :refer (chan go <! >!! mult tap untap)])
(let [to-mult (chan 1)
      m (mult to-mult)]

  ;;consumer 1
  (let [c (chan 1)]
    (tap m c)
    (go (loop []
          (when-let [v (<! c)]
            (println "1 Got! " v)
            (recur))
          (println "1 Exiting!"))))

  ;;consumer 2
  (let [c (chan 1)]
    (tap m c)
    (go (loop []
          (when-let [v (<! c)]
            (when (= v 42)  ;; exit when value is not 42
              (println "2 Got! " v)
              (recur)))
          (println "2 about to leave!")
          (Thread/sleep 5000) ;; wait a bit to exacerbate the race condition
          (untap m c) ;; before unsubscribing this reader
          (println "2 Exiting."))))

   (println "about to put a few messages that work")
   (doseq [a (range 10)]
     (>!! to-mult 42))
   (println "about to put a message that will force the exit of 2")
   (>!! to-mult 43)
   (println "about to put a few more messages before reader 2 is unsubscribed to show the deadlock")
   (doseq [a (range 10)]
     (println "putting msg" a)
     (>!! to-mult 42))
   (println "I'm done. You will never see this"))
about to put a few messages that work
2 Got!  42
1 Got!  42
2 Got!  42
1 Got!  42
1 Got!  42
2 Got!  42
1 Got!  42
1 Got!  42
2 Got!  42
2 Got!  42
2 Got!  42
2 Got!  1 Got!  42
422 Got!  42

1 Got!  42
1 Got!  42
2 Got!  42
1 Got!  42
about to put a message that will force the exit of 2
1 Got!  42
2 Got!  about to put a few more messages before reader 2 is unsubscribed to show the deadlock
42
putting msg 1 Got!  0
2 about to leave!
43
1 Got!  42
putting msg 1
putting msg 2
putting msg 3
1 Got!  42
2 Exiting.


 Comments   
Comment by Ghadi Shayban [ 22/Apr/14 10:18 AM ]

Mathieu, this is probably expected. It's important to note that to guarantee correct ordering/flow when using a mult, you should enforce it on the source/producer side of the mult, and not asynchronously on the tap side.

Mult will deref a stable set taps just before distributing a value to them, and does not adjust dynamically during value distribution except when a tap has been closed [1]. If you would like to stably untap without closing the tap you can/should let the 'producer' do it in an ordered fashion in between values on the input channel.

Knowing that a put occurred to a closed channel is new on release 0.1.278.

In general, walking away on the consuming side of a channel is tricky. Depending on the semantics of your processes, if the producer side of a channel isn't aware that a close! can happen from the consumer side, you might have to launch a draining operation.

(defn drain [c] (go (when (some? (<! c)) (recur))))

Golang disallows closing a read-only channel FWIW [2]

Better documentation is probably warranted.

[1] https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async.clj#L680-L682
[2] http://golang.org/ref/spec#Close





[ASYNC-25] Dependency on clojurescript is introduced into clojure only projects Created: 05/Oct/13  Updated: 23/Jun/14  Resolved: 23/Jun/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Hugo Duncan Assignee: Unassigned
Resolution: Not Reproducible Votes: 0
Labels: None


 Description   

When using core.async in a clojure only project, a dependency on Clojurescript is introduced.

Consider making the clojurescript dependency have "provided" scope, or moving it to a :dev profile.



 Comments   
Comment by Alex Miller [ 23/Jun/14 10:05 AM ]

Looks like this was already done - seems to be marked provided in pom and does not show up as transitive dependency in using project.





[ASYNC-4] Record literals become ordinary maps Created: 15/Jul/13  Updated: 23/Jun/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Alex Miller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

CLJS


Approval: Triaged

 Description   
(require '[clojure.core.async :refer (<!! go)])
(defrecord Foo [x])
(def f (Foo. 4))
(<!! (go f))
;; => #user.Foo{:x 4}
;; OK

(<!! (go #clojure.core.async.Foo{:x 4}))
;; CLJ: => #user.Foo{:x 4}   ;; expected
;; CLJS => {:x 4}            ;; wrong

Approach: Query the analyzer to know if we have a record or not.

(Copied from GitHub issue #13 - https://github.com/clojure/core.async/issues/13)



 Comments   
Comment by Ghadi Shayban [ 03/Aug/13 2:08 PM ]

0c6e663493 contains a fix on the Clojure side, would appreciate help porting to cljs.

Comment by David Nolen [ 08/Aug/13 8:35 AM ]

I think on the ClojureScript we'll have to query the analyzer to know if we have a record or not.





[ASYNC-73] try-catch-finally broken inside go-blocks Created: 21/Jun/14  Updated: 21/Jun/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Moritz Ulrich Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

clojure 1.6.0, clojurescript 0.0-2234, core.async 0.1.303.0-886421-alpha1



 Description   

The following will cause the failed assertion (ioc_helpers.cljs:155)
when evaluated as a whole, and will correctly catch the Error when
just the `try' gets evaluated. The `finally' block runs only if the
inner block is evaluated:

(go
  (try
    (throw (js/Error. "asdf"))
    (catch ExceptionInfo e
      (println "ExceptionInfo"))
    (catch js/Error e
      (println "js/Error"))
    (finally
      (println "finally"))))

Another notable observation is that changing the order of the `catch'
blocks will change the behavior: If the (catch js/Error ...) is the
first catch block, it will work just as expected.






[ASYNC-69] How to better communicate "mix" lifecycle wrt coordination Created: 09/May/14  Updated: 09/May/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Timothy Baldridge Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Approval: Vetted

 Description   

Confusion regarding 'mix' lifecycle. Users have a false expectation that a mix's output should magically close when its source channels close, not realizing its a dynamic coordination process. Actually ensuring correctness when using mult/mix/pub isn't so straightforward, people need to control consistency from the producer end, and not ad hoc from the consumer side (e.g. late taps of a mult).






[ASYNC-66] Add drain! to consume and discard a channel Created: 09/May/14  Updated: 09/May/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Enhancement Priority: Minor
Reporter: Ghadi Shayban Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Approval: Vetted

 Description   

Do we want this for consumer walkaway scenarios or something baked into the channel impl?

(defn drain!
 "Fully consumes and discards a channel. Returns a channel that
  closes when ch is fully drained"
 [ch]
 (go-loop []
   (if (some? (<! ch))
     (recur))))





[ASYNC-48] Recur fails within catch Created: 21/Dec/13  Updated: 09/May/14  Resolved: 09/May/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Gerrit Jansen van Vuuren Assignee: Timothy Baldridge
Resolution: Declined Votes: 0
Labels: bug


 Description   

This is related to
http://dev.clojure.org/jira/browse/ASYNC-7

Having a loop in a go block and the recur inside a catch statement causes an IllegalArgumentException.

The code is:

(go
(loop [i 0]
(try
(do
(+ 1 1))
(catch Exception e (do
(prn e)
(recur (inc i)))))))

;; IllegalArgumentException No implementation of method: :emit-instruction of protocol: #'clojure.core.async.impl.ioc-
;; macros/IEmittableInstruction found for class: clojure.core.async.impl.ioc_macros.Jmp clojure.core/-cache-protocol-fn
;;(core_deftype.clj:541)



 Comments   
Comment by Leon Grapenthin [ 25/Dec/13 9:14 AM ]

This doesn't work outside of a go block either.

Comment by Timothy Baldridge [ 09/May/14 11:30 AM ]

Closing due to example not being valid code.





[ASYNC-21] CLJS -> ioc-macros/fixup-aliasses throws exception Created: 02/Sep/13  Updated: 09/May/14  Resolved: 09/May/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Creighton Kirkendall Assignee: Timothy Baldridge
Resolution: Declined Votes: 0
Labels: None

Attachments: Text File 0001-added-a-check-to-fixup-aliases-for-namespace-to-stop.patch    
Patch: Code

 Description   

Under certain circumstances I get the following exception when using a 'go' block.

Caused by: java.lang.ClassCastException: clojure.lang.Namespace cannot be cast to clojure.lang.Named
             core.clj:1505 clojure.core/name
        ioc_macros.clj:613 cljs.core.async.impl.ioc-macros/fixup-aliases
        ioc_macros.clj:625 cljs.core.async.impl.ioc-macros/eval4971[fn]
        ioc_macros.clj:110 cljs.core.async.impl.ioc-macros/all[fn]

This is caused by the calling (name ns) where ns is namespace instead of calling (ns-name) first. The exception is thrown by the 'go' block in 'basic-test' here: https://gist.github.com/ckirkendall/6415293



 Comments   
Comment by Ghadi Shayban [ 18/Nov/13 11:12 PM ]

Creighton, do you still encounter the issue against release 0.1.256.0-1bf8cf-alpha?

Comment by Timothy Baldridge [ 09/May/14 11:29 AM ]

Closed due to no further information.





[ASYNC-8] Add constrained read/write-only ports Created: 15/Jul/13  Updated: 09/May/14  Resolved: 09/May/14

Status: Closed
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Brandon Bloom Assignee: Timothy Baldridge
Resolution: Declined Votes: 0
Labels: None


 Description   

Proof of concept patch here:

https://github.com/clojure/core.async/pull/20/files

As discussed in IRC, this adds support for read-only and write-only "ports", which are constrained wrappers around channels. Write-only ports allow both put and close operations, were as read-only ports allow only take operations. Unsupported operations throw.

I'm open to suggestions for better names than <port and >port.



 Comments   
Comment by Timothy Baldridge [ 09/May/14 11:28 AM ]

Do we do not have plans to implement this at this time, however, impl/ReadPort and impl/WritePort could accomplish this if needed. These interfaces are still considered "alpha" and may change, but if you need to write your own port< feel free to do so and maintain it in your project.





[ASYNC-27] Compilation errors inside go block always reported as first line of block Created: 08/Oct/13  Updated: 09/May/14

Status: Open
Project: core.async
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Defect Priority: Major
Reporter: Howard Lewis Ship Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: errormsgs


 Description   

I've noticed that when there are any errors inside a go block, the line number of the error is always the line containing the go symbol.

I suspect that some meta data on the forms that are converted into a state machine is being lost in the process.

This is quite annoying and quite leaky (in the abstraction sense). It makes it that much harder to track down the source of errors.



 Comments   
Comment by